Generative media

Generative media

  • Seven parts · about four hours

This module covers models that generate images, video, music and 3D objects. Part 1 explains the machine they share, and the second design that competes with it. Parts 2 to 5 take each medium in turn. Part 6 covers how models are adapted with fine-tunes, LoRAs, control inputs and style transfer, where to run them, and the questions of rights and provenance. Part 7 covers how to prompt them. You can read this module without the first. The whole module is about four hours of reading.

  1. Part 1: How generative media models work

    After it you can: Explain how a model turns noise and a prompt into a picture, a clip or a song, and how the token design differs

    45 minutes

  2. Part 2: Images

    After it you can: Say what image models do reliably, what still goes wrong and why

    15 minutes

  3. Part 3: Video

    After it you can: Plan a generated sequence as shots, and say why clips are short and what breaks in them

    30 minutes

  4. Part 4: Music and audio

    After it you can: Tell the two designs of music model apart by ear and by behaviour, and say what you may do with a generated song

    30 minutes

  5. Part 5: 3D

    After it you can: Judge whether a generated 3D object is usable for a job, and what it will cost to make it so

    25 minutes

  6. Part 6: Shaping models

    After it you can: Choose between a base model, a fine-tune, a LoRA and starting from existing media for a job

    50 minutes

  7. Part 7: Prompting

    After it you can: Write prompts that suit the model in front of you, and improve them with purpose

    45 minutes