Running AI locally

Running AI locally

  • Five parts · about two and a half hours

This module is hands-on, and it dates faster than the others. It covers the hardware, where models are published and how to read their names, the software that runs them, agents and the harnesses that drive them, and building on a local model. It names products throughout and says when each table was written. It explains what the tools do and leaves the commands to their own documentation. It leans on part 3 of the language models module, and can be read without it. The whole module is about two and a half hours of reading.

  1. Part 1: What you need

    After it you can: Look at a machine and a model and say whether it will run, and roughly how fast

    30 minutes

  2. Part 2: Finding and reading models

    After it you can: Read a model’s name as a specification, and choose one for your memory, your job and your licence

    30 minutes

  3. Part 3: Running a model

    After it you can: Choose an engine and an app, and set the three settings that decide whether a local model works

    30 minutes

  4. Part 4: Agents on local models

    After it you can: Pick a harness and a task a local model can handle, and keep the agent contained

    30 minutes

  5. Part 5: Building on it

    After it you can: Build on a local model from your own code, and know when a hosted one is the better tool

    25 minutes