Running AI locally
Running AI locally
This module is hands-on, and it dates faster than the others. It covers the hardware, where models are published and how to read their names, the software that runs them, agents and the harnesses that drive them, and building on a local model. It names products throughout and says when each table was written. It explains what the tools do and leaves the commands to their own documentation. It leans on part 3 of the language models module, and can be read without it. The whole module is about two and a half hours of reading.
Part 1: What you need
After it you can: Look at a machine and a model and say whether it will run, and roughly how fast
30 minutes
Part 2: Finding and reading models
After it you can: Read a model’s name as a specification, and choose one for your memory, your job and your licence
30 minutes
Part 3: Running a model
After it you can: Choose an engine and an app, and set the three settings that decide whether a local model works
30 minutes
Part 4: Agents on local models
After it you can: Pick a harness and a task a local model can handle, and keep the agent contained
30 minutes
Part 5: Building on it
After it you can: Build on a local model from your own code, and know when a hosted one is the better tool
25 minutes