Finding and Reading Models: Hugging Face and What a Name Tells You

011. Hugging Face

The Hugging Face Hub hosts millions of models. When a lab releases open weights, this is where they appear, usually within minutes of the announcement, and the tools in part 3 download from it directly.

What is on a model’s page

• The model card is the description, written by the publisher. A good one states what the model is for, its size, its context length, how it was trained, how it performs, its known limits and how to run it. A thin or absent card is a warning.
• The files. The weights, usually as several large files, alongside small configuration files that say how the model is built and how its prompts must be formatted. Part 3 explains why that last file matters.
• The licence, shown as a tag at the top. Section 4 covers what to look for.
• Signals of use: downloads in the last month, likes, and a discussion tab where problems are reported. High numbers do not prove quality. They do mean that faults have probably been found and talked about.
• The family tree. The page links to the model this one was derived from, and to everything derived from it: fine-tunes, merges and, most usefully, quantised versions.

Originals and conversions

A lab publishes a model once, at full precision, in a format meant for training and for servers. Almost nobody runs that file locally. What you run is a conversion: the same model compressed to 4 or 5 bits and repackaged for the software in part 3.

Conversions are published separately, sometimes by the lab itself and more often by a handful of well-known community publishers who convert every notable model within hours of its release. So a single model appears on the Hub many times over: the original, and then a dozen repositories of conversions, each holding a dozen files at different compression levels. This is the main reason the Hub looks bewildering, and section 3 explains how to read those files.

Who to trust

Anyone can upload anything under any name, and names are easy to imitate.

• Prefer the lab’s own organisation page for originals, and check that it is the verified one.
• For conversions, prefer the established publishers, which you will recognise after a week because every guide and tool points to the same few.
• Prefer the safetensors and GGUF formats. Both hold only numbers. Older formats could run code when loaded, as the generative media module explains, and there is no longer any reason to accept one.
• Be wary of a model with a famous name, no card and no history.

Gated models

Some publishers require you to log in and accept terms before downloading, and a few review requests by hand. The page says so. A gated model needs an access token in whatever tool downloads it, which the tools in part 3 all support. Read what you are agreeing to: it is the licence, and section 4 explains why it is not always what “open” suggests.

022. Reading a name

Take four names that were current in September 2026.

Qwen3.6-35B-A3B
gemma-4-E4B-it
gpt-oss-20b
Mistral-Small-4-119B-2603

Family and version

Qwen3.6, gemma-4, gpt-oss, Mistral-Small-4. The family tells you the publisher and the lineage, and the version tells you the generation. Part 2 of the language models module explains why generation matters more than size: a recent small model regularly beats a large one from two years earlier. When in doubt, take the newer generation.

Family names carry tier words too, and they mislead. Small, Medium and Large, or Flash and Pro, describe a publisher’s own range and not any absolute size. The fourth example is a “Small” of 119 billion parameters, which needs about 68 GB at 4 bits. Read the number, not the word.

The size: 27B, 8B, 120b

The number followed by B is the parameter count in billions. It is the single most useful fact in the name, because part 1’s sum turns it directly into memory: at the usual compression, a little over half a gigabyte per billion.

A few publishers leave it out, and then the card has it. Very large models use T for trillions.

Active parameters: A3B

A second number prefixed with A marks a mixture-of-experts model, and gives the parameters active for each token. 35B-A3B has 35 billion parameters and uses 3 billion at a time.

Part 1’s deep dive explains what that means, and it is worth repeating because it is the commonest misreading on the Hub. The first number sets the memory. The second sets the speed. A 35B-A3B needs the 20 GB of any 35 billion parameter model, and then runs about as fast as a 3 billion parameter one. It is not a 3 billion parameter model that fits in 2 GB.

Older mixture-of-experts names used a different form, such as 8x7B for eight experts of seven billion each. Such a model holds rather less than the product suggests, since the experts share some layers, and the convention has largely been replaced by stating total and active sizes.

On quality, a mixture-of-experts model generally performs below a dense model of the same total size and well above a dense model of its active size. A 35B-A3B is not as capable as a dense 35 billion parameter model, and is much more capable than a dense 3 billion one.

Effective parameters: E4B

A newer prefix, E, stands for effective. One current family of small models keeps a large table of per-layer lookup values that can sit in slow storage and be fetched as needed, and need not occupy fast memory. Its E4B model has 8 billion parameters in all and behaves, in memory and speed, like one of about 4.5 billion. The E number is the one to use in part 1’s sum. These models are aimed at phones and laptops.

What it was trained to do

Many recent models are hybrids that think or not according to a setting, with no suffix. The card says how to switch.

Dates and revisions

A four-digit number such as 2603 or 2507 is a date, year then month: March 2026, July 2025. Some publishers use it in place of a version number and reissue a model under the same name with a new date. When two files differ only in this, take the later.

033. Reading the rest: formats and quantisation

Qwen3.6-27B-Q4_K_M.gguf
gemma-4-26B-A4B-it-qat-q4_0
Qwen3.6-27B-FP8
gpt-oss-20b-MLX-8bit

Formats

For a laptop or a desktop, you are looking for GGUF, or MLX on a Mac. The others matter when you serve a team.

Quantisation codes

Part 3 of the language models module explains quantisation: storing each weight in fewer bits. GGUF files carry a code that says how.

Q4_K_M reads as follows. Q4 is the nominal bits per weight. K names the method, the current standard, which quantises weights in small blocks that each carry their own scale. The final letter is a size variant, S, M or L for small, medium and large: the larger variants keep the most sensitive parts of the model at higher precision. So a Q4_K_M file averages closer to 4.8 bits than to 4.

Codes beginning IQ use a newer method that calibrates against sample text to decide which weights matter most. They give better quality for a given size, particularly below 4 bits.

Choosing a level

• Start at Q4_K_M. It is the default for good reason.
• If you have room to spare, go up, to 5 or 6 bits. Beyond 6 you are spending memory on nothing you will notice.
• Below 4 bits, prefer a smaller model at 4 bits. A 14 billion parameter model at Q4 is usually better than a 27 billion one at Q2, and faster.
• Large models tolerate more. A 70 billion parameter model at 3 bits remains useful. An 8 billion parameter one does not.
• Reasoning, code and long context suffer first, as the language models module notes. For those, stay at 5 bits or above if you can.
• The context needs memory too. Part 1 explained that the weights should leave a quarter to a third of memory free. A smaller file with a usable context beats a larger one without.

Models made for four bits

Two recent developments blur the table above, both for the better.

Some publishers now release models trained to be quantised. qat, for quantisation-aware training, means the model was taught during training to work at low precision, so its 4-bit version loses far less than an ordinary conversion would. Where a qat version exists, prefer it.

Others publish at 4 bits in the first place. MXFP4 and NVFP4 are 4-bit number formats that some recent models are trained and released in, so that there is no higher-precision original to lose quality from. One widely used open model of 20 billion parameters ships this way and occupies about 12 GB as published.

044. Licences

Four points are worth knowing.

• The local-sized models are, at the moment, unusually permissive. A year or two ago the best small models carried restrictive licences. In 2026 nearly every leading family offers its small and mid-sized models under Apache 2.0 or MIT. Check anyway: it has changed before and will change again.
• A fine-tune inherits its parent’s licence and may add its own. A conversion carries the licence of what it converts.
• The licence covers the weights, not what you generate. Some licences do restrict using outputs to train competing models.
• The training data is a separate question, which part 3 of the AI in the organisation module covers, and a permissive licence on the weights says nothing about it.

055. Choosing a model

The method

1. Set the memory budget from part 1: what your machine has, less a quarter to a third for the context and everything else.
2. That gives a size. At 4 to 5 bits: 8 GB of memory takes up to about 9 billion parameters, 16 GB about 14 billion, 24 GB about 27 to 32 billion, 32 GB about 35 billion with a long context, 64 GB about 70 billion, and 128 GB about 120 billion. A mixture-of-experts model may go larger than this on a unified-memory machine, since it stays fast.
3. Shortlist the current generation of two or three families at that size. Rankings of local models are published monthly, and the tools in part 3 show what is popular.
4. Match it to the job. A coding model for code. A model that accepts images if you need that. A thinking model for hard problems where you can wait. Check the context length on the card against what part 3 says your work needs.
5. Test on your own tasks. Part 2 of the language models module explains why public benchmarks are a poor guide, and its method applies: a handful of real examples from your work, tried on each candidate. Locally, this is free.

The field in September 2026

The last row is worth a moment. Several of the strongest open models in 2026 have between 280 billion and 2.4 trillion parameters. Their weights can be downloaded by anyone and run by almost no one. When a model is described as open, ask whether it is open in the sense that you could run it. Some tools now list such models beside local ones and quietly run them on the vendor’s servers, which is convenient, and is not local.

The whiteboard version

Models live on Hugging Face, where anyone can publish, so download originals from the lab’s own page and conversions from the well-known publishers, in safetensors or GGUF. A name is a specification. The number before B is parameters in billions, and sets the memory. A number after A is the active parameters of a mixture-of-experts model, and sets the speed. E means effective parameters. it or instruct is the one you want, and base is not. GGUF is the format for most local software and MLX for Macs. Q4_K_M means about 4.8 bits per weight, the usual choice: go higher if there is room, and below 4 bits choose a smaller model. qat and models released at 4 bits lose least. Check the licence of the exact file. Then work out what fits, shortlist the current generation at that size, and test on your own work.