Guide · snapshot from July 2026

Which AI models actually
run on your machine?

This guide explains, in plain language, which AI models run directly on a computer or a small server — no cloud required. The point: your data never leaves your machine. For a professional, that means a client file, a statement or a meeting recording stays private. Prices in CAD, current as of July 2026 — this field moves fast.

In short

As of July 2026, useful local AI models range from 2.4 GB (Qwen3.5-4B) to ~81 GB (Qwen3.5-122B): a MacBook Pro M5 Pro 48 GB comfortably runs the recommended 35B MoE class (Ornith 1.0, Qwen3.5-35B-A3B), an RTX 4090 24 GB PC gets there in Q4, and transcription (Cohere Transcribe, ~5 GB) fits almost anywhere. Available memory decides everything.

The key concept: memory decides everything

An AI model must fit entirely in the machine's fast memory. Think of it as a desk: if the document doesn't fit on it, everything slows down.

On Windows, that memory is the graphics card's VRAM — separate and limited (12, 16, 24 or 32 GB). If the model doesn't fully fit, performance drops 10 to 30×.
On Mac, memory is unified: CPU and GPU share one pool. A 64 GB Mac can devote nearly all of it to the model. A major structural advantage.
On small servers like the DGX Spark: unified memory too (128 GB), so very large models fit.

Two useful terms:

Quantization (Q4, Q5, Q8): compressing a model so it takes less space. Q4 is the standard (good quality/size balance); Q8 is more faithful but twice as heavy.
MoE (mixture of experts): the model holds a lot of knowledge but only activates a small part per word — big-model quality at small-model speed. Careful: the full model still has to fit in memory. MoE saves compute, not memory.

The models covered

Ornith 1.0 — the agentic-coding specialist (DeepReinforce, MIT)

Released June 25, 2026. Built for agentic coding: it doesn't just write code, it plans, runs tools, tests and fixes its own mistakes. Field note: on a MacBook Pro M5 Max, Ornith 1.0 35B beats Qwen3.6 at coding and agentic workflows — the value of specialization. MIT license, unrestricted commercial use. Available as GGUF (Ollama, LM Studio).

Model	Q4 memory	SWE-Bench	Who it's for
Ornith-1.0-9B	~6 GB	69.4	Entry point, modest machines
Ornith-1.0-31B	~20 GB	—	Dense, stable
Ornith-1.0-35B MoE	~25 GB	75.6	Recommended for most
Ornith-1.0-397B MoE	~200 GB FP8	82.4	Servers, production

Qwen3.5 / Qwen3.6 — the multimodal generalist (Alibaba, Apache 2.0)

Excellent in French, multimodal, 262,000-token context. Where Ornith wins at code, Qwen is unbeatable at language: summaries, document Q&A, contract analysis, drafting. Qwen3.6 has the same memory footprint as 3.5 — a drop-in replacement.

Model	Q4 memory	Who it's for
Qwen3.5/3.6-4B	~2.4 GB	Very modest machines, simple tasks
Qwen3.5/3.6-9B	~5.5 GB	A serious entry point
Qwen3.5/3.6-27B	~16.5 GB	Excellent all-rounder, fits in 24 GB
Qwen3.5/3.6-35B-A3B MoE	~21-22 GB	The star: 35B quality, 3B speed
Qwen3.5/3.6-122B-A10B MoE	~74-81 GB	Frontier class, beats GPT-5 mini at tool use
Qwen3.5/3.6-397B MoE	~242 GB	Very powerful servers

Qwen for: documents, files, drafting, summaries. Ornith for: code, automation, agents, scripts.

Cohere Transcribe — speech to text (Apache 2.0)

#1 on Hugging Face's Open ASR Leaderboard: 5.42% error rate vs 7.44% for Whisper Large v3. 14 languages including French, ~5 GB, runs on almost any recent hardware, ~525 minutes of audio transcribed per minute. Known limits: no speaker identification, no native timestamps, and it's trained mostly on European French — test it on your own Québécois recordings, especially if you mix French and English in one sentence.

The classic safe bets

Llama 3.3 70B (Meta): ~40-43 GB at Q4. A proven generalist with the largest ecosystem. Needs two GPUs or a big-memory Mac.
Mistral Small 24B (Mistral): ~13-14 GB at Q4. Excellent in French, fits in 16 GB.

Mac — unified memory, the big advantage

✅ works well · 🟡 works but tight or slow · ❌ does not fit. Apple's native MLX format is ~10-15% leaner than GGUF and often 15-30% faster.

MacBook Air M4 (fanless — ideal for transcription and small models)

Model	16 GB	32 GB
Cohere Transcribe	✅	✅
Qwen3.5-9B / Ornith-1.0-9B	✅	✅
Qwen3.5-27B	❌	✅
Mistral Small 24B	🟡	✅
Qwen3.5-35B-A3B / Ornith-1.0-35B	❌	🟡
Llama 3.3 70B	❌	❌

MacBook Pro M5 Pro

Model	24 GB	48 GB
Cohere Transcribe	✅	✅
Qwen3.5-27B	✅	✅
Qwen3.5-35B-A3B MoE	✅	✅
Ornith-1.0-35B MoE	🟡	✅
Llama 3.3 70B (Q4)	❌	🟡
Qwen3.5-122B-A10B	❌	❌

MacBook Pro M5 Max

Model	48 GB	64 GB	128 GB
Cohere Transcribe	✅	✅	✅
Qwen3.5-35B-A3B	✅	✅	✅
Ornith-1.0-35B MoE	✅	✅	✅
Llama 3.3 70B (Q4)	🟡	✅	✅
Qwen3.5-122B-A10B (~74-81 GB)	❌	❌	✅
Qwen3.5-397B	❌	❌	❌

The M5 Max at 128 GB is the only laptop that can load Qwen3.5-122B-A10B — a frontier-class model that beats GPT-5 mini by 30% at tool use.

Mac mini M4 / M4 Pro and Mac Studio

Model	mini M4 (16-32 GB)	mini M4 Pro (24-64 GB)	Studio M4 Max (128 GB)	Studio Ultra (192-256 GB)
Cohere Transcribe	✅	✅	✅	✅
Qwen3.5-27B	🟡 (32 GB)	✅	✅	✅
Ornith-1.0-35B / Qwen3.5-35B-A3B	❌	✅ (48-64 GB)	✅	✅
Llama 3.3 70B	❌	🟡 (64 GB)	✅	✅
Qwen3.5-122B-A10B	❌	❌	✅	✅
Qwen3.5-397B	❌	❌	❌	✅ (256 GB+)

The Mac mini M4 Pro (up to 64 GB) is an excellent, affordable little AI workstation.

Consumer Windows — VRAM decides

✅ works well · 🟡 works but tight or slow · ❌ does not fit. Reminder: the model must fit entirely in VRAM, or performance drops 10 to 30×.

Model	RTX 4070 (12 GB) PC ~$1,500-2,000	RTX 5060 Ti (16 GB) PC ~$1,500-2,000	RTX 4090 (24 GB) PC ~$3,000-4,000	RTX 5090 (32 GB) PC ~$4,500-6,000
Cohere Transcribe	✅	✅	✅	✅
Qwen3.5-9B / Ornith-1.0-9B	✅	✅	✅	✅
Mistral Small 24B	❌	🟡	✅	✅
Qwen3.5-27B	❌	🟡	✅ (Q4)	✅ (Q8)
Qwen3.5-35B-A3B	❌	❌	✅ (Q4)	✅
Ornith-1.0-35B	❌	❌	🟡 (very tight)	✅
Llama 3.3 70B	❌	❌	❌	🟡 (tight Q3)
Qwen3.5-122B-A10B	❌	❌	❌	❌

The 16 GB card is far more useful than the 12 GB one. The RTX 5090 (1.79 TB/s of bandwidth — nearly 3× the M5 Max) excels up to 32B, but even 32 GB isn't enough for a clean 70B: that's the structural limit of discrete GPUs vs Apple's unified memory.

Small desk-side servers

NVIDIA DGX Spark (~$7,348) — 128 GB of unified memory, models up to ~200 billion parameters. Important weak point: only 273 GB/s of bandwidth — a dense Llama 3.3 70B generates ~2-3 tokens/second in conversation. It shines on MoE models and batch processing. Linux only (DGX OS).
Dual RTX 4090 server PC (48 GB VRAM) — Llama 3.3 70B at Q4 at ~10-15 tokens/second. The proven path for dense models up to 70B.
Mac Studio M4 Ultra (192 GB) — ~819 GB/s of bandwidth: loads the 122B-A10B and full-precision 70B effortlessly. Quiet, efficient, and clearly faster than the DGX Spark on big models.

The recommendation matrix

Need	Recommended hardware	Suggested model
Transcribe client meetings in French	16 GB Mac or PC with 8 GB+ GPU	Cohere Transcribe
Daily text assistant (summaries, documents)	MacBook Pro M5 Pro 48 GB or Mac mini M4 Pro 64 GB	Qwen3.5-35B-A3B
Agentic coding, automation, local agents	MacBook Pro M5 Max, Mac Studio or RTX 4090+ PC	Ornith-1.0-35B MoE
Minimal budget, discovery	MacBook Air M4 16 GB or RTX 5060 Ti 16 GB PC	Qwen3.5-9B or Ornith-1.0-9B
Best affordable desktop	RTX 4090 24 GB PC or Mac mini M4 Pro 64 GB	Qwen3.5-35B-A3B or Ornith-1.0-35B
Load the frontier class (122B) locally	MacBook Pro M5 Max 128 GB or Mac Studio	Qwen3.5-122B-A10B
Shared AI lab for a team	NVIDIA DGX Spark 128 GB	Models up to 122B
Fast dense 70B for several users	Dual RTX 4090 PC or Mac Studio Ultra	Llama 3.3 70B

Caveats

Prices fluctuate. A 2026 GPU memory shortage pushed cards above list price. Check before buying.
Vendor benchmarks. Ornith and Qwen scores come partly from their own evaluations. Test on your own tasks before production.
Quantization has a cost. A 27B at Q4 often beats a 70B at Q2. Don't force an oversized model.
Specialization sometimes beats size. Ornith-1.0-35B, a coding specialist, beats Qwen3.6 35B at programming. For general text, the reverse can be true.
Cohere Transcribe and Québécois French. Trained mostly on European French; mixing FR and EN in one sentence can trip it. Test with your own recordings.
This field moves very fast. This guide is a snapshot from July 2026.

This model class — the one that runs on a very good workstation — is what powers Luge's managed sovereign modules: small verified steps, on Canadian infrastructure, for Law 25 and PIPEDA. And if you'd rather run them yourself: download Luge — the catalog is in the app.

Which AI models actuallyrun on your machine?