This guide explains, in plain language, which AI models run directly on a computer or a
small server — no cloud required. The point: your data never leaves your
machine. For a professional, that means a client file, a statement or a meeting
recording stays private. And when a task deserves a big hosted model, Luge lets you
plug in your own key — Anthropic, OpenAI, Azure, and now xAI (Grok), for text and
voice alike: local and cloud get picked task by task.
Prices in CAD, current as of August 2026 — this field moves fast.
In a hurry? Jump straight to the recommendation matrix —
what to buy for your budget, in one table.
Snapshot from August 2026: useful local models range from 2.4 GB (Qwen3.5-4B) to ~81 GB (Qwen3.5-122B). A MacBook Pro M5 Pro 48 GB comfortably runs the 35B MoE class we recommend (Ornith 1.0, Qwen3.5-35B-A3B), an RTX 4090 24 GB PC gets there at Q4, and transcription (Cohere Transcribe, ~5 GB) fits almost anywhere. If you remember one thing: available memory decides.
The key concept: memory decides everything
An AI model must fit entirely in the machine's fast memory. Think of
it as a desk: if the document doesn't fit on it, everything slows down.
On Windows, that memory is the graphics card's VRAM — separate and limited (12, 16, 24 or 32 GB). If the model doesn't fully fit, performance drops 10 to 30×.
On Mac, memory is unified: CPU and GPU share one pool. A 64 GB Mac can devote nearly all of it to the model. A major structural advantage.
On small servers like the DGX Spark: unified memory too (128 GB), so very large models fit.
Two useful terms:
Quantization (Q4, Q5, Q8): compressing a model so it takes less space. Q4 is the standard (good quality/size balance); Q8 is more faithful but twice as heavy.
MoE (mixture of experts): the model holds a lot of knowledge but only activates a small part per word — big-model quality at small-model speed. Careful: the full model still has to fit in memory. MoE saves compute, not memory.
The models covered
Ornith 1.0 — the agentic-coding specialist (DeepReinforce, MIT)
Released June 25, 2026. Built for agentic coding: it doesn't just write code, it
plans, runs tools, tests and fixes its own mistakes. Field note: on a MacBook Pro M5
Max, Ornith 1.0 35B beats Qwen3.6 at coding and agentic workflows —
the value of specialization. MIT license, unrestricted commercial use. Available as
GGUF (Ollama, LM Studio).
Model
Q4 memory
SWE-Bench
Who it's for
Ornith-1.0-9B
~6 GB
69.4
Entry point, modest machines
Ornith-1.0-31B
~20 GB
—
Dense, stable
Ornith-1.0-35B MoE
~25 GB
75.6
Recommended for most
Ornith-1.0-397B MoE
~200 GB FP8
82.4
Servers, production
Qwen3.5 / Qwen3.6 — the multimodal generalist (Alibaba, Apache 2.0)
Excellent in French, multimodal, 262,000-token context. Where Ornith wins at code,
Qwen is unbeatable at language: summaries, document Q&A, contract analysis,
drafting. Qwen3.6 has the same memory footprint as 3.5 — a drop-in replacement.
#1 on Hugging Face's Open ASR Leaderboard: 5.42% error rate vs 7.44% for Whisper Large
v3. 14 languages including French, ~5 GB, runs on almost any recent hardware, ~525
minutes of audio transcribed per minute. Known limits: the model alone doesn't
identify speakers or timestamp natively — but the platform handles attribution
another way. In Luge voice rooms, each person is listened to separately: attribution
is real, not guessed. And diarization (pyannote) stays available as an opt-in on the
edge node. One more limit: it's trained mostly on European French — test it
on your own Québécois recordings, especially if you mix French and English
in one sentence.
The classic safe bets
Llama 3.3 70B (Meta): ~40-43 GB at Q4. A proven generalist with the largest ecosystem. Needs two GPUs or a big-memory Mac.
Mistral Small 24B (Mistral): ~13-14 GB at Q4. Excellent in French, fits in 16 GB.
Mac — unified memory, the big advantage
✅ works well · 🟡 works but tight or slow · ❌ does not fit. Apple's native MLX format is ~10-15% leaner than GGUF and often 15-30% faster.
MacBook Air M4 (fanless — ideal for transcription and small models)
Model
16 GB
32 GB
Cohere Transcribe
✅
✅
Qwen3.5-9B / Ornith-1.0-9B
✅
✅
Qwen3.5-27B
❌
✅
Mistral Small 24B
🟡
✅
Qwen3.5-35B-A3B / Ornith-1.0-35B
❌
🟡
Llama 3.3 70B
❌
❌
MacBook Pro M5 Pro
Model
24 GB
48 GB
Cohere Transcribe
✅
✅
Qwen3.5-27B
✅
✅
Qwen3.5-35B-A3B MoE
✅
✅
Ornith-1.0-35B MoE
🟡
✅
Llama 3.3 70B (Q4)
❌
🟡
Qwen3.5-122B-A10B
❌
❌
MacBook Pro M5 Max
Model
48 GB
64 GB
128 GB
Cohere Transcribe
✅
✅
✅
Qwen3.5-35B-A3B
✅
✅
✅
Ornith-1.0-35B MoE
✅
✅
✅
Llama 3.3 70B (Q4)
🟡
✅
✅
Qwen3.5-122B-A10B (~74-81 GB)
❌
❌
✅
Qwen3.5-397B
❌
❌
❌
The M5 Max at 128 GB is the only laptop that can load Qwen3.5-122B-A10B — a
frontier-class model that beats GPT-5 mini by 30% at tool use.
Mac mini M4 / M4 Pro and Mac Studio
Model
mini M4 (16-32 GB)
mini M4 Pro (24-64 GB)
Studio M4 Max (128 GB)
Studio Ultra (192-256 GB)
Cohere Transcribe
✅
✅
✅
✅
Qwen3.5-27B
🟡 (32 GB)
✅
✅
✅
Ornith-1.0-35B / Qwen3.5-35B-A3B
❌
✅ (48-64 GB)
✅
✅
Llama 3.3 70B
❌
🟡 (64 GB)
✅
✅
Qwen3.5-122B-A10B
❌
❌
✅
✅
Qwen3.5-397B
❌
❌
❌
✅ (256 GB+)
The Mac mini M4 Pro (up to 64 GB) is an excellent, affordable little AI workstation.
Consumer Windows — VRAM decides
✅ works well · 🟡 works but tight or slow · ❌ does not fit. Reminder: the model must fit entirely in VRAM, or performance drops 10 to 30×.
Model
RTX 4070 (12 GB) PC ~$1,500-2,000
RTX 5060 Ti (16 GB) PC ~$1,500-2,000
RTX 4090 (24 GB) PC ~$3,000-4,000
RTX 5090 (32 GB) PC ~$4,500-6,000
Cohere Transcribe
✅
✅
✅
✅
Qwen3.5-9B / Ornith-1.0-9B
✅
✅
✅
✅
Mistral Small 24B
❌
🟡
✅
✅
Qwen3.5-27B
❌
🟡
✅ (Q4)
✅ (Q8)
Qwen3.5-35B-A3B
❌
❌
✅ (Q4)
✅
Ornith-1.0-35B
❌
❌
🟡 (very tight)
✅
Llama 3.3 70B
❌
❌
❌
🟡 (tight Q3)
Qwen3.5-122B-A10B
❌
❌
❌
❌
The 16 GB card is far more useful than the 12 GB one. The RTX 5090 (1.79 TB/s of
bandwidth — nearly 3× the M5 Max) excels up to 32B, but even 32 GB isn't enough for a
clean 70B: that's the structural limit of discrete GPUs vs Apple's unified memory.
Small desk-side servers
NVIDIA DGX Spark (~$7,348) — 128 GB of unified memory, models up to ~200 billion parameters. Important weak point: only 273 GB/s of bandwidth — a dense Llama 3.3 70B generates ~2-3 tokens/second in conversation. It shines on MoE models and batch processing. Linux only (DGX OS).
Dual RTX 4090 server PC (48 GB VRAM) — Llama 3.3 70B at Q4 at ~10-15 tokens/second. The proven path for dense models up to 70B.
Mac Studio M4 Ultra (192 GB) — ~819 GB/s of bandwidth: loads the 122B-A10B and full-precision 70B effortlessly. Quiet, efficient, and clearly faster than the DGX Spark on big models.
The recommendation matrix
Need
Recommended hardware
Suggested model
Transcribe client meetings in French
16 GB Mac or PC with 8 GB+ GPU
Cohere Transcribe
Daily text assistant (summaries, documents)
MacBook Pro M5 Pro 48 GB or Mac mini M4 Pro 64 GB
Qwen3.5-35B-A3B
Agentic coding, automation, local agents
MacBook Pro M5 Max, Mac Studio or RTX 4090+ PC
Ornith-1.0-35B MoE
Minimal budget, discovery
MacBook Air M4 16 GB or RTX 5060 Ti 16 GB PC
Qwen3.5-9B or Ornith-1.0-9B
Maximum quality per answer, throughput less of a rush (dense)
Mac 32 GB+ or PC RTX 4090 24 GB
Qwen3.8-27B (17.9 GB) — every parameter active on every token, where MoE models activate a fraction
Best affordable desktop
RTX 4090 24 GB PC or Mac mini M4 Pro 64 GB
Qwen3.5-35B-A3B or Ornith-1.0-35B
Load the frontier class (122B) locally
MacBook Pro M5 Max 128 GB or Mac Studio
Qwen3.5-122B-A10B
Shared AI lab for a team
NVIDIA DGX Spark 128 GB
Models up to 122B
Fast dense 70B for several users
Dual RTX 4090 PC or Mac Studio Ultra
Llama 3.3 70B
And once you've picked a model?
No server to stand up. You connect your machine to Luge with a single command —
luge-edge pair — then pick your model from the edge node's catalog,
pinned by SHA-256: the downloaded file is verified bit for bit.
Two lists, two jobs. The edge catalog holds the pinned subset — the models we
install and verify for you. This guide covers everything that runs on your machine,
whatever the tool. And if a number differs between the two — 2.4 GB here, 2.7 GB
there for the same small Qwen — that's because the guide quotes memory occupied once
loaded, while the catalog quotes file weight on disk.
Download Luge: the edge node comes with it.
Caveats
Prices fluctuate. A 2026 GPU memory shortage pushed cards above list price. Check before buying.
Vendor benchmarks. Ornith and Qwen scores come partly from their own evaluations. Test on your own tasks before production.
Quantization has a cost. A 27B at Q4 often beats a 70B at Q2. Don't force an oversized model.
Specialization sometimes beats size. Ornith-1.0-35B, a coding specialist, beats Qwen3.6 35B at programming. For general text, the reverse can be true.
Cohere Transcribe and Québécois French. Trained mostly on European French; mixing FR and EN in one sentence can trip it. Test with your own recordings.
This field moves very fast. This guide is a snapshot from August 2026.
This model class — the one that runs on a very good workstation — is what powers
Luge's managed sovereign modules: small verified steps,
on Canadian infrastructure, for Law 25 and PIPEDA. And if you'd rather run them
yourself: download Luge — the catalog is in the app.
The catalog lives in the app.
The edge node ships with Luge: one line to paste, one code to type, and you pull your models straight from the interface — no terminal.
