a one-machine research lab · MIT
posttrainllm turns a stock open model into a routed specialist that earns its keep — then shows you exactly where it fails. Distill, fine-tune, gate on real evals, package for MLX, and open the model up with interpretability. One Apple Silicon machine. No cloud. No hand-waving.
the thesis
Win on the Mac. Be best-in-class at what one Apple Silicon machine can actually do — train, post-train, evaluate, serve, inspect — with no cluster and no cloud dependency.
Learn the whole space. From y = mx + b to a self-improving factory. Every technique is built from scratch, anchored to the paper, and mapped to the file where it lives.
Show the scars. Failed runs are first-class. Every attempt is logged with the decision it forced. A number without its regression is marketing, not a result.
Build everything buildable here. If it fits on this Mac, it gets built — and packaged so that when compute arrives, the lab scales from a running start.
Each surface emits the input the next one needs. Every run ends in a schema-valid folder — config, dataset, eval, decision, report — that factory-run validates.
frozen5,675 rowsdpo · 200 stepsexec 0.920tgla · mlxretry-datalatest · 2026-07-11 · qwen06-sql-hygiene-dpo · reference-anchored DPO fixed an earlier collapse, execution 0.860→0.920 with no regression — but output hygiene is a base-model prior, so the decision is retry-data. the one proof still missing is a run whose decision is ship.
A specialist beats a generalist on its target and routes away when it shouldn't answer. Every result ships with the regression it costs.
Distilled to 100% on the multi-turn file-ops hard gate (BFCL), up from 58%. The honest cost: out-of-domain breadth dropped 59.6% → 42.3% — real catastrophic forgetting. It ships routed, never as a general planner.
Two adapters behind a router. Two reference-anchored DPO retries cured a policy collapse and pushed execution to 0.920 — but output hygiene is a base-model prior a small adapter can't strip. Decision: retry-data.
Every figure below was recorded on a single Apple M5 Pro / 48 GB. No estimates, no extrapolation.
Not a wrapper. Every capability is a real subcommand with tests behind it.
Pretrain, SFT, DPO / SimPO / KTO / ORPO, distillation, ES. Full PEFT — LoRA, LoRA+, DoRA, VeRA, LoftQ, AdaLoRA, PISSA. WSD schedules, spike recovery, z-loss.
BFCL, τ-bench, lm-eval (MLX adapter), HumanEval + sandbox, SQL execution, router, MILU, MTEB. Frozen baselines, slice metrics, non-zero-exit gates.
OpenAI- and Ollama-compatible on one socket. Agent loop, tool dispatch, FSM-constrained JSON, persistent KV cache, speculative decoding, optional cloud escalation.
Export to MLX, safetensors, CoreML. Quantize (GGUF / AWQ / GPTQ / HQQ), prune, merge, bake-LoRA with DoRA magnitudes. Specialist model cards.
SAE features, ROME, MEMIT, tuned / logit lens, activation patching, linear probes, attention heatmaps. Know where the model decides.
The same model trains in a browser tab via hand-written WebGPU kernels — Memory64, FlashAttention-2, blocked matmul. A from-scratch learning track, honest negative results included.
Every experiment, lesson, and recipe is written down — with the decision it forced and the file it lives in.
From y = mx + b to a self-improving factory. Every subsystem maps to an anchor in the coverage map — nothing is a black box.
Worked · failed · regressed · retried — each with a decision.
devlogKernels, bugs, and the surprises worth more than the code.
recipesNot "try DPO" — a named failure mode, data, gate, and stop rule.
interpretabilityLens, patching, SAE, ROME, MEMIT.