the recipe log
Recipes, not vibes.
A recipe names the failure mode, the data, the eval gate, the slice gate, and the stop rule. "Try DPO" is not a recipe — it's a wish. These are the named, reproducible post-training and packaging recipes the factory runs, plus cookbooks for real integrations. Companion logs: the devlog and theexperiment lab.
Post-training recipes
Method vs recipe — the standard
Why "try DPO" isn't a plan. A recipe names the failure mode, the data, the eval gate, the slice gate, and the stop rule.
Open recipe →Distill a specialist from a bigger model
Teacher → student on a local machine. KL + NLL mix, temperature, α, rejection sampling on a checker.
Open recipe →Distill function-calling: Phi-3-mini → tiny
The full function-calling distillation arc — the recipe behind the frontier-parity result.
Open recipe →From .atraj rollouts to a trained specialist
Turn agent trajectories into SFT data: filter, dedupe (MinHash), emit ChatML, train. The self-improving loop.
Open recipe →eval-gate as a CI / pre-commit gate
Wire a frozen eval into CI so a regression exits non-zero. The discipline that keeps main honest.
Open recipe →Balanced training mix
Mixing corpora without one domain drowning the others — ratio search with a micro-AutoMixer.
Open recipe →Micro-AutoMixer for data-mix ratios
Search data-mix ratios cheaply before committing compute to a full run.
Open recipe →ScaleDown — extractive context compression
A specialist that compresses context extractively — smaller prompts, same answer.
Open recipe →packagingMLX export
Package a trained adapter/model for the MLX runtime — the last mile before it serves.
Open recipe →Cookbooks — real integrations
Character specialist
Train a small model with a consistent voice/persona, end to end.
Open cookbook →Personal code specialist
A code model tuned on your own repos and conventions.
Open cookbook →posttrainllm + Pydantic AI
Drive a local posttrainllm model from Pydantic AI via the OpenAI-compatible endpoint.
Open cookbook →posttrainllm + smolagents
Run smolagents against a local specialist — agents on one machine, no cloud.
Open cookbook →Complete technique → recipe registry
Every retained technique has a final project disposition and a concrete lab contract: target, failure mode, data, method, eval, regressions, budget, stop rule, decision rule, exercise, and mastery gate. These are ready learning labs, not an unfinished AI backlog.
Constrained generation
A capable model emits invalid syntax or forbidden output shapes.
- Lab
- Run a tiny legal-token or JSON-schema fixture and compare raw versus constrained validity.
- Mastery
- Explain why legal output is not the same as correct output.
Specialist distillation
A small base lacks a narrow behavior that a qualified teacher can demonstrate.
- Lab
- Trace the file-ops lineage from stock 4B through distilled, multibackend, and ReST candidates.
- Mastery
- State the target gain and breadth cost that force routing.
Evolution strategies
The objective cannot provide useful gradients or the model/policy is treated as a black box.
- Lab
- Simulate one two-parameter ES update on a deterministic toy reward.
- Mastery
- Explain the evaluation-cost tradeoff versus backpropagation.
GaLore and training stability
Optimizer state or unstable gradients block an otherwise valid training recipe.
- Lab
- Compare the recorded GaLore smoke with the gradient-checkpointing result and identify what each actually saves.
- Mastery
- Distinguish optimizer-state memory from activation memory.
Interpretability probes
Aggregate scores cannot explain which layers, tokens, or representations drive a failure.
- Lab
- Inspect one logit-lens or activation-patching fixture and write the causal claim it does and does not support.
- Mastery
- Separate correlation, localization, and causal intervention.
LoRA, DoRA, and adapter geometry
The base misses a bounded task and full fine-tuning is unnecessary or too costly.
- Lab
- Run or inspect LoRA geometry for one successful and one failed adapter.
- Mastery
- Explain rank, target modules, effective rank, and why lower loss is insufficient.
Sparse mixture of experts
A dense model lacks capacity and a true sparse execution path is available.
- Lab
- Calculate active versus total parameters for top-k expert routing and identify why dense execution saves nothing.
- Mastery
- Explain the difference between sparse parameters and sparse compute.
Multi-token prediction
Single-token supervision or decode throughput is the measured bottleneck.
- Lab
- Draw the label shift for two future-token heads and compute the added loss terms.
- Mastery
- Explain why training more heads does not guarantee faster decoding.
Optimizer and schedule selection
Loss spikes, under-updating, or optimizer memory blocks a frozen recipe.
- Lab
- Diagnose three recorded loss curves as learning-rate, capacity, or data failures.
- Mastery
- Choose an optimizer change from evidence rather than novelty.
Parameter-efficient fine-tuning variants
Plain LoRA has a measured geometry, capacity, quantization, or many-adapter limitation.
- Lab
- Create a one-page comparison of LoRA, DoRA, QLoRA, and VeRA parameter/memory assumptions.
- Mastery
- Select a PEFT variant from the bottleneck, not from its name.
Precision and numerical gates
Precision choice causes overflow, drift, excess memory, or slow execution.
- Lab
- Measure fp32/fp16 error on a tiny matrix operation and choose a tolerance before looking at the result.
- Mastery
- Explain why a faster kernel with wrong numerics is not an optimization.
Unstructured and structured pruning
Artifact size or dense topology is the measured deployment bottleneck.
- Lab
- Compare 50% zero weights with physical layer removal and predict which metrics change.
- Mastery
- Distinguish statistical sparsity, compressed storage, and runtime sparsity.
Quantization and storage formats
Full precision exceeds the target's distribution or memory budget.
- Lab
- Trace one fp16-to-int4 artifact from calibration through load and decode measurement.
- Mastery
- Explain why four-bit storage can still execute with wider arithmetic.
Speculative heads and decoding
Autoregressive decode latency is the measured user-visible bottleneck.
- Lab
- Compute expected speedup from draft length, acceptance rate, and verification cost.
- Mastery
- Identify when speculative decoding becomes slower than ordinary decoding.
StreamingLLM and KIVI cache compression
KV cache growth, not model weights, is the measured memory blocker.
- Lab
- Calculate full, windowed, int8, and int4 KV bytes for one model/context.
- Mastery
- Choose between eviction and quantization from the actual context failure mode.
SQL specialist lineage
Small-model SQL gains fail to generalize across public and synthetic domains or emit prose wrappers.
- Lab
- Reconstruct the SQL lineage and mark which variable changed at each step.
- Mastery
- Explain why routing beat blending and why DPO failed to remove the wrapper.
Needle 45M successor factorial
Tiny overfit can pass while held-out selection remains below stock and destructive confirmation behavior fails completely.
- Lab
- Reconstruct the three factorial effects, apply the logical early-stop rule, and design the next 1.7B capacity test without changing the public gate.
- Mastery
- Explain why 100% tiny overfit coexists with a conclusive failed promotion and why sealed V2 stayed closed.
Parakeet WGSL browser ASR
Browser audio ingestion may be too slow, too large, nondeterministic, or network-dependent.
- Lab
- Re-score the two prediction files, identify which WhisperKit errors come from numeric normalization, and compare aggregate with median per-clip RTFx.
- Mastery
- Explain why 0.0% WER and a 3.51x paired latency win can coexist with a correct reject decision under the frozen 50x gate.
Technique registry
SQL technique lineage
The closed ledger of SQL recipes and their final dispositions — each tied to its reward surface.
Open →TrainLoop teardown
What was learned (and stolen) from an external post-training platform.
Open →Audit 2026 technique inventory
Row-level treatment of every technique in the 2026 audit: keep, experimental, flagged, delete.
Open →