← research lab

Mac-local specialist field guide

How to train and fine-tune your own LLM on a Mac

Practical guide to train your own specialist LLM on Apple Silicon: define a target, LoRA/QLoRA post-train, evaluate against a frozen baseline, and package with evidence.

Fine-tuning an LLM on a Mac is practical when the target is narrow, the model fits Apple Silicon memory, and the evaluation is designed before training begins. The useful goal is rarely to build a smaller general chatbot. It is to create a specialist that improves on a defined task, routes away from tasks it should not answer, and ships with evidence about both gains and regressions.

Start with a target, not a trainer

Write down the task, input and output contract, baseline model, acceptance slices, and failure conditions. A target such as file operations, SQL hygiene, or intent routing can be evaluated. “Make the model smarter” cannot. Freeze a holdout before generating or cleaning training data.

Choose a method that fits the Mac

LoRA updates a small set of adapter weights while leaving the base model frozen. QLoRA applies low-rank adaptation to a quantized base and can reduce memory pressure further. Full-weight fine-tuning is possible for smaller models but is not the default starting point for a one-machine workflow. Apple’s MLX LM supports text generation, quantization, LoRA, QLoRA, and full fine-tuning on Apple Silicon.

Build the factory loop

  1. Target: define the behavior and routing boundary.
  2. Data: create train, validation, and frozen test splits with provenance.
  3. Post-train: run a bounded recipe and preserve the configuration.
  4. Evaluate: compare the unchanged baseline and candidate on the same gates.
  5. Package: export the adapter or fused model with runtime metadata.
  6. Report: publish gains, regressions, missing measurements, and the ship, retry, or reject decision.

PostTrainLLM implements this loop as a Mac-local specialist factory. Its public evidence includes specialist packages and report cards, including a file-operations specialist that improves its hard gate from 58% to 100% while regressing out-of-domain breadth from 59.6% to 42.3%. That tradeoff is why it is routed rather than presented as a general planner.

What success looks like

Success is a reproducible candidate that beats the frozen baseline on the target, stays within explicit regression limits, fits the intended runtime, and has enough evidence to make a decision. A falling training loss is only an intermediate signal.

Check memory feasibility without promising that a model fits

Start with the model configuration and actual weight files. Parameter count times bits per weight gives an idealized weight-storage lower bound, not peak unified-memory use. Quantization metadata, unquantized layers, adapters, activations, gradients, optimizer state, temporary buffers and the operating system add to it. Inference fit does not establish training fit.

parameters = 4_000_000_000  # illustrative count, not a measured model
for bits in (4, 8, 16):
    gib = parameters * bits / 8 / 2**30
    print(f"{bits}-bit idealized weights: {gib:.2f} GiB")

This CPU arithmetic prints 1.86 GiB at 4-bit, 3.73 GiB at 8-bit and 7.45 GiB at 16-bit. Those numbers omit all overhead and are not RAM requirements. Do not conclude that a 4B model will train on a given Mac from this table.

Record before a runWhy it changes the result
Exact model revision, quantization and trainable adapter layers/rankDetermines stored weights and the parameters being updated
Sequence length, microbatch and gradient-accumulation policyChanges token exposure and activation/storage requirements; accumulation is not a larger microbatch
Physical memory, other memory pressure and free output spaceUnified memory is shared; downloaded and fused artifacts also need disk space
Observed peak memory, swap pressure and completed steps under the intended recipeEstablishes what actually ran; a successful model load is only a load check

If an authorized smoke run runs out of memory, change one variable at a time: shorten sequences or reduce the microbatch, reduce trained layers, or choose a smaller/quantized compatible base. Preserve the failed configuration. These options follow the source-pinned MLX LM memory guidance; they can alter the learning problem and need the same evaluation gates. No training or memory measurement was performed for this worksheet.

CTA: Inspect the public specialist artifacts, then follow the MLX LoRA and evaluation guides.