Required files
8config, dataset, train log, evals, report, artifact, decisionderivedFactory Run Schema v1
A specialist factory needs proof folders, not vibes. This schema defines the local run directory, required metrics, and decision vocabulary that every public artifact should eventually satisfy.
Headline Numbers
Evidence class applies to each value: measured = retained evaluation or runtime result; historical = recorded result not freshly reproduced; derived = calculated or decided from records; observed = current artifact state; not-measured = an explicit evidence gap.
Decisions
6ship, reject, retry-data, retry-training, retry-eval, parkderivedFirst-class outputs
5data, training, eval, package, reportderivedCompetitive Context
| System | Metric | Score | Size / Class | Comparable? | Readout |
|---|---|---|---|---|---|
| posttrainllm factory schema | required public run files | 8 | repo-local contract | Direct | Defines the minimum evidence bundle each future public model artifact must carry. |
| Ad hoc model card only | before/after reproducibility | weak | single document | Directional | Useful for release notes, but insufficient for a factory claim without eval JSON, decision, and blockers. |
Direct rows share this artifact's eval setup. Directional rows are useful market context but should not be read as leaderboard claims.
Run folder contract
| File | Purpose | Public relevance |
|---|---|---|
| config.json | Target, base, method, thresholds | Explains what was attempted |
| dataset.json | Sources, rows, filtering, heldout | Provenance |
| eval-baseline.json | Frozen baseline result | Before number |
| eval-candidate.json | Candidate result | After number |
| decision.json | Ship/reject/retry call | Honest release status |
Release Blockers
Live commands do not emit the whole schema automatically
The routed SQL report card is now the canonical rendered example, but training and evaluation commands still require orchestration to assemble every run fragment.
Unblock: Make future train/eval commands emit schema-complete run folders and enforce them with publish-check.