Turn proprietary data into
a model you can ship.
Open models. Your infra. Your control.

Measured

A real model, on a real GPU

The task is one nothing could have been pre-trained on: shelf photos answered with an arbitrary internal bin code. That is why a near-floor zero-shot is the expected result here, and why the delta means what a customer would get on their own data. Every figure below is generated from the run's own verdict.json.

Adapted vs prompted baseline

McNemar exact, p = 0.0009766 measured

Envelope

p50 99.3 ms, p95 103.2 ms, peak 2.62 GB, adapter 34.2 MB, measured on geforce-rtx-5090-34gb — the device that ran it measured

Training

179 s, 3.40 GB peak VRAM, 240 examples measured

Evaluation data

private — generated locally from a seed with a private answer vocabulary; the images did not exist before this run. A model guessing uniformly over the answer vocabulary would score 6.2%, which is what makes "near the floor" a number you can check

Split integrity

12 train scenes, 8 eval scenes, disjoint; 0 exact and 0 perceptual-hash collisions

• ✓evidencecomplete
• –accuracy11.46%
• ✓significancep = 0.0010, adapted +11.46% vs B1 (engineered prompt)
• ✓slice regressionno regression across 4 slice(s)
• –latency103.2 ms
• –memory2.6208 GB
• –artifact size34.1535 MB

Pipeline validation

The same shape, on the reference harness

A pipeline validation on a deterministic CPU harness — not a benchmark, not a real vision-language model, and not a customer result. It is here because the shape of the evidence is the point.

Open the full arm table, significance, envelope and gates

Build f7534abc — structured-evaluator 1.0.0, eval split 868cb284a22e

Arm	Accuracy	95% Wilson	n	Seeds	Lineage
Zero-shot baseline	6.1%	[4.7%, 8.0%]	800	single run	test_fixture
Engineered-prompt baseline	15.2%	[12.9%, 17.9%]	800	single run	test_fixture
Default LoRA	56.2%	[54.2%, 58.1%]	800	3 seeds, 54.8%–56.9%	test_fixture
Planner-selected adaptation	55.2%	[53.2%, 57.2%]	800	3 seeds, 53.9%–56.0%	test_fixture

Adapted vs prompted baseline

McNemar exact, p < 1e-15 test_fixture

Adapted vs default LoRA

McNemar exact, p = 0.7561 — no planner advantage is claimed, and the fixture is built so it cannot encode one test_fixture

Split integrity

20 train scenes, 20 eval scenes, disjoint; 0 exact and 0 perceptual-hash collisions across splits

Envelope

p95 0.17 ms, peak memory 0.09 GB, artifact 2.10 MB, measured on cpu-local measured

What this run is

A pipeline validation on the deterministic reference harness, not a benchmark and not a real vision-language model. The accuracy figures are labelled test_fixture for that reason; the envelope figures are labelled measured because the latency and memory really were measured, on cpu-local and nothing else.

What we are not claiming: that the planner beats a competent default recipe. The comparison above shows no such advantage, and the fixture is deliberately built so it cannot encode one. Whether planner-selected recipes beat default LoRA is an open research question with a written kill criterion; until it concludes, it stays off this page.

Build contract

What the system will and will not do

Enforced in code and covered by tests, not stated as intentions.

• 01
  Every number carries its lineage. There is one number renderer and it refuses to run without a label, so an unlabelled figure is unreachable rather than discouraged.
• 02
  Significance is measured against the prompted baseline. Not the naive zero-shot one — that is the comparison that flatters a demo and means least. McNemar is two-sided, so the gate also requires the adapted arm to be ahead: a significant regression is never a SHIP.
• 03
  Hardware is never extrapolated. Declare a Jetson, run on a workstation GPU, and the latency gate fails as evidence_incomplete rather than passing on the wrong evidence.
• 04
  A gate that could not be checked is reported, not skipped. A must-not-regress slice that never appeared in the evaluation is a NO SHIP with a reason code, not a quiet pass.
• 05
  Splits are scene-disjoint and leakage-gated. Hash collisions across train and eval block the build. A model that improves on every slice is usually a leaked split, so regressions are surfaced rather than filtered out.
• 06
  Non-commercial weights are refused. The licence is verified at the pinned commit, before planning. Unknown is never assumed permissive.
• 07
  Costs are reported or absent, never estimated into a result. No per-job billing from the backend means the cost is null. A guess is not an honest value.
• 08
  Re-executable, not "reproducible". GPU nondeterminism makes bitwise reproduction unachievable, so we publish the run-to-run spread across seeds instead.