The Off-the-Shelf Alternative

A baseline has to be something we did not build

Our first detector was YOLO26n fine-tuned on our own synthetic renders. Using it as the left column of an ablation measures our training against itself, which answers nothing. The question a baseline has to answer is narrower and less comfortable: does our data pipeline beat what a competitor downloads on a Friday afternoon, on real frames?

So the reference is now off-the-shelf, our own detector is a result rather than a baseline, and the ablation reads B0/B1 → D1 → D2 → D3. The headline claim is the step from the B row to the D rows. If there is no step, that is the finding.

Without a task definition the comparison produces two artefacts, both wrong

B0 and B1 detect people in any posture. D1 detects people lying down. Score them naively on the same frames and you get two numbers that look decisive and mean nothing.

So the task is fixed once, before anything runs, and every arm is held to it: does this frame contain a person lying on the ground? One binary decision per frame. And the off-the-shelf alternative is not a bare person detector — it is a person detector plus a cheap posture rule, which is exactly what a competitor would build in an afternoon.

Rule r, chosen from geometry and recorded before the run: box aspect ratio w/h ≥ 1.3. A prone adult at rover camera height is wider than tall. Nothing is tuned on evaluation data; a tuned threshold, if anyone wants one later, is fitted on the calibration frames only and reported as its own arm.

B0 and B1 are not the same experiment

Both are off-the-shelf, so both test the training question. Only one of them can also be the answer — because "off the shelf" turns out to be three independent things (no training cost, a permissive licence, and a deployment path already built) and they do not line up across the arms.

Two instruments, opposite asymmetries

The two frame sets are not a big one and a small one. They fail in opposite directions, and that is what makes the pair worth running rather than either alone. Read the matrix cell by cell: the same outcome means different things depending on whose training distribution it happened on.

The v2 validation set supersedes an earlier ruling of ours. We had written off "the synthetic val set" as saturated — 53 of 54 boxes in every arm and every seed — but that was the old val, 54 boxes of one camera view. The v2 split carries roughly 1,100 upright-person boxes across six hard-negative pose classes plus 386 lying-vest positives. It is a different instrument and it separates arms on false alarms. The saturation objection does not transfer.

What is held identical, and the one thing that cannot be

Three ways this produces a confident wrong answer

1. A box-norm scale error that confirms our prior. Get DetectNet_v2's bbox scaling wrong and the boxes come out plausibly placed and systematically the wrong size. Nothing downstream errors. It surfaces as "PeopleNet has poor recall on prone people" — which is precisely the assumption we are testing. We would confirm ourselves for the wrong reason. Mitigation: draw decoded boxes back onto ten frames and look at them before scoring anything.
2. A posture rule tuned on the evaluation set. The rule is the baseline's whole posture capability, so a threshold fitted where it is scored hands the baseline an advantage D1 never had. w/h ≥ 1.3 comes from geometry and is written down first. Any tuned variant is fitted on calibration frames and reported as a separate arm, never swapped in.
3. Blaming the detector for the rule. If person_seated and person_crouching dominate the false alarms, that is a rule problem, not a PeopleNet problem — a seated person is genuinely wider than tall. Mitigation: per-pose confusion for every arm, so the rule's failures are visible separately from the detector's.

Every cell carries its N

A tie is not a tie

Accuracy on a workstation, runtime on the board — and a gate between them

The obvious move is to run everything on the Orin Nano, since that is the hardware that ships. It is the right instinct and the wrong order: these are two measurements with different failure modes, and merging them costs both.

The gate that joins them. An accuracy number taken on a workstation does not transfer to the board for free — INT8 quantisation and a different decode path can move it. So before any L1 claim, the two runs are compared per frame on a seeded subsample, not on aggregate accuracy: two runs can agree on the total while disagreeing on a quarter of the frames. Expect ≥ 99 % agreement; anything worse is a build problem to fix before the numbers mean anything. Same gate we wrote for the 4-bit reasoner's engine, same reason.

This is the evaluation ladder's own division, applied one layer down: the static comparison ranks the models, the board supplies the runtime. Neither figure is the other's, and no table should carry both in the same column.

Designed, not measured

Everything above is a protocol. There is no number behind any of it yet, and the one claim we have made in writing about PeopleNet's behaviour — that recall drops on prone people — is an assumption, labelled as one.

Stopping rule: if the decoder resists past 90 minutes, report B0 alone with B1 marked blocked on the DetectNet_v2 decode. B0 against D1 — COCO person plus ten lines of rule against our whole synthetic pipeline — is already a real result, and a partial table with a named blocker beats an afternoon lost to scale factors.