UBR Stack · NVIDIA Open Models Codefest 2026 · 11 Sep

Datasets

The pipeline's first stage is a data-quality gate, so the data itself is a deliverable. This is what we published, the sources we used and cited, and the data challenges still open.

PUBLISHEDWhat we put on Hugging Face

Open artefacts under the ubr-physical-ai organisation. Licences are split by content: annotations are ours to release; synthetic renders inherit the terms of the NVIDIA assets they were rendered from, which is the open licence question below.

Dataset	What it is	Scale	Licence
isaac-sdg-rescue-target	The hybrid search-and-rescue set: Isaac Replicator renders passed through Cosmos Transfer, with segmentation masks and merged detection annotations over 15 classes. main = v3, tag v3.0	133,760 images 37 webdataset shards 109 clips w/ masks 559,085 train boxes 139 provenance runs	annotations CC BY 4.0; renders under NVIDIA asset terms	Public
cosmos3-i2v-survival-sdg	Image-to-video synthetic clips explored as an alternative augmentation route. Published as a negative result — the yield did not justify the route, and saying so is the point.	I2V clip set	CC BY 4.0 (ours) / NVIDIA terms on conditioning	Negative result
ubr-maze-nav	Navigation dataset from before the Codefest, kept public as prior work and reused for context, not for the SAR target.	pre-Codefest	CC BY 4.0	Prior work
cosmos-i2v-ugv-v0	Rover-domain conditioning clips. Kept private: derived from a workplace recording of identifiable people. Persons in the clips were anonymised before any run; raw segments never leave the bench.	private	not distributed	Private

SOURCESWhat we used, and cited

Not ours; used under their own terms and never redistributed. Where a source contains identifiable people, only derived frames are used and only aggregate numbers leave the bench.

Source	How it is used	Terms we respect
PhysicalAI-Spatial-Intelligence-Warehouse NVIDIA, Hugging Face	The L0 spatial-reasoning benchmark: 1,942 validation items scored across distance, left/right and multiple-choice. Cited, gated, never redistributed.	CC BY 4.0; annotations also carry the Llama 3.1 Community License
PhysicalAI — SmartSpaces · Spatial-QA · NuRec NVIDIA	Subsets only: SmartSpaces negatives, Spatial-QA items, and NuRec reconstructed rooms turned into Isaac benchmark scenes. Attribution logged per subset.	NVIDIA dataset terms; the 210 GB Spatial-QA train image set is deliberately not pulled
Rover blackbox recording ours, private	The real sensor domain: ~108 h indoor, 791 person segments. Supplies the 474-frame real-mission slice that the L1 instrument scores. Never uploaded; derived frames only after face/plate review.	identifiable people — private, anonymised before use
SARD public	External reference for lying-person recall per size bucket — an outside yardstick for the rescue posture the detector must catch.	public research dataset, cited

METHODWhat we did with the data

Rendered a hybrid set through Cosmos Transfer 2.5. Isaac Replicator orbit clips plus masks, translated to photoreal on the B300, so the training domain carries real-world texture without leaving the lab.
Measured where synthetic generation breaks, and reported it. A segmentation-only control is under-constrained: a single-instance mask covers ~1% of the frame, only ~13% of labels survive the translation, and 0% keep the hi-vis vest — the one attribute the mission depends on. A static-subject control reaches ~0.90, so the mechanism works when adequately supplied. The fix is multi-control (edge + segmentation + depth), not turning the strength dial.
Ran a five-layer QC gate before any training. Hygiene and standby removal, perceptual-hash dedupe, a coverage table with its empty cells named, label QA against a teacher, and a distribution check against real frames — every source through the same gate, subsets logged with attribution.
Kept provenance. 139 render runs tracked into the published set, so any image can be traced to the run that made it.
Held the judge set out of training entirely. The mannequin capture (measured distances, lying positives) and the 474-frame real slice feed only the scorer — never a training loader.

CHALLENGESStill open

The render licence question. Whether rendering with NVIDIA SimReady assets constrains the licence of the published images. The one public thread was closed with "contact Enterprise Sales", so it needs an answer from inside NVIDIA — asked 11 Sep. Until then the split-licence note stands.
The judge set is not captured yet. The mannequin session supplies the lying positives with measured distances; the 474-frame slice's false-alarm labels are still empty (a ~40-minute human pass). Every downstream accuracy number waits on it.
Transfer generates negatives, not positives. Image-to-video Transfer reliably produces false and negative examples, but we could not steer the labels and transferred entities well enough to make labeled positives consistently — segmentation-only survives with 13% of boxes and 0% of the hi-vis vest, the feature that defines the target. Multi-control (edge + seg + depth) is the next render run, still unrun.
Cosmos 3 clip generation resists control. Cosmos3 Nano and Super generate video, but not to a specification — we found no reliable way to hold the view and composition, so clips come out with framings and layouts other than the ones the target scenes need.

UB Robotics · team UBR Stack · NVIDIA Open Models Codefest 2026

Datasets · 11 Sep