UBR Stack · team 15 · NVIDIA Open Models Codefest 2026

Codefest Blocker Board

Where we need something from you, ordered by what it costs us. Seven questions and one offer. Only two need a yes or a no: an ambiguity in the Transfer documentation, and a licence reading we have already acted on and now need confirmed or reversed. The offer is not a question at all — a 4B reasoner running in ~4,830 MB on an 8 GB edge board, with the number that went wrong reported next to the numbers that went right.

2 · need a yes or no

2 · costs throughput

3 · costs accuracy or planning

1 · offered, not asked

Stops work

The two we cannot resolve ourselves

Cosmos Transfer 2.5 — which control stack keeps Replicator's boxes valid?

Needs a yes or no

The constraint: We run Transfer 2.5 on Isaac orbit clips, and the renderer's COCO boxes must still describe the output. Our plan said segmentation control alone; the cookbook says seg should always be used with a mask and as part of multi-control, and the photoreal recipe is edge 1.0 + seg 0.9 + depth 0.9 at guidance 7.
The ask: Is that the right starting point for Replicator output where geometry must not move at all — or does edge at 1.0 suppress exactly the appearance change we are paying for? If edge must come down, what range still holds geometry? And since weights above 1.0 are renormalised to sum to 1.0, are the documented values pre- or post-normalisation? We would rather not sweep around a misread scale.
What we are doing anyway: A 13-clip × 3-strength sweep, scoring label survival two ways — an impartial detector, and Cosmos3-Super judging "person lying, and where" against the renderer's box. A correct starting configuration saves a day of B300 time.

Isaac-rendered images: we have taken a licence position and published it. Confirm it, or tell us to pull it.

Needs a yes or no

What is public, exactly: ubr-physical-ai/isaac-sdg-rescue-target is public and un-gated, with three revisions live: main, v2 (23,960 images, 14 classes, night and fog, out to 30 m) and v3 (37 webdataset shards, 18.68 GB, plus 109 masked 93-frame orbit clips). Hugging Face visibility is per-repository, not per-branch, so revision="v3" is downloadable by anyone today. 26 downloads so far. An earlier draft of this page said we were holding it. That was wrong, and this is the correction.
The position we took: The v2 and v3 cards split the licence rather than claiming CC BY 4.0 outright: annotations, metadata, configs and card CC BY 4.0 (UB Robotics); images rendered from NVIDIA-distributed assets, with the note that the Isaac Sim Additional Software and Materials License forbids redistributing assets and does not address rendered output. No source assets are included — renders only. The repository's own licence tag, however, still reads a flat cc-by-4.0, so the Hugging Face page advertises more than the cards do. That mismatch is ours to fix and we are fixing it.
The ask: Does that reading hold? A yes lets us align the repository tag to the cards and keep going. A no is equally useful and we act on it the same day: we can make the repository private, drop the affected revisions, or re-render with our own assets — tell us which and it is done. We would rather hear it from you now than from someone else in week 4.
Why we did not wait: Because the honest version of this question is not "may we?" but "we have, on this reading — is it right?" The public developer-forum thread asking exactly this was closed with "contact Enterprise Sales", and a public dataset plus protocol is one of our Codefest deliverables. Publishing and then asking is a position we can defend; asking while quietly already published is not.

Offered, not asked

One result we would like to hand over

Getting a 4B VLM onto an 8 GB Orin Nano was our job, not yours, and it is done — it runs, fast, and fits the 8 GB board with room to spare. It also gets the answers wrong in a specific and interesting way. Both halves are below, because the half that worked is not the useful half.

◆

Cosmos3-Edge INT4 on Orin Nano 8 GB — it runs. Here are the numbers, and the one failure we cannot diagnose.

Offer

What we built: INT4 AWQ quantisation and ONNX export on a B300, engine and visual_build on a bench Orin Nano at JetPack 7.2 / CUDA 13.2, TensorRT Edge-LLM v0.10.1 (e8b2952). Calibrated on 198 frames drawn from a frozen 256-frame slice of our own rover footage, disjoint from every set we later score on. 169 INT4 nodes verified in the built engine; build time 2 m 21 s.
What it costs on the board — the half that worked: llm.engine 838.9 MiB + visual.engine 942 MiB + embeddings 512 MiB = 2.24 GiB of artefacts. Peak resident ~4,830 MB for the full VLM (3,377 MB the LLM engine alone), against 7,485 MB usable. Decode 52.8 tok/s (18.92 ms/token), prefill 2,306 tok/s, visual encoder 259.8 ms at 896×448 — so camera frame to decision is 633 ms at 8 output tokens, 1,087 ms at 32. Earlier arms on the same board, for scale: NF4 2.67 GiB @ 8.67 tok/s · bf16 4.58 GiB @ 10.6 tok/s. One correction worth stating: it does not fit the 4 GB variant — even the 3,377 MB LLM engine alone exceeds the roughly 1,900–2,200 MB free after the OS.
What it gets wrong — the half that matters: Grounded F1 0.0 on a 24-item set; 0 of 12 targets found; 0 of 24 outputs obey the output schema. The failure is not truncation or parsing: results are identical at 192 and 512 token caps, 23 of 24 ended on end-of-sequence, and 24 of 24 are well-formed JSON with no fences. They ignore the contract — 10 omit task_relevance on entities, 14 omit certainty on attributes, several emit out-of-enum values, 3 hoist a field to the top level, 2 drop schema_version. The earlier NF4 arm on the same board scored F1 0.40 with 3 of 24 schema-valid — paired, that is a 3–0 discordance, exact p = 0.25, so we are not claiming AWQ is worse than NF4.
The two things we cannot find out ourselves: Is a 4-bit instruction-following regression known for this model? We cannot run the control that would settle it: full precision needs ~9 GiB and will not fit the 7.3 GB board, so we cannot ask whether bf16 honours the same contract on the same hardware. If you already know that a W4A16 AWQ build stops honouring a JSON schema the full-precision model honours, that closes the question today.
Does Edge-LLM expose constrained decoding at v0.10.1 — a logit processor, a grammar, anything that forces the output shape at decode time? If it does, the fix is hours rather than a training run, and it is the serving behaviour a rover needs regardless.
What we can and cannot hand over: The prompt template (frozen text) and the scorer (stdlib only) are yours if you want them, and they are enough to reproduce the failure on your own images. The 24 evaluation items are frames of identifiable people in a workplace, so they do not leave our bench and we cannot send them — metrics and structural diagnostics only. If the artefact is useful, tell us who should review it and we will write it up to whatever standard you need; we read the model card as OpenMDW 1.1 and take publishing a derived engine to be permitted.
Either way: This is reportable as it stands. "A 4B reasoner fits an 8 GB edge board at ~4,830 MB and 633 ms to a decision, and then fails on output contracts" is a more useful finding for anyone deploying these than a number that flattered us.

Costs throughput

We can work around these, slower

Is there an RT-core node in the cluster for Isaac Sim?

Open

Why we ask: Isaac Sim 6.0's requirements state that GPUs without RT cores (A100, H100) are not supported, so the B300 cannot render and all Isaac work runs on one RTX 4060 Ti 16 GB workstation of ours — below Isaac 6.0's own recommended tier. That box is now double-booked: the SDG render backlog and the closed-loop sim missions, where the renderer and the reasoner share the one card.
What it already costs us: Because renderer and brain share a GPU, that layer can report mission outcomes but cannot produce deployment latency figures — those come only from the Orin. We had to narrow what the layer is allowed to claim, and retire a sim-to-real subtraction we had designed the evaluation around.
The ask: Is an L40S or RTX PRO 6000 node available, or is local-render plus cluster-train the expected split? Without a node we sequence the two and document it — we would rather not discover that in week 3.

One B300, three GPU-bound jobs queued

Open

Serialising on the single card: The Transfer 2.5 strength sweep; the round-2 4-bit accuracy arms (same 986 pairs and harness as round 1, so the quantisation delta is measured on matched hardware); and one Cosmos3-Super pass as teacher and QC judge over the Transfer output.
The ask: Is a second card available for the Transfer week? Smaller, same topic: our detector sweeps turned out CPU-bound rather than GPU-bound (16 CPUs, load 16.7), so we now submit with --cpus-per-task=32. Is there a per-task CPU ceiling, or a queue convention we should be using better?
And one we should have raised sooner: dgx02 has been sitting in IDLE+DRAIN, and our team storage lives on it. We do not know whether that is planned maintenance or a fault, and we have been working around it instead of asking — which was the wrong call. Is it coming back, and should we be staging our data somewhere else?

Costs accuracy or planning time

Would help, and shapes what we build next

A small-scale or LoRA SFT configuration for Cosmos3-Edge

Open

What we are aiming at: The SFT target moved on 11 Sep. L0 shows 4-bit costs nothing once calibrated; the perception false-alarm rate is under re-measurement on the 474-frame slice. Any SFT budget now goes to the operator-language side: the August tool-calling gate (0 of 21) re-run under the conversational prompt that carried commands to 13/15, and an UNKNOWN reply so the model can decline rather than force a command. A text-only Gemma E2B (14/14 including Portuguese) is the measured fallback if training is not ready in time.
Why the published path does not fit: The post-training write-up we found is 76k DROID trajectories on 64 nodes of 4× GB200, ~17.4K GB200-hours — and it goes through cosmos-framework, which we had to drop from our container because it pins transformers < 5 and breaks vLLM and the guardrail.
The ask: Is there a LoRA or small-data SFT configuration for the 2.4B/4B reasoner that is known not to damage what it already does well (65.2 % on left/right)? And does post-training invalidate the INT4 path — requantise, re-export and rebuild, or can an existing engine be reused?

Rover-height person data — anything at ~0.5 m, ideally prone?

Open

Checked and ruled out for that viewpoint: Spatial-Intelligence-Warehouse, SmartSpaces 2026 (including both real warehouse captures), NuRec, SDG-SynHuman, Embodied-Robot-Scenes — all ceiling, drone or humanoid viewpoint. We use them as negatives and for spatial QA, and render the positives ourselves.
The ask: Is there a PhysicalAI subset at ground-robot height with low or prone poses that we have missed?
Why we care: Our real-side instrument is a frozen 474-frame slice of past rover missions (max 2 frames per segment, minimum dHash Hamming 12 across the selection). It is dense in standing and walking people and almost empty of lying ones — hence a mannequin capture to supply the positives and the measured distances.

Showcase format for GTC Berlin

Open

The ask: How long is the top-5 slot, and is it a live demo, a recorded demo or slides?
What it changes: Week 4. If a live run is viable we bring the rover and pull the network cable on stage; if it is recorded, we shoot in week 3 and spend week 4 on the numbers.