# MotionOrigin tomato-sorting experiment

Research record checked November 19, 2026. The work reconstructs the method of a physical SO-101 tutorial, then uses it to test a timing hypothesis on a simplified tomato workstation.

## Evidence and design decisions

| Status | Source | What it established | Decision |
| --- | --- | --- | --- |
| `observed_current` | [User-supplied SO-101 video at 8:44](https://www.youtube.com/watch?v=Q4p03s60MQQ&t=524s), supplied transcript SHA-256 `5e85d4ff1deee827eb2985ae26f185c41e6525720dbd8c7056bf97612fb349f6` | At 9:09–11:43 the presenter describes calibration. At 12:22–15:58 the presenter describes wrist and overhead views, capture faults, 50 teleoperated episodes, varied starts and six action coordinates. At 16:06–18:59 the presenter describes ACT training and SmolVLA fine-tuning. At 19:08–24:17 the presenter narrates retries, resets and sensitivity to lighting, orientation and height. | Record simulated observations and targets together. Use 50 episodes per seed and model fit. Evaluate on unseen matched initial cases and harder conditions. The video fetch was throttled. The workflow and rollout descriptions come from the provided transcript, with the recording/training workflow cross-checked against [the official LeRobot documentation](https://huggingface.co/docs/lerobot/il_robots). Video publication date, dataset ID, checkpoints, exact training settings and rollout success denominators remain unverified. |
| `historical_reference` | [Flash ALOHA project](https://johnyao93.github.io/flash-aloha/), [paper](https://johnyao93.github.io/static/files/flash-aloha.pdf?v=2), [capture](https://github.com/JohnYao93/flash-aloha/blob/aec35bb/src/contracts/standard_capture.py) and [policy chunks](https://github.com/JohnYao93/flash-aloha/blob/aec35bb/src/inference/policy_chunks.py#L357-L390), source commit `aec35bb7d29bf16fe360fd5abe71f976356b5a24` | Capture snapshots robot state, then selects a camera bundle and checks timing. The chunk scheduler enforces an overlap latency budget. Table III reports 55% whole-task completion for baseline battery insertion, 15% at double execution frequency, and 85% for Flash ALOHA. The pinned repository tree did not load through the browser tool. The individual source files and project paper were accessible. | Keep 30 Hz control fixed, vary the demonstration schedule, and log observation lag. Model delay compensation uses extrapolated encoder readings. The experiment does not reuse Flash ALOHA hardware, model weights or its chunk scheduler. |
| `historical_reference` | [RoboTryOn v3, Tables 1 and 2](https://arxiv.org/html/2611.05163v3), revision November 5, 2026 | Human video translated into robot-view demonstrations with action labels. The paper includes synthetic demonstrations alone at the policy fine-tuning stage and a retiming comparison on physical robots. At recorded human pace, OpenArm in-domain success was 75.9% and mean placement time 11.9 s. At 2× relative to teleoperation, the corresponding values were 57.1% and 7.4 s. | Strong precedent for the mechanism. Evaluate robot-view, action-labeled, phase-paced demonstrations. Distinguish target-task teleoperation from broader robot data already used to train the backbone and action labeler. |
| `historical_reference` | [EgoSpeedUp v2, Table I](https://arxiv.org/html/2609.29310v2), revision September 25, 2026 | The physical rope and block experiments report 33/40 successes after human-informed phase retiming and 22/40 for uniform 2×. The phase timing was applied to robot demonstrations. | Compare phase-aware timing and 2× using the same simulated task, with a joint correct-and-on-time endpoint. |
| `observed_current` | WP23 supplied generated clip, SHA-256 `56c5ffe0e0664dab7e52b5406837880b0d3bf002fe20ccef24ac5c4c510e9589`, and the concept timing record in `data/concept-produce-sort.json` | Visible palm support, roll, regrasp and separate release. The authored robot loop's second-tomato approach starts at 5.23 s and release ends at 12.78 s. | Human clip provides the task sequence. Its illustrated robot counterpart provides a 7.55-second test deadline. All camera crops, servo commands and disturbances used in the numerical experiment are generated independently from the assumed model. |
| `requires_owner_verification` | MotionOrigin owner’s description of the training run | The owner reports successful sorting at the fastest expert-human reference used. | Preserve it as a team report. Confirm with the weights or run ID, hardware and camera setup, unsped rollout, synchronized logs and matched expert benchmark. |
| `proposal` | MotionOrigin hardware preregistration outline | More correct, on-time sorts with phase-aware timing than 2× and shorter cycles than native timing without losing more than 10 percentage points in overall successful sorts. | Pilot calibration first, then fix the limit and measure test power using paired data. Run delayed-sensing and lighting shifts independently before combining them. |

The two reverse-engineering prompt attachments are byte-identical (SHA-256 `3f9523ab96059e1f72ed6bb09263346b99c656681e1a77e82779961d39d91262`). They were examined as source material. The user request defines the scope.

## Reproduce the numbers

From the workspace root, run with Python 3 and NumPy installed:

```sh
python3 research/tomato-sorting/simulate.py --output-dir /tmp/motionorigin-tomato
cmp research/tomato-sorting/results.json /tmp/motionorigin-tomato/results.json
cmp research/tomato-sorting/trials.csv /tmp/motionorigin-tomato/trials.csv
```

The simulator fits a ridge velocity controller from sampled teacher commands and a regularized logistic classifier from RGB-derived features. The classifier fit is shared across timing conditions and the teacher's input has the true route during data collection. During a test rollout, the route uses only the learned image classifier. Motion targets and phase boundaries are provided by the task model. There is no learned phase recognition, full hand dynamics, video-to-action model or RGB-only dexterity policy. Model dimensions, contact gates, actuator limits and latency define a sensitivity study and have not been calibrated to hardware.

Each reference or shifted condition has 360 matched cases, comprising 120 unseen cases for each of three independent data seeds. Pickup of moving fruit is simplified to an estimated interception, with belt drift assessed at grasp. Object placement uses a stability gate and a model-space tolerance, not contact force. A trial is scored from its first failure. For forensic diagnostics the integrator continues to the end of the attempted path, and geometry or time on an already-failed CSV row should not be interpreted as hardware telemetry. Route metrics count only cases that reach inspection. Interval estimates resample matched cases inside the fixed seeds and characterize Monte Carlo variability for this experiment.

The delay compensation follow-up was motivated by the first stress test. Reporting it on disjoint new cases, with the uncompensated condition rerun, avoids using the initial stress batch to score the fix. The stress tests combine multiple changes, so the follow-up tests the benefit of a specific correction within that combined scenario. It does not attribute every failure to a single variable.

The SVG on the study page shows the authored robot concept. The executable simulation and result tables provide the quantitative experiment.

## Motion and inspection follow-ups

- `historical_reference` · [Islam et al., RSS 2020](https://roboticsconference.org/2020/program/papers/25.html) studied online replanning from updated observations to intercept objects on a moving belt, including a real-robot test. MotionOrigin's reduced model forecasts the grasp position and updates it at the control rate. It does not implement that paper's planner.
- `observed_current` · The motion experiment uses 50 reference cases per model seed, six conditions and three disjoint test distributions. The moving-grasp intervention integrates delayed, noisy belt-speed readings. It predicts the arm's current position and velocity from delayed encoder measurements and issued commands under a single assumed time constant (0.137 s). The feedback gate uses these estimates. The evaluator separately verifies sustained alignment against the simulated true state. Expanded stress adds belt acceleration of −0.012 to +0.016 m/s², camera yaw bias of ±0.16 rad, and a synthetic edge occlusion. The controller uses a noisy and two-tick-delayed belt signal. It has no access to future encoder values or the test route label.
- `observed_current` · The inspection comparison fixes moving-grasp control, the two camera angles and the 180° target sweep. It contrasts 24×24 and 40×40 crops and a lower classifier cutoff fitted during development. Cutoffs are on the model's logit scale and have no physical probability interpretation. Each arm uses the same 50 training scenarios per seed and the same test fruit, with independent simulated pixel noise across different resolutions. Matched same-resolution arms share noise and geometry. Final seeds and their case offsets differ from the development and motion-comparison sets. Every case contributes to the recall denominator, including failures before inspection.
- `observed_current` · The first normalization pass improved overall sorting in motion tests while reducing end-to-end defect recall relative to moving-grasp alone. The independent inspection follow-up addressed that failure by increasing the visible pixel count and favoring rejection when crop evidence is weak. In expanded stress, on-time sorts rose from 324 to 351 of 360, correct defect routes rose from 144 to 174 of 180, and correct sound-fruit routes fell from 180 to 177 of 180.
- `proposal` · Hardware release requires measuring contact and stable object placement, then checking that camera magnification, exposure and motion blur leave the smallest target mark visible across viewpoints. Include real defects and lookalike cosmetic marks in blinded test labels. Actual gripper mechanics, reach limits, correlated crop misses, multi-fruit contention and retries are open validation items. The 7.55-second reference is the authored animation interval. At 45 mm/s, 7.55 seconds implies a nominal pitch of 0.340 m for one non-overlapping sort. Compare steady-state correct throughput separately on hardware.

### Re-run the follow-ups

```sh
python3 research/tomato-sorting/followup.py --output-dir /tmp/motionorigin-tomato
python3 research/tomato-sorting/inspection.py --output-dir /tmp/motionorigin-tomato
cmp research/tomato-sorting/followup-results.json /tmp/motionorigin-tomato/followup-results.json
cmp research/tomato-sorting/followup-trials.csv /tmp/motionorigin-tomato/followup-trials.csv
cmp research/tomato-sorting/inspection-results.json /tmp/motionorigin-tomato/inspection-results.json
cmp research/tomato-sorting/inspection-trials.csv /tmp/motionorigin-tomato/inspection-trials.csv
```

## Embodiment and contact observability

Checked November 19, 2026.

- `historical_reference` · [EgoEngine v1, Tables 3–4](https://arxiv.org/html/2606.12604v1), first posted June 10, 2026. Four physical-task averages are 3% for direct retargeting from human video, 5% with the visual branch added, 43% with the action branch added and 51% with both. Those outcomes are task- and pipeline-specific.
- `historical_reference` · [RoboTryOn v3](https://arxiv.org/html/2611.05163v3), revision November 5, 2026. Image-feature distance, action replay in simulated replicas and physical policy success are separate outcomes. Keyframes, hand-skeleton flow and an inverse-dynamics model trained on teleoperated robot data provide structured conditioning and pseudo-labels. Where this report gives the hand-skeleton ablation, it quotes the authors’ physical OpenArm in-domain average from Table 3.
- `historical_reference` · [EgoMimic](https://arxiv.org/html/2410.24221), [Phantom](https://arxiv.org/html/2503.00779) and [Supervise What Survives](https://arxiv.org/html/2606.24448v1) are evidence that human demonstrations can supply useful coverage or geometry once appropriate alignment or measured control is available. They do not compare sample efficiency on the MotionOrigin workcell.
- `historical_reference` · In [Calandra et al.](https://www.robertocalandra.com/papers/2018_calandra_ral.pdf), physical cup experiments report grip outcomes with vision and tactile feedback as well as vision alone. That real sensor comparison motivates an observability experiment with the same assumed visible cue across two mechanically different packages.
- `observed_current` · The new contact analysis holds load and actuator calibration identical within each pair. Its chosen friction and damage ranges isolate a grip decision for which the contact state can be ambiguous. A simplified Coulomb bound sets the safe minimum normal force. The sensor arm receives a simulated quantized onset-of-slip estimate with noise. The analysis scores a lift under this model and does not render video, train a dexterous policy, or estimate physical sample efficiency.

Run `python3 research/tomato-sorting/contact_observability.py --output-dir /tmp/motionorigin-contact` and compare the two generated files to `contact-results.json` and `contact-trials.csv`. Every condition accounts for all attempts, including slips and damage.

## Offline demonstration generation

Checked November 19, 2026.

- `historical_reference` · `research-dreamgen` · [DreamGen, original arXiv paper](https://arxiv.org/abs/2505.12705) and [the authors’ NVIDIA Research project page](https://research.nvidia.com/labs/gear/dreamgen/). Primary research preprint by Jang et al., first posted May 19, 2025. For the physical humanoid behavior study, the authors used real pick-and-place teleoperation to fit the video and inverse-dynamics models, then generated new task videos and estimated actions for policy training. The paper also includes separate simulation experiments. The reported humanoid outcomes support video-based training-data generation and give no direct estimate of MotionOrigin workstation setup cost.
- `historical_reference` · `research-robotryon` · [RoboTryOn v3, §3.4, §4 and Table 1](https://arxiv.org/html/2611.05163v3). Primary research preprint by Lee et al., first posted November 3, 2026 and revised November 5, 2026. The synthetic-only OpenArm fine-tuning condition averaged 48.5% across seven out-of-domain tasks, compared with 3.1% for teleoperation-only from in-domain tasks. The generator and action labeler had been trained on robot demonstrations and additional robot video. The authors used simulation separately to evaluate replay. Scope and conditions are specified for the quoted comparison.
- `observed_current` · The [offline export audit](robot-view-audit.md#offline-export-check) verified that the existing SVG and capture archive contain the same 2,196 camera images and that the archive’s 730 authored pose targets match the next observations. This checks file and timestamp integrity.
- `proposal` · Treat generated camera episodes as an alternative to building a full task simulator for data production. Compare physical performance and setup effort against equal-format offline data collected from interactive simulation. The public research items here are cited by link with original summaries in a local research preview. No third-party assets were incorporated.
