Suite index
Search and filter the named suites. Open a row for the identity sheet. Do not rank scores across classes.
Showing 28 suites. Scale units stay in their own class; this is not a leaderboard.
| Suite | Role | First public | Published scale | Medium | Next action |
|---|---|---|---|---|---|
| RLBench | sim manipulation eval | arXiv 2019-09-26 [R1] | 100 unique hand-designed tasks [R1] | simulation [R1] | Open sheet |
| CALVIN | sim manipulation eval | arXiv 2021-12-06 [R3] | 4 environments; 34 subtasks versus 18 prior; ~24 h play data; 20K language directives [R3] | simulation [R3] | Open sheet |
| LIBERO | sim manipulation eval | arXiv 2023-06-05 [R5] | 130 language-conditioned tasks in 4 suites (Spatial, Object, Goal: 10 tasks each; LIBERO-100) [R5] | simulation [R5] | Open sheet |
| ManiSkill2 | sim manipulation eval | arXiv 2023-02-09 [R7] | 20 task families; 2000+ objects; 4 million+ demonstration frames [R7] | simulation [R7] | Open sheet |
| Meta-World | sim manipulation eval | arXiv 2019-10-24 [R9] | 50 distinct manipulation tasks [R9] | simulation [R9] | Open sheet |
| RoboCasa | sim manipulation eval | arXiv 2024-06-04 [R11] | 100 tasks (25 atomic + 75 composite); 120 kitchen scenes; 2,509 objects; 153 categories; 100K trajectories [R11] | kitchen simulation [R11] | Open sheet |
| VLABench | vla manipulation eval | arXiv 2024-12-24 [R32] | 100 task categories: 60 primitive + 40 composite [R32] | simulation [R32] | Open sheet |
| RoboTwin 2.0 | vla manipulation eval | arXiv 2025-06-22 [R34] | 50 dual-arm tasks in the paper; the project describes over 50 supported tasks [R34] [R35] | simulation benchmark and data generator with a separate sim-to-real study [R34] | Open sheet |
| Google Robot | real to sim eval | arXiv 2024-05-09 [R13] | ~1,500 paired sim-and-real evaluation episodes across both embodiments in the paper, not a per-robot task count [R13] | paired real and simulated evaluation [R13] | Open sheet |
| WidowX | real to sim eval | arXiv 2024-05-09 [R13] | ~1,500 paired sim-and-real evaluation episodes across both embodiments in the paper, not a per-robot task count [R13] | paired real and simulated evaluation [R13] | Open sheet |
| FurnitureBench | real furniture assembly | arXiv 2023-05-22 [R15] | 8 furniture models; 5,000+ teleoperation demonstrations; 200+ hours [R15] | real [R15] | Open sheet |
| BEHAVIOR-1K | everyday activity sim | arXiv 2024-03-14 [R16] | 1,000 activities; 50 scenes; 9,000+ objects [R16] | simulation [R16] | Open sheet |
| DexGraspNet | grasp synthesis dataset | arXiv 2022-10-06 [R39] | 1.32 million grasps; 5,355 objects [R39] | synthetic [R39] | Open sheet |
| DexGraspNet 2.0 | grasp synthesis dataset | arXiv 2024-10-30 [R40] | 1,319 objects; 8,270 scenes; 427 million grasps [R40] | synthetic clutter [R40] | Open sheet |
| GAPartNet | articulated part dataset | arXiv 2022-11-09 [R41] | 9 GAPart classes; 27 object categories; 8,489 part instances; 1,166 objects [R41] | dataset of articulated objects [R41] | Open sheet |
| ALFRED | household instruction sim | arXiv 2019-12-03 [R18] | 25,743 English language directives; 8,055 expert demonstrations; 120 indoor scenes [R18] | simulation (AI2-THOR) [R18] | Open sheet |
| Open6DOR | open instruction rearrangement | IROS 2024 Oral [R38] | — | real rearrangement evaluation as described on the project page [R38] | Open sheet |
| Room-to-Room | vision language navigation | arXiv 2017-11-20 [R19] | 21,567 instructions; average 29 words [R19] | real Matterport3D indoor imagery [R19] | Open sheet |
| EVT-Bench | embodied visual tracking | arXiv 2025-05-29 [R19] | 25,986 episodes; 100 avatars; 804 scenes (HM3D + MP3D); train 21,771 / test 4,215; three sub-tasks [R19] | simulation (HM3D and MP3D scenes) [R19] | Open sheet |
| OmniSpatial | spatial vlm benchmark | arXiv 2025-06-03 [R37] | 4 categories; 50 subcategories; 8.4K+ QA pairs [R37] | image / VQA benchmark [R37] | Open sheet |
| OpenEQA | embodied question answer | CVPR 2024 [R21] | 1,600+ human-generated questions; 180+ real-world environments [R21] | real environments (episodic memory and active exploration settings) [R21] | Open sheet |
| HumanTracker | humanoid motion tracking bench | arXiv 2026-08-24 [R36] | ~153 hours optical mocap; 4 motion families; ~25K clips; HumanScore trained on 12K pairs / 24K motions [R36] | optical motion capture retargeted to a simulated humanoid [R36] | Open sheet |
| HumanoidBench | humanoid wbc sim | arXiv 2024-03-15 [R22] | 15 manipulation + 12 locomotion tasks (27 tasks) [R22] | simulation [R22] | Open sheet · Open world |
| Open X-Embodiment | robot training corpus | arXiv 2023-10-13 [R24] | 1M+ trajectories; 22 embodiments; 60 datasets; 34 labs; 527 skills; 160266 tasks (body). Abstract: 22 robots, 21 institutions, 527 skills, 160266 tasks [R24] | real robot trajectories pooled from many datasets [R24] | Open sheet |
| DROID | robot training corpus | arXiv 2024-03-19 [R26] | 76k trajectories; 350 hours; 564 scenes; 86 tasks [R26] | real teleoperated trajectories [R26] | Open sheet |
| BridgeData V2 | robot training corpus | arXiv 2023-08-24 [R28] | 60,096 trajectories; 24 environments [R28] | real demonstrations [R28] | Open sheet |
| EI-30k | robot training corpus | arXiv 2026-02-16 [R9] | 30k+ hours mixed-quality human and robot data [R9] | mixed real, sim, labeled, and unlabeled human and robot data [R9] | Open sheet |
| SynGrasp-1B | robot training corpus | arXiv 2025-05-06 [R17] | 1 billion synthetic VLA frames [R17] | synthetic [R17] | Open sheet |
World support
237 previews across 13 families; 91 with recorded replay. Homes in one family and partial scenes are listed separately. A recording is not an evaluated benchmark result.
Showing 237 published courses. Animation is not a recorded replay or an evaluated result.
| World | Scene | Suite | Scene family | Runtime | Embodiment | Success | Replay origin | Validation | Catalogued | Preview | Recorded replay | Evaluated result | Reproduced | Next action |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Stair | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 700 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Hurdle | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 700 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Maze | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 1200 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Pole | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 700 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Slide | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 700 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Crawl | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 700 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| High bar | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 | source-parsed · bar 750 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Cabinet | Full scene | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 2500 | site-generated | site-generated recorded-rollout; natively_verified_tasks stays 0 | Yes | Yes | Yes | Not available | Not available | Open world |
| Door | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 600 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Window | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 650 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Room | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 400 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Truck | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 3000 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Basketball | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 1200 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Spoon | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 650 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Insert | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 350 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Push | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 700 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Package | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 1500 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Cube | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 370 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Powerlift | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 800 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Sit (hard) | Preview | HumanoidBench | Unknown | mujoco 3.1.6 | Unitree H1 with two Shadow Hands | source-parsed · bar 750 | none | fragment preview only | Yes | Yes | Not available | Not available | Not available | Open world |
| Wooden cabinet | Full scene | LIBERO | 797b1c3e6c70cdc6e64ae1715b9bf411c403fc882d070ed0b94a7cf37ed9dd29 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| Bowl between plate and ramekin | Full scene | LIBERO | 797b1c3e6c70cdc6e64ae1715b9bf411c403fc882d070ed0b94a7cf37ed9dd29 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | site-generated | reconstructed reset scene; one learned-policy trial with native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 0 | Full scene | LIBERO | f0ab3a7d0a1feecb80c94689ca7db848c1b0b866fcab20881b7454522d977370 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 1 | Full scene | LIBERO | 12dce7a20d394134d199fb8bd29b173748cfdaf6f469ac284587137ef813c9cd | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 2 | Full scene | LIBERO | 8f7f4229445aab69fed1ad151c44535bdff4f7827fcc6b876bf76cb06afdf14a | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 3 | Full scene | LIBERO | ec94854a66408a6449e072c372c5b6131eca4bf9f4b5d0587677d07eb1bdcce5 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 4 | Full scene | LIBERO | 95aa1c1bca0d78c9a9b59a2f6d3957ffc32b0aedccda923823a178015dac2686 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 5 | Full scene | LIBERO | db80edc0b5ba121f8b1a7cd82caf080e41d59fd0f378656edb234dfb46ac4e0d | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 6 | Full scene | LIBERO | 421faace152453b6a33e71652f411d7656560da3d99af1c68b1e2f288c978f0c | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 7 | Full scene | LIBERO | 834c3312132f11d3e526ba6e45be319b9b94f3decbd0b8f0cd054b7ba18599fb | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 8 | Full scene | LIBERO | 9251b0bd85fb2f0bf0ed4bad4cb20d51569e69afd1a25063e8e64bb936fe7293 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Object task 9 | Full scene | LIBERO | af36da8ca8c6f0aac6b281005f42b1f730f4b3bcd44d27ecee9550c865159491 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-Goal task 0 | Full scene | LIBERO | 6ef1aef9ae8799327aca513809629188ebc56407887cfb689348891eeb114038 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 0 · KITCHEN_SCENE10 | Full scene | LIBERO | b4a31ee3b4d843d36ae21b7db427f33ced737f1c2f613963b7fecb75b04319a2 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 6 · KITCHEN_SCENE1 | Full scene | LIBERO | 8c70e8f67f40be5fc729ba133e92197981054a56104f7af59c60a8e3afda9876 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 11 · KITCHEN_SCENE2 | Full scene | LIBERO | 215733e99c6b9eb3701f9a92f30d3efd3548ad0ab81a6a9442df983975c3c7a7 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 18 · KITCHEN_SCENE3 | Full scene | LIBERO | dfdce5b2847731110725dbf911be969ab21ffe338f5b96825276e2393605e1ea | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 22 · KITCHEN_SCENE4 | Full scene | LIBERO | 820ed396f14930b2c987d442b189e2988151c32cc2b5f821e19d0d29ca19ffed | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 28 · KITCHEN_SCENE5 | Full scene | LIBERO | fc6c1fa47cd82b54dfe27c2e7087f3a7c5c349464476d092c5a02d1c1ce2f9d6 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 33 · KITCHEN_SCENE6 | Preview | LIBERO | d7c7b2047b9a940b807beb8ae881f749ecff06b9b6089379b5bdf0c36e006162 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | partial scene: some visible geoms are not drawn; native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 35 · KITCHEN_SCENE7 | Preview | LIBERO | 02998e2a52f6db1536a2f791bf828fe70acbf44c4f91e826948a4bed401a1e18 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | partial scene: some visible geoms are not drawn; native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 38 · KITCHEN_SCENE8 | Full scene | LIBERO | dc5a92707c5952a75b23ab6fe7e8a54cd7edc6ce2172195c50981d624a909f40 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 40 · KITCHEN_SCENE9 | Full scene | LIBERO | eb9b67152cd17b83c4fd723203513cacff236ca979af4c3c1ede5dffbac97a89 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 46 · LIVING_ROOM_SCENE1 | Full scene | LIBERO | 641da6dc258672723e585d2591622e6bdbf419b2fa68a21606d6908873e68c7d | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 50 · LIVING_ROOM_SCENE2 | Full scene | LIBERO | 2b8ca95c9d37c392dc474a3a6d2fbe91b80cc1d6b9e7e296d23c56e31a1bf22f | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 55 · LIVING_ROOM_SCENE3 | Full scene | LIBERO | f9c1fe10003f4b6fed056f8347786d609528c178237e3cc28367e55aa4d852b0 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 60 · LIVING_ROOM_SCENE4 | Full scene | LIBERO | 57c6352dac11d127e8b3edfe5b60081efe193151dcfb7bb73c321b792c52eb8c | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | none | native set_init_state scene exported; no recording bound; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| LIBERO-90 task 65 · LIVING_ROOM_SCENE5 | Full scene | LIBERO | 1a0e8d5ac22568e478022c76e5292ba1579c5067056e133e07e368912e8f513b | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 69 · LIVING_ROOM_SCENE6 | Full scene | LIBERO | df6a6a118ecd251658f61f41b3856cc4547edde8f1e5b1cb6e0f0aca2f2b3f3b | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 73 · STUDY_SCENE1 | Full scene | LIBERO | 8aee652876d468162485217303735e0650e44b0b48e1ca277292c6b1eea23d70 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 77 · STUDY_SCENE2 | Full scene | LIBERO | a6ea6a540363843e0bb565fac0a14eea180a195c30791634cf2155587ffb2f07 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 81 · STUDY_SCENE3 | Full scene | LIBERO | c23df364d416b4f4c5fe7cb78627936a264b1067bc62b3548dfddc93d7740058 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| LIBERO-90 task 86 · STUDY_SCENE4 | Full scene | LIBERO | bfd54261e480df93c3ec6b306fc55ffc56d9fc754c422991a632f7a7290d6019 | robosuite 1.4.0 / robosuite 1.4.0 | Franka Emika Panda | ControlEnv.check_success delegates to the native task _check_success. set_init_state calls set_state, sim.forward, check_success, _post_process, and _update_observables. Parsed from env_wrapper.py, not evaluated. A visual assembly is not a success signal. Do not reconstruct a reward stub. | upstream | native set_init_state scene exported; official demo_0 state replay ends in native check_success; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| Lift | Full scene | robosuite | 90d49dfbd80649c2aea909f980f39e2fa755d435db01d51214c48c8ec562da44 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | source-parsed · bar 0.04 | upstream | native reset export; published file compiled against native body poses and geom frames; shipped human demonstration as state replay with native success recorded; not a benchmark score; scene family 90d49dfbd806… | Yes | Yes | Yes | Not available | Not available | Open world |
| Stack | Full scene | robosuite | 3c62680d6ff5763e37502e9ee58f4de53ea0931db2fc586142260cd3daa7b982 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | Stack._check_success returns True when staged_rewards() r_stack > 0 (cubeA lifted, touching cubeB, not grasped). robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 3c62680d6ff5… | Yes | Yes | Not available | Not available | Not available | Open world |
| Nut assembly | Full scene | robosuite | 93033e81e8a60a97201aeba80c2a5ad12e5d90617a009f2364f5e54a108cbc75 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | NutAssembly._check_success returns True when every nut is on its peg and released (r_reach < 0.6); single-object modes need one nut on its peg. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 93033e81e8a6… | Yes | Yes | Not available | Not available | Not available | Open world |
| Nut assembly · one random nut | Full scene | robosuite | 93033e81e8a60a97201aeba80c2a5ad12e5d90617a009f2364f5e54a108cbc75 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | NutAssembly._check_success returns True when every nut is on its peg and released (r_reach < 0.6); single-object modes need one nut on its peg. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 93033e81e8a6… | Yes | Yes | Not available | Not available | Not available | Open world |
| Nut assembly · square nut | Full scene | robosuite | 93033e81e8a60a97201aeba80c2a5ad12e5d90617a009f2364f5e54a108cbc75 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | NutAssembly._check_success returns True when every nut is on its peg and released (r_reach < 0.6); single-object modes need one nut on its peg. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 93033e81e8a6… | Yes | Yes | Not available | Not available | Not available | Open world |
| Nut assembly · round nut | Full scene | robosuite | 93033e81e8a60a97201aeba80c2a5ad12e5d90617a009f2364f5e54a108cbc75 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | NutAssembly._check_success returns True when every nut is on its peg and released (r_reach < 0.6); single-object modes need one nut on its peg. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 93033e81e8a6… | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick and place | Full scene | robosuite | 9ae4baa247d73c3fb4eacdb4c49595462ac525ef449eb9947b9f2298ae161f32 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | PickPlace._check_success returns True when every object is inside its bin and released (r_reach < 0.6); single-object modes need one object in its bin. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 9ae4baa247d7… | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick and place · one random object | Full scene | robosuite | 9ae4baa247d73c3fb4eacdb4c49595462ac525ef449eb9947b9f2298ae161f32 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | PickPlace._check_success returns True when every object is inside its bin and released (r_reach < 0.6); single-object modes need one object in its bin. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 9ae4baa247d7… | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick and place · milk | Full scene | robosuite | 9ae4baa247d73c3fb4eacdb4c49595462ac525ef449eb9947b9f2298ae161f32 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | PickPlace._check_success returns True when every object is inside its bin and released (r_reach < 0.6); single-object modes need one object in its bin. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 9ae4baa247d7… | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick and place · bread | Full scene | robosuite | 9ae4baa247d73c3fb4eacdb4c49595462ac525ef449eb9947b9f2298ae161f32 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | PickPlace._check_success returns True when every object is inside its bin and released (r_reach < 0.6); single-object modes need one object in its bin. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 9ae4baa247d7… | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick and place · cereal | Full scene | robosuite | 9ae4baa247d73c3fb4eacdb4c49595462ac525ef449eb9947b9f2298ae161f32 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | PickPlace._check_success returns True when every object is inside its bin and released (r_reach < 0.6); single-object modes need one object in its bin. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 9ae4baa247d7… | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick and place · can | Full scene | robosuite | 9ae4baa247d73c3fb4eacdb4c49595462ac525ef449eb9947b9f2298ae161f32 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | PickPlace._check_success returns True when every object is inside its bin and released (r_reach < 0.6); single-object modes need one object in its bin. This registered variant runs in a single-object mode, so one correctly placed object satisfies it. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 9ae4baa247d7… | Yes | Yes | Not available | Not available | Not available | Open world |
| Door | Full scene | robosuite | c3e3c8b6c7026c3c63f7c6f3cde6465b1aa373a2991da4eda2f118f86a5b04e0 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | source-parsed · bar 0.3 | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family c3e3c8b6c702… | Yes | Yes | Not available | Not available | Not available | Open world |
| Wipe | Full scene | robosuite | 34da968e6bdcd78496ea152f5597d37783016fce59c2c53d7e487e0bdb3e2f53 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | Wipe._check_success returns True when len(wiped_markers) == num_markers. The marker set is updated by reward() during env.step, so it cannot be read from a restored state. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 34da968e6bdc… | Yes | Yes | Not available | Not available | Not available | Open world |
| Two-arm lift | Full scene | robosuite | 0b87ac8404537633b82afae4cd72431c1411c62809b0fc86535345ebe1b82029 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | source-parsed · bar 0.1 | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 0b87ac840453… | Yes | Yes | Not available | Not available | Not available | Open world |
| Two-arm peg in hole | Full scene | robosuite | 54316c03d39b6f9e1fa18ce284d7c051c3c59a52d9762700e67e7c237f5198f1 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | TwoArmPegInHole._check_success returns True when d < 0.06 and -0.12 <= t <= 0.14 and cos > 0.95. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 54316c03d39b… | Yes | Yes | Not available | Not available | Not available | Open world |
| Two-arm handover | Full scene | robosuite | 5fe072da4c169ef341c116a47106ea2d3b85baf1c7a302f376c8956b7876a5b6 | robosuite 1.4.0 / MuJoCo 3.1.6 | Franka Emika Panda | TwoArmHandover._check_success returns True when arm1_grasp_handle and not arm0_grasp_any and hammer_height - table_height > height_threshold. robosuite environments do not terminate on success; they run to the horizon. Parsed from the pinned source. A reset scene is not a scored run. | none | native reset export; published file compiled against native body poses and geom frames; no recording; not a scored run; scene family 5fe072da4c16… | Yes | Yes | Not available | Not available | Not available | Open world |
| Reach | Full scene | Meta-World | de0d14b91b2df41159a83477704ea578f50521eb17aa02f4c978723aae74ce26 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family de0d14b91b2d… | Yes | Yes | Yes | Not available | Not available | Open world |
| Push | Full scene | Meta-World | de0d14b91b2df41159a83477704ea578f50521eb17aa02f4c978723aae74ce26 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family de0d14b91b2d… | Yes | Yes | Yes | Not available | Not available | Open world |
| Pick place | Full scene | Meta-World | de0d14b91b2df41159a83477704ea578f50521eb17aa02f4c978723aae74ce26 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family de0d14b91b2d… | Yes | Yes | Yes | Not available | Not available | Open world |
| Door open | Full scene | Meta-World | ebcbd573f23415110d1297a196ffda3a9a9fa711739b8d93a5a04d05c4df4ee3 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.08 | site-generated | native reset-state measured; browser preview; not a scored run; scene family ebcbd573f234… | Yes | Yes | Yes | Not available | Not available | Open world |
| Drawer open | Full scene | Meta-World | debd830ac07cfddd6ed7b97a20c49ee4dc2e35aac68b4c78bdb3678183030674 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.03 | site-generated | native reset-state measured; browser preview; not a scored run; scene family debd830ac07c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Button press | Full scene | Meta-World | 6377e72af839674cef70e72623449afc755a154347824528457977fbd0b52542 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 6377e72af839… | Yes | Yes | Yes | Not available | Not available | Open world |
| Window open | Full scene | Meta-World | a2601fdd6fff88eec9ba3cb8fd2e716a7abbcacba34114a749ced611b3e45093 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family a2601fdd6fff… | Yes | Yes | Yes | Not available | Not available | Open world |
| Faucet open | Full scene | Meta-World | a1fd491ee4882119c678429509cc920ba0f81830a82cef4c2ff24eb90a6ea4d0 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family a1fd491ee488… | Yes | Yes | Yes | Not available | Not available | Open world |
| Peg insert side | Full scene | Meta-World | 0028b47cb59b2188783868e0071eddf43751f94d0a876be17f18d1d567db19e9 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 0028b47cb59b… | Yes | Yes | Yes | Not available | Not available | Open world |
| Sweep into | Full scene | Meta-World | 34f004ae3c50d83a20cb72eb725cc5e16c0731560c80a910c284303ade992f62 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 34f004ae3c50… | Yes | Yes | Yes | Not available | Not available | Open world |
| Basketball | Full scene | Meta-World | c96cd6d3f3835e7bbd5c16da45422479a65e4e1fce0f7044da9aced09ddb5c22 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.08 | site-generated | native reset-state measured; browser preview; not a scored run; scene family c96cd6d3f383… | Yes | Yes | Yes | Not available | Not available | Open world |
| Hammer | Full scene | Meta-World | e758a850e93b809ec2b2658e02287fb53cf8fe0a95f4245af5a5254dec2aa5d0 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.09 | site-generated | native reset-state measured; browser preview; not a scored run; scene family e758a850e93b… | Yes | Yes | Yes | Not available | Not available | Open world |
| Dial turn | Full scene | Meta-World | ccf6b632ec734252b4f009fe9f0d5b536c485e0af6f7c595697215cb20d5cdd6 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family ccf6b632ec73… | Yes | Yes | Yes | Not available | Not available | Open world |
| Lever pull | Full scene | Meta-World | c18534c6579b0547dcc4304c9f2275cc5772ee5d76e6c1acb7b6a36a280f3cc8 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.1308996938995747 | site-generated | native reset-state measured; browser preview; not a scored run; scene family c18534c6579b… | Yes | Yes | Yes | Not available | Not available | Open world |
| Box close | Full scene | Meta-World | f0eb6ac64895785a0ddc80d9e1ce94318f9c022df1080dc2ccc02a94fff110e5 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.08 | site-generated | native reset-state measured; browser preview; not a scored run; scene family f0eb6ac64895… | Yes | Yes | Yes | Not available | Not available | Open world |
| Assembly | Full scene | Meta-World | c9be9e861a2179ae65348b2cacf36b51753a4cdbc0861f50b78289c76f790732 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family c9be9e861a21… | Yes | Yes | Yes | Not available | Not available | Open world |
| Bin picking | Full scene | Meta-World | f71ec4465771bc1aefd4e8f8364ecebea43075e5030192671d65dca31d7b8814 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family f71ec4465771… | Yes | Yes | Yes | Not available | Not available | Open world |
| Button press topdown | Full scene | Meta-World | 13c41dee577de4547b16df9ff181c71690f0701439a6014eadbff84abc429850 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.024 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 13c41dee577d… | Yes | Yes | Yes | Not available | Not available | Open world |
| Button press topdown wall | Full scene | Meta-World | 27c9a4bc4024c3c09d6b74167728e9b5fd8fe475ebd34320b1adcc821cbcf64a | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.024 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 27c9a4bc4024… | Yes | Yes | Yes | Not available | Not available | Open world |
| Button press wall | Full scene | Meta-World | 5090c24a5ddb7792df04b64fdb93e8b40e6539836a65149387e143319b514932 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.03 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 5090c24a5ddb… | Yes | Yes | Yes | Not available | Not available | Open world |
| Coffee button | Full scene | Meta-World | 062547b6628c1afcad39d6831e5c1940842903546585007c688573d0a009497c | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 062547b6628c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Coffee pull | Full scene | Meta-World | 062547b6628c1afcad39d6831e5c1940842903546585007c688573d0a009497c | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 062547b6628c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Coffee push | Full scene | Meta-World | 062547b6628c1afcad39d6831e5c1940842903546585007c688573d0a009497c | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 062547b6628c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Disassemble | Full scene | Meta-World | c9be9e861a2179ae65348b2cacf36b51753a4cdbc0861f50b78289c76f790732 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | SawyerNutDisassembleEnvV3.evaluate_state sets success when obs[6] > target_pos[2]: the nut is above the target height; no distance threshold. Base SawyerXYZEnv.step returns terminated=False even at success and truncates at 500. AutoTerminateOnSuccessWrapper defaults terminate_on_success=False on MT1. A scripted-policy success flag is not a published benchmark score. Parsed from source, not a paper v1/v2 recipe merge. | site-generated | native reset-state measured; browser preview; not a scored run; scene family c9be9e861a21… | Yes | Yes | Yes | Not available | Not available | Open world |
| Door close | Full scene | Meta-World | ebcbd573f23415110d1297a196ffda3a9a9fa711739b8d93a5a04d05c4df4ee3 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.08 | site-generated | native reset-state measured; browser preview; not a scored run; scene family ebcbd573f234… | Yes | Yes | Yes | Not available | Not available | Open world |
| Door lock | Full scene | Meta-World | a80c2e483b396a1d213ed7aad1fbeb81c4cc0261e224577ebbc8bdbf5d7e8987 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family a80c2e483b39… | Yes | Yes | Yes | Not available | Not available | Open world |
| Door unlock | Full scene | Meta-World | a80c2e483b396a1d213ed7aad1fbeb81c4cc0261e224577ebbc8bdbf5d7e8987 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family a80c2e483b39… | Yes | Yes | Yes | Not available | Not available | Open world |
| Drawer close | Full scene | Meta-World | debd830ac07cfddd6ed7b97a20c49ee4dc2e35aac68b4c78bdb3678183030674 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.065 | site-generated | native reset-state measured; browser preview; not a scored run; scene family debd830ac07c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Faucet close | Full scene | Meta-World | a1fd491ee4882119c678429509cc920ba0f81830a82cef4c2ff24eb90a6ea4d0 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family a1fd491ee488… | Yes | Yes | Yes | Not available | Not available | Open world |
| Hand insert | Full scene | Meta-World | 34f004ae3c50d83a20cb72eb725cc5e16c0731560c80a910c284303ade992f62 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 34f004ae3c50… | Yes | Yes | Yes | Not available | Not available | Open world |
| Handle press side | Full scene | Meta-World | b820e36794772326a37aa000dfecafc9e06cb84aa4c6fdccd28bf426e034bb86 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family b820e3679477… | Yes | Yes | Yes | Not available | Not available | Open world |
| Handle press | Full scene | Meta-World | 3c329de9b41c232b050335549350e282e5c6c1ac9c9492218d90227d32c1c578 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.02 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 3c329de9b41c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Handle pull side | Full scene | Meta-World | b820e36794772326a37aa000dfecafc9e06cb84aa4c6fdccd28bf426e034bb86 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.08 | site-generated | native reset-state measured; browser preview; not a scored run; scene family b820e3679477… | Yes | Yes | Yes | Not available | Not available | Open world |
| Handle pull | Full scene | Meta-World | 3c329de9b41c232b050335549350e282e5c6c1ac9c9492218d90227d32c1c578 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 3c329de9b41c… | Yes | Yes | Yes | Not available | Not available | Open world |
| Peg unplug side | Full scene | Meta-World | 7b11a368d5ae8934e2cbbae901b25074f65e83410a24998b05c6e8832d0115f6 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 7b11a368d5ae… | Yes | Yes | Yes | Not available | Not available | Open world |
| Pick out of hole | Full scene | Meta-World | 6f05bf0c51282c9cb173f5c3b6f7945d9b5df2a8c2c52292cd156355fd7ba4c8 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 6f05bf0c5128… | Yes | Yes | Yes | Not available | Not available | Open world |
| Pick place wall | Full scene | Meta-World | 4c2524ca2f9e19100b8382a4fd5493e118348658e2222bcec5e9bef45447706a | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 4c2524ca2f9e… | Yes | Yes | Yes | Not available | Not available | Open world |
| Plate slide back side | Full scene | Meta-World | b9673e9ae8d6de3278ca51378df05a4c8295575ec153dab2f029e551ccbe0f05 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family b9673e9ae8d6… | Yes | Yes | Yes | Not available | Not available | Open world |
| Plate slide back | Full scene | Meta-World | aa1174e6b32956cc1258bd5fc7e407b98d06992cfa303a803c2badfe34244a93 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family aa1174e6b329… | Yes | Yes | Yes | Not available | Not available | Open world |
| Plate slide side | Full scene | Meta-World | b9673e9ae8d6de3278ca51378df05a4c8295575ec153dab2f029e551ccbe0f05 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family b9673e9ae8d6… | Yes | Yes | Yes | Not available | Not available | Open world |
| Plate slide | Full scene | Meta-World | aa1174e6b32956cc1258bd5fc7e407b98d06992cfa303a803c2badfe34244a93 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family aa1174e6b329… | Yes | Yes | Yes | Not available | Not available | Open world |
| Push back | Full scene | Meta-World | b425a791f61471bed98e80f57a830226930a5ebfe811fbe3129ec7f759682c91 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family b425a791f614… | Yes | Yes | Yes | Not available | Not available | Open world |
| Push wall | Full scene | Meta-World | 4c2524ca2f9e19100b8382a4fd5493e118348658e2222bcec5e9bef45447706a | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 4c2524ca2f9e… | Yes | Yes | Yes | Not available | Not available | Open world |
| Reach wall | Full scene | Meta-World | d64e388cb1022d6812bb1e3c3b2edbfccb9651eabd99134ee0acd22bf368a396 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family d64e388cb102… | Yes | Yes | Yes | Not available | Not available | Open world |
| Shelf place | Full scene | Meta-World | 5af22f11a7139b54ba927fb5f6b407ba4ee6ef17358dad8d15416b4374c9a3b9 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 5af22f11a713… | Yes | Yes | Yes | Not available | Not available | Open world |
| Soccer | Full scene | Meta-World | 3c36a7279065d84c4bb342b2ee196b59471fb9624435997c283e64f592ec39f6 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.07 | none | native reset-state measured; browser preview; not a scored run; scene family 3c36a7279065… | Yes | Yes | Not available | Not available | Not available | Open world |
| Stick pull | Full scene | Meta-World | 058f89ad4a6399678d8be26d897fdf8dfbc49ccbc3eebf8c948f048ea1ed5915 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.12 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 058f89ad4a63… | Yes | Yes | Yes | Not available | Not available | Open world |
| Stick push | Full scene | Meta-World | 058f89ad4a6399678d8be26d897fdf8dfbc49ccbc3eebf8c948f048ea1ed5915 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.12 | site-generated | native reset-state measured; browser preview; not a scored run; scene family 058f89ad4a63… | Yes | Yes | Yes | Not available | Not available | Open world |
| Sweep | Full scene | Meta-World | f43f31b0b1ef5c83d14325c021d17d34f3005f44b12a42454b818152c76b2506 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | site-generated | native reset-state measured; browser preview; not a scored run; scene family f43f31b0b1ef… | Yes | Yes | Yes | Not available | Not available | Open world |
| Window close | Full scene | Meta-World | a2601fdd6fff88eec9ba3cb8fd2e716a7abbcacba34114a749ced611b3e45093 | mujoco 3.3.0 / Gymnasium 1.3.0 | Rethink Sawyer (Meta-World mocap XYZ) | source-parsed · bar 0.05 | none | native reset-state measured; browser preview; not a scored run; scene family a2601fdd6fff… | Yes | Yes | Not available | Not available | Not available | Open world |
| Cube pickup | Full scene | ManiSkill 3 | pickcube-table-panda-seed0 | SAPIEN / PhysX CUDA 3.0.3 | Franka Panda · ManiSkill model | Cube within 0.025 m of goal and robot joint velocity below 0.2. This recording first succeeds after the 50-step task horizon. | site-generated | 83 recorded native states; browser body-frame comparison; goal after deadline, no in-budget success credit | Yes | Yes | Yes | Not available | Not available | Open world |
| Pass and place a block | Full scene | RoboTwin | native-handover-block-aloha-agilex | SAPIEN / PhysX CPU 3.0.0b1 | Aloha-AgileX · RoboTwin dual-arm model | Unmodified native predicate: block/target functional-point x,y errors <0.03m, z error <0.01m, and right gripper open. Fresh expert and native planned-path repeat both satisfy it. | site-generated | official expert native success; independent stored-path repeat; no expert horizon; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| Lift and place the red block | Full scene | CALVIN | calvin-scene-D | CALVIN / PyBullet 3.2.7 | Franka Panda · CALVIN longer-finger variant | Native lift predicate, then placement on the sliding-cabinet shelf; no reset between phases | site-generated | 101 native states; both source task predicates satisfied; exact independent repeat; not an official sequence score | Yes | Yes | Yes | Not available | Not available | Open world |
| Open drawer | Full scene | RoboCasa | layout1-style1-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | OpenDrawer.check_success is drawer.is_open after ManipulateDrawer.check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | site-generated | human-action native success within 750 steps; two independent runs; source-matched recorded kitchen | Yes | Yes | Yes | Not available | Not available | Open world |
| Close cabinet | Full scene | RoboCasa | layout1-style1-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | CloseCabinet.check_success is cabinet.is_closed after ManipulateDoor.check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | site-generated | human-action native success within 750 steps; two independent runs; source-matched recorded kitchen | Yes | Yes | Yes | Not available | Not available | Open world |
| Close blender lid | Full scene | RoboCasa | layout2-style2-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | CloseBlenderLid.check_success is blender.get_state()['lid_on_blender'] and the gripper is farther than 0.15 m from the lid (OU.gripper_fxtr_far). Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Close fridge | Full scene | RoboCasa | layout3-style3-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | CloseFridge.check_success is fridge.is_closed after ManipulateDoor.check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Close toaster oven door | Full scene | RoboCasa | layout4-style4-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | CloseToasterOvenDoor.check_success is toaster_oven.is_closed in CloseToasterOvenDoor._check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Navigate kitchen | Full scene | RoboCasa | layout5-style5-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | NavigateKitchen.check_success is mobile base within 0.20 m of the target base pose and cos(yaw error) >= 0.98 in NavigateKitchen._check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Open cabinet | Full scene | RoboCasa | layout7-style7-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | OpenCabinet.check_success is cabinet.is_open after ManipulateDoor.check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Open stand mixer head | Full scene | RoboCasa | layout8-style8-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | OpenStandMixerHead.check_success is stand_mixer.get_state(env)['head'] > 0.99 in OpenStandMixerHead._check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick place counter to cabinet | Full scene | RoboCasa | layout9-style9-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | PickPlaceCounterToCabinet.check_success is OU.obj_inside_of(obj, cabinet) and OU.gripper_obj_far in PickPlaceCounterToCabinet._check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Pick place drawer to counter | Full scene | RoboCasa | layout10-style10-seed0 | robosuite 1.5.2 / MuJoCo 3.3.1 | Franka Panda on Omron mobile base (RoboCasa PandaOmron) | PickPlaceDrawerToCounter.check_success is OU.check_obj_any_counter_contact(obj) and OU.gripper_obj_far in PickPlaceDrawerToCounter._check_success. Native reset and articulated fixture qpos are assembly evidence, not a scored task. Parsed from source, not a paper or leaderboard result. | none | native reset-state recorded; official Box selected members on media remotes; not a scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Navigate to the sugar box | Full scene | Habitat / ReplicaCAD | replicacad-v3-sc3-staging-18 | Habitat-Lab / Habitat-Sim / Bullet 0.3.3 | Fetch · Habitat suction variant | Horizontal distance below 1.5 m, facing-angle error below 0.261799 rad, and explicit stop. This episode reaches the 400-action limit without success; final distance 4.11349 m. Task type is a Habitat registry identifier, not an upstream Gym registration. | site-generated | 401 native states; exact independent repeat and desktop frame checks; failed at original task deadline | Yes | Yes | Yes | Not available | Not available | Open world |
| Navigate to the candle | Full scene | MolmoSpaces navigation | procthor-test-0 | MuJoCo 3.5.0 | RB-Y1 · source holonomic simulation preset | Planar distance below 1.5 m and positive head-camera visibility | site-generated | 212 native states; selected goal reached; exact independent repeat; not a suite score | Yes | Yes | Yes | Not available | Not available | Open world |
| Four-room home | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 1 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 2 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 3 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 4 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 5 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 6 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 7 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 8 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 9 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 10 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 11 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 12 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 13 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 14 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 15 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 16 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 3-room home · 17 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 18 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 19 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 20 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 21 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 22 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 23 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 24 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 25 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 26 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 27 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 28 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 29 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 30 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 6-room home · 31 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 32 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 3-room home · 33 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 5-room home · 34 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 35 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 36 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 6-room home · 37 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 38 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 39 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 40 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 10-room home · 41 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 42 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 43 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 44 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 10-room home · 45 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 46 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 47 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 48 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 49 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 50 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 51 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 52 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 5-room home · 53 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 54 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 55 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 5-room home · 56 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 57 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 5-room home · 58 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 3-room home · 59 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 60 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 61 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 62 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 6-room home · 63 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 64 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 65 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 6-room home · 66 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 5-room home · 67 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 10-room home · 68 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 69 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 70 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 71 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 72 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 73 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 74 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 75 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 76 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 77 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 10-room home · 78 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 3-room home · 79 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 80 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 81 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 82 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 5-room home · 83 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 8-room home · 84 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 85 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 86 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 87 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 88 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 7-room home · 89 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 90 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 4-room home · 91 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 8-room home · 92 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 93 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 2-room home · 94 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 95 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 96 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 3-room home · 97 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 10-room home · 98 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| 1-room home · 99 | Full scene | ProcTHOR via MolmoSpaces | ProcTHOR via MolmoSpaces | MuJoCo 3.5.0 | No robot assigned | No evaluation task defined | none | complete converted scene; native kinematics checked; no scored run | Yes | Yes | Not available | Not available | Not available | Open world |
| Navigate the warehouse | Full scene | Open Navigation | opennav-warehouse | Gazebo Sim / DART / ROS Jazzy / Nav2 8.15.0 | Open Navigation benchmark forklift | Both navigation legs and timed handling waits complete; no physical pallet manipulation | site-generated | 11,847 measured body states and 960 camera images; original full warehouse; native mission completion, not a workload score | Yes | Yes | Yes | Not available | Not available | Open world |
| Navigate the woodland | Full scene | Forest3D | forest3d | Gazebo Sim / DART / ROS Jazzy 8.15.0 | Clearpath Husky | Reach and hold within 0.6 m of the declared goal for one second under a 120-second limit | site-generated | 1,163 measured body states and 315 camera images; generated woodland; authored goal completion, not an upstream benchmark score | Yes | Yes | Yes | Not available | Not available | Open world |
Class definition
Columns identify suites, datasets or training corpora; rows preserve each source’s scale unit.
What stays out
Policy scores, payload and IFR counts belong elsewhere. Galbot G1, Unitree G1 and Unitree H1 are distinct embodiments, never interchangeable. Announced robots without a sheet do not enter these tables.
One class per table
No two classes share a “task count” column.
The classes: closed-loop sim manipulation, VLA-focused manipulation, real-to-sim eval, grasp synthesis, VLN, tracking, spatial VQA, embodied QA, humanoid tracking, humanoid whole-body sim, and training corpora.
Paper scale, not a ranking
Published counts stay in the paper’s unit: tasks, episodes, grasps, hours, frames. Derived charts on this page count named columns, not performance.
Evidence grade
RSuite identity is a research claim. Official competition pages are manuals for status only. — means the inspected source did not publish a comparable figure.
Not a table row. Galbot ET1 has no public eval sheet.
ET1 software lineage
AstraBrain-Agent is a launch architecture. LDA-1B, WAM-TTT, LATENT, and Humanoid-GPT explain research lineage; they are not copied into these suite columns as product checkpoints.
Coverage on this page
These totals are derived from the tables below. They are not hub data-stat counters and not a quality score. “Task” is not a shared unit across tables.
- Identity tables
- 16
- One
data-classeach - Named suite columns
- 28
- Eval suites and corpora, not robots
- Role groups
- 8
- Charts count columns, not scores
- Training corpora
- 5
- Hours ≠ trajectories ≠ frames
Retrieved from the papers and official project pages cited in each caption.
Suite map
Physical form still controls hardware comparability on other pages. Here the split is evaluation role. Cards jump to the owning table.
Sim manipulation
RLBench, CALVIN, LIBERO, ManiSkill2, Meta-World, RoboCasa.
Open the sim sheet → VLA manipulationVLABench and RoboTwin 2.0 isolate language reasoning and robust bimanual transfer.
Open the VLA sheet → Real-to-simSIMPLER on Google Robot and WidowX setups.
Open SIMPLER → Real furnitureFurnitureBench assembly demonstrations.
Open FurnitureBench → Everyday activityBEHAVIOR-1K household activities.
Open BEHAVIOR → Grasp / partsDexGraspNet, DexGraspNet 2.0, GAPartNet.
Open grasp sheets → InstructionALFRED, Open6DOR, Room-to-Room.
Open language sheets → Perception / QAEVT-Bench, OmniSpatial, OpenEQA.
Open perception → Humanoid motionHumanTracker and HumanoidBench stay apart.
Open humanoid sheets → Explore worldsOfficial HumanoidBench courses. Preview assembles published objects; Full scene adds the complete-world file where one exists; Cabinet includes a recorded replay. Animation is not a score or ranking.
Open Explore worlds → Training corporaOXE, DROID, BridgeData V2, EI-30k, SynGrasp-1B.
Open corpora → PlatformsHabitat-Sim is a renderer stack, not R2R.
Open platforms →Role mix
Derived from this page’s 28 named columns. Not a ranking.
Closed-loop sim manip6
VLA-focused manipulation2
Real / real-to-sim3
Activity / instruction / VLN4
Grasp / parts3
Perception / QA / tracking3
Humanoid motion2
Training corpora5
Medium mix
Same 28 columns, grouped by the medium the source describes.
Simulation12
Real or real imagery6
Dataset / mocap / corpus9
Spatial VLM images1
Simulation columns: the six closed-loop manipulation suites, VLABench, RoboTwin 2.0, BEHAVIOR-1K, ALFRED, EVT-Bench, and HumanoidBench.
Real, dataset and image-QA columns
Real or real-imagery: SIMPLER’s two setups, FurnitureBench, Open6DOR, Room-to-Room, and OpenEQA. Dataset / mocap / corpus: two DexGraspNet columns, GAPartNet, HumanTracker, and the five training corpora. OmniSpatial is image QA.
Manipulation evals
Closed-loop simulation, VLA-focused simulation, paired real-to-sim, and real furniture assembly stay in four tables. Do not add their task counts.
Closed-loop simulated manipulation
| Metric | RLBench | CALVIN | LIBERO | ManiSkill2 | Meta-World | RoboCasa |
|---|---|---|---|---|---|---|
| Identity | ||||||
| Kind# | closed-loop simulated manipulation benchmark [R1] | language-conditioned simulated manipulation benchmark [R3] | lifelong language-conditioned simulated manipulation benchmark [R5] | simulated manipulation benchmark across embodiments and object sets [R7] | multi-task / meta-learning simulated manipulation benchmark [R9] | kitchen simulation framework and evaluation task set [R11] |
| First public date# | arXiv 2019-09-26 [R1] | arXiv 2021-12-06 [R3] | arXiv 2023-06-05 [R5] | arXiv 2023-02-09 [R7] | arXiv 2019-10-24 [R9] | arXiv 2024-06-04 [R11] |
| Paper# | arXiv:1909.12271 [R1] | arXiv:2112.03227 [R3] | arXiv:2306.03310 [R5] | arXiv:2302.04659 [R7] | arXiv:1910.10897 [R9] | arXiv:2406.02523 [R11] |
| Scale | ||||||
| Published scale# | 100 unique hand-designed tasks [R1] | 4 environments; 34 subtasks versus 18 prior; ~24 h play data; 20K language directives [R3] | 130 language-conditioned tasks in 4 suites (Spatial, Object, Goal: 10 tasks each; LIBERO-100) [R5] | 20 task families; 2000+ objects; 4 million+ demonstration frames [R7] | 50 distinct manipulation tasks [R9] | 100 tasks (25 atomic + 75 composite); 120 kitchen scenes; 2,509 objects; 153 categories; 100K trajectories [R11] |
| Medium# | simulation [R1] | simulation [R3] | simulation [R5] | simulation [R7] | simulation [R9] | kitchen simulation [R11] |
| Embodiment in the source# | Franka Panda [R1] | Franka Panda 7-DOF [R3] | tabletop manipulator in the paper’s suites [R5] | stationary or mobile-base; single-arm or dual-arm; rigid or soft-body [R7] | Sawyer [R9] | kitchen robots in the paper, including mobile platforms and humanoid robots [R11] |
| Engine / renderer# | V-REP (CoppeliaSim) [R1] | PyBullet [R3] | — | SAPIEN [R7] | MuJoCo [R9] | simulation framework on the project site [R12] |
| Primary metric family# | task success on the paper’s 100-task set [R1] | multi-step language instruction success in the paper’s protocol [R3] | lifelong / sequential transfer success on the four suites [R5] | task-family success in the paper’s protocol [R7] | multi-task and meta-learning success on the 50-task set [R9] | atomic and composite kitchen-task success in the paper’s protocol [R11] |
| Language-conditioned# | task names and descriptions; not a CALVIN-style language-conditioned suite [R1] | yes — 20K language directives [R3] | yes — 130 language-conditioned tasks [R5] | not published as a language-instruction suite [R7] | not published as a language-instruction suite [R9] | yes — language goals in the paper’s BC policy; composite tasks from LLM guidance [R11] |
| Access | ||||||
| Public code# | yes — official project publishes code [R2] | yes — official project publishes code [R4] | yes — official project publishes code [R6] | yes — official project publishes code [R8] | yes — official project publishes code [R10] | yes — official project publishes code [R12] |
| Public leaderboard# | — | — | — | — | — | — |
| Claim boundary# | identity and published scale only; not a method leaderboard [R1] | identity and published scale only; not a method leaderboard [R3] | paper training protocol is 50 demonstrations per task; that is a protocol, not a second task count, and not a 65k-demo figure from third-party pages [R5] | identity and published scale only; “task family” is not the same unit as RLBench’s 100 tasks [R7] | identity and published scale only; Sawyer in simulation is not a product SKU column [R9] | body figures 2,509 objects and 153 categories; abstract rounds to 2,500+ / 150+. Not a success-rate table. Current docs advertise 365 tasks; that branding is not this paper’s 100/120 scale and is not a verified explorer count [R11] [R46] |
VLA-focused manipulation
These suites target different failure surfaces. Their success rates are not joined.
- VLABench emphasizes language, world knowledge, and long-horizon reasoning on a standard single arm.
- RoboTwin 2.0 emphasizes domain randomization, bimanual data generation, and sim-to-real robustness.
| Metric | VLABench | RoboTwin 2.0 |
|---|---|---|
| Identity | ||
| Kind# | language-conditioned manipulation benchmark for VLAs, foundation-model workflows, and VLMs [R32] | scalable bimanual data generator and policy-generalization benchmark [R34] |
| First public date# | arXiv 2024-12-24 [R32] | arXiv 2025-06-22 [R34] |
| Paper# | arXiv:2412.18194; ICCV 2025 [R32] [R33] | arXiv:2506.18088 [R34] |
| Scale and protocol | ||
| Task structure# | 100 task categories: 60 primitive + 40 composite [R32] | 50 dual-arm tasks in the paper; the project describes over 50 supported tasks [R34] [R35] |
| Asset library# | 2,164 objects across 163 categories [R32] | RoboTwin-OD: 731 objects across 147 categories [R34] |
| Released trajectories# | standard evaluation episodes and primitive-task fine-tuning data are public; no one suite-wide trajectory count is stated [R33] | 100,000+ pre-collected expert trajectories across 50 tasks and 5 embodiments [R34] |
| Evaluation scope# | 6 capability dimensions: mesh / texture, spatial, common sense / knowledge, semantic instruction, physical laws, and long-horizon reasoning [R32] | 5 randomization axes: clutter, lighting, background, tabletop height, and language; clean versus randomized policy evaluation [R34] |
| Medium# | simulation [R32] | simulation benchmark and data generator with a separate sim-to-real study [R34] |
| Embodiment in the source# | standard evaluation: 7-DoF Franka Emika Panda with parallel gripper; framework also supports other forms [R32] | dual-arm pairings of Aloha-AgileX, ARX-X5, Piper, Franka, and UR5 manipulators [R34] |
| Engine / renderer# | MuJoCo + dm_control [R32] | — |
| Primary metric family# | interactive task success + graduated Progress Score; non-interactive VLM action-sequence matching [R32] | task success under clean and hard randomized conditions on the full 50-task protocol [R34] |
| Language-conditioned# | yes — natural instructions include implicit intent, common sense, and multi-step reasoning [R32] | yes — trajectory-level language is one randomization axis [R34] |
| Access | ||
| Public code# | yes — MIT-licensed official repository [R33] | yes — MIT-licensed official repository [R35] |
| Public data# | yes — evaluation episodes and fine-tuning datasets are linked from the official project [R33] | yes — object library and pre-collected trajectories are linked from the official project [R35] |
| Public leaderboard# | incomplete — the official repository still lists standard VLA / VLM leaderboard integration as open [R33] | yes — official leaderboard linked from the project [R35] |
| Claim boundary# | 100 task categories are not RoboTwin’s bimanual tasks; performance tables stay in the paper [R32] | 50 benchmark tasks are distinct from the 10-task code-generation study and 4-task real-world study; those method scores stay in the paper [R34] |
Real-to-sim (SIMPLER)
| Metric | Google Robot | WidowX |
|---|---|---|
| Identity | ||
| Kind# | real-to-sim evaluation environment for RT-series setups [R13] | real-to-sim evaluation environment for the BridgeData V2 WidowX setup [R13] |
| First public date# | arXiv 2024-05-09 [R13] | arXiv 2024-05-09 [R13] |
| Paper# | arXiv:2405.05941 [R13] | arXiv:2405.05941 [R13] |
| Scale | ||
| Published scale# | ~1,500 paired sim-and-real evaluation episodes across both embodiments in the paper, not a per-robot task count [R13] | ~1,500 paired sim-and-real evaluation episodes across both embodiments in the paper, not a per-robot task count [R13] |
| Medium# | paired real and simulated evaluation [R13] | paired real and simulated evaluation [R13] |
| Embodiment in the source# | Google Robot (RT-series evaluation setup) [R13] | WidowX / BridgeData V2 setup [R13] |
| Engine / renderer# | open-sourced SIMPLER environments (SAPIEN in the default paper setup) [R13] | open-sourced SIMPLER environments [R13] |
| Primary metric family# | correlation of simulated vs real policy success in the paper; not copied here [R13] | correlation of simulated vs real policy success in the paper; not copied here [R13] |
| Language-conditioned# | yes — language-conditioned Google Robot tasks in the paper [R13] | yes — BridgeData V2 WidowX tasks in the paper [R13] |
| Access | ||
| Public code# | yes — official project publishes code [R14] | yes — official project publishes code [R14] |
| Public leaderboard# | — | — |
| Claim boundary# | setup identity only. Do not read the paper’s policy tables into this atlas [R13] | setup identity only. WidowX here is the Bridge evaluation body, not a product SKU [R13] |
Real furniture assembly
| Metric | FurnitureBench |
|---|---|
| Identity | |
| Kind# | real furniture-assembly benchmark with teleoperation demonstrations [R15] |
| First public date# | arXiv 2023-05-22 [R15] |
| Paper# | arXiv:2305.12821 [R15] |
| Scale | |
| Published scale# | 8 furniture models; 5,000+ teleoperation demonstrations; 200+ hours [R15] |
| Medium# | real [R15] |
| Embodiment in the source# | real robot furniture assembly as described in the paper [R15] |
| Engine / renderer# | — |
| Primary metric family# | assembly success on the paper’s furniture set [R15] |
| Language-conditioned# | — |
| Access | |
| Public code# | yes — paper points to public code [R15] |
| Public leaderboard# | — |
| Claim boundary# | identity and published scale only; not a simulated kitchen suite [R15] |
Everyday activity sim
BEHAVIOR-1K publishes activities, scenes, and objects. That is not CALVIN’s language-directive count.
| Metric | BEHAVIOR-1K |
|---|---|
| Identity | |
| Kind# | everyday household-activity simulation benchmark [R16] |
| First public date# | arXiv 2024-03-14 [R16] |
| Paper# | arXiv:2403.09227 [R16] |
| Scale | |
| Published scale# | 1,000 activities; 50 scenes; 9,000+ objects [R16] |
| Medium# | simulation [R16] |
| Embodiment in the source# | simulated household agents in the paper’s scenes [R16] |
| Engine / renderer# | OmniGibson [R16] |
| Primary metric family# | activity success in the paper’s protocol [R16] |
| Language-conditioned# | activities are named household tasks; not mixed into CALVIN’s 20K directives [R16] |
| Access | |
| Public code# | yes — official project publishes code [R17] |
| Public leaderboard# | — |
| Claim boundary# | identity and published scale only; 1,000 activities are not 100 RLBench tasks [R16] |
Grasp and articulated parts
Synthesis datasets are not closed-loop policy suites. DexGraspNet 2.0 uses a LEAP hand in the paper; that is not a robot payload cell.
| Metric | DexGraspNet | DexGraspNet 2.0 |
|---|---|---|
| Identity | ||
| Kind# | large-scale synthetic dexterous grasp dataset [R39] | generative dexterous grasp dataset in synthetic clutter [R40] |
| First public date# | arXiv 2022-10-06 [R39] | arXiv 2024-10-30 [R40] |
| Paper# | arXiv:2210.02697 [R39] | arXiv:2410.23004 [R40] |
| Scale | ||
| Published scale# | 1.32 million grasps; 5,355 objects [R39] | 1,319 objects; 8,270 scenes; 427 million grasps [R40] |
| Medium# | synthetic [R39] | synthetic clutter [R40] |
| Embodiment in the source# | ShadowHand [R39] | LEAP hand [R40] |
| Engine / renderer# | — | — |
| Primary metric family# | grasp synthesis dataset; not a closed-loop policy suite [R39] | grasp synthesis dataset; not a closed-loop policy suite [R40] |
| Language-conditioned# | — | — |
| Access | ||
| Public code# | yes — paper and project publish code [R39] | yes — paper and project publish code [R40] |
| Public leaderboard# | — | — |
| Claim boundary# | dataset identity only. Author method success rates are not this table [R39] | LEAP hand here is the dataset’s synthesis hand, not a robot payload cell [R40] |
Articulated parts
| Metric | GAPartNet |
|---|---|
| Identity | |
| Kind# | cross-category articulated-part perception and manipulation dataset [R41] |
| First public date# | arXiv 2022-11-09 [R41] |
| Paper# | arXiv:2211.05272 [R41] |
| Scale | |
| Published scale# | 9 GAPart classes; 27 object categories; 8,489 part instances; 1,166 objects [R41] |
| Medium# | dataset of articulated objects [R41] |
| Embodiment in the source# | — |
| Engine / renderer# | — |
| Primary metric family# | part-class perception / actionable-part evaluation in the paper [R41] |
| Language-conditioned# | — |
| Access | |
| Public code# | yes — paper and project publish code [R41] |
| Public leaderboard# | — |
| Claim boundary# | part instances are not grasp counts and not a robot SKU [R41] |
Instruction, rearrangement, navigation
ALFRED is household instruction in AI2-THOR. Open6DOR is open-instruction 6-DoF rearrangement with no public scale here. Room-to-Room is VLN on real indoor scans.
Household instruction
| Metric | ALFRED |
|---|---|
| Identity | |
| Kind# | household instruction-following benchmark in interactive visual simulation [R18] |
| First public date# | arXiv 2019-12-03 [R18] |
| Paper# | arXiv:1912.01734 [R18] |
| Scale | |
| Published scale# | 25,743 English language directives; 8,055 expert demonstrations; 120 indoor scenes [R18] |
| Medium# | simulation (AI2-THOR) [R18] |
| Embodiment in the source# | egocentric household agent in AI2-THOR 2.0 [R18] |
| Engine / renderer# | AI2-THOR 2.0 [R18] |
| Primary metric family# | goal and step-instruction success in the paper’s protocol [R18] |
| Language-conditioned# | yes — high-level goals and low-level instructions [R18] |
| Access | |
| Public code# | yes — paper publishes the dataset and evaluation [R18] |
| Public leaderboard# | — |
| Claim boundary# | body count is 25,743 directives; abstract says 25k. Simulated household instruction, not real 6-DoF rearrangement [R18] |
Open-instruction rearrangement
| Metric | Open6DOR |
|---|---|
| Identity | |
| Kind# | open-instruction 6-DoF object rearrangement benchmark [R38] |
| First public date# | IROS 2024 Oral [R38] |
| Paper# | — |
| Scale | |
| Published scale# | — |
| Medium# | real rearrangement evaluation as described on the project page [R38] |
| Embodiment in the source# | — |
| Engine / renderer# | — |
| Primary metric family# | open-instruction placement, not pick-only grasping [R38] |
| Language-conditioned# | yes — open language instructions [R38] |
| Access | |
| Public code# | project page [R38] |
| Public leaderboard# | — |
| Claim boundary# | no arXiv on the homepage; scale cells stay empty rather than guessed [R38] |
Vision-and-language navigation
| Metric | Room-to-Room |
|---|---|
| Identity | |
| Kind# | vision-and-language navigation on real indoor scans [R19] |
| First public date# | arXiv 2017-11-20 [R19] |
| Paper# | arXiv:1711.07280 [R19] |
| Scale | |
| Published scale# | 21,567 instructions; average 29 words [R19] |
| Medium# | real Matterport3D indoor imagery [R19] |
| Embodiment in the source# | navigation agent on Matterport3D scans [R19] |
| Engine / renderer# | Matterport3D [R19] |
| Primary metric family# | instruction-following navigation success in the paper’s protocol [R19] |
| Language-conditioned# | yes — English navigation instructions [R19] |
| Access | |
| Public code# | yes — official project publishes the dataset [R20] |
| Public leaderboard# | — |
| Claim boundary# | R2R is a VLN dataset on real scans. Habitat is a later simulator stack and is not this column [R19] |
Tracking, spatial VQA, embodied QA
EVT-Bench episode counts are not TrackVLA’s 1.7 million training samples. OmniSpatial is not OpenEQA. ERQA’s 400 questions stay on the foundation-models page.
Embodied visual tracking
| Metric | EVT-Bench |
|---|---|
| Identity | |
| Kind# | embodied visual tracking benchmark introduced with TrackVLA [R19] |
| First public date# | arXiv 2025-05-29 [R19] |
| Paper# | arXiv:2505.23189 [R19] |
| Scale | |
| Published scale# | 25,986 episodes; 100 avatars; 804 scenes (HM3D + MP3D); train 21,771 / test 4,215; three sub-tasks [R19] |
| Medium# | simulation (HM3D and MP3D scenes) [R19] |
| Embodiment in the source# | simulated tracking agent; 100 humanoid avatars in the benchmark [R19] |
| Engine / renderer# | — |
| Primary metric family# | success rate, episode length, tracking rate, and collision rate in the paper [R19] |
| Language-conditioned# | yes — language descriptions of the target [R19] |
| Access | |
| Public code# | paper states EVT-Bench would be made public [R19] |
| Public leaderboard# | — |
| Claim boundary# | 1.7 million samples in the TrackVLA abstract is the training mixture (recognition + tracking), not the EVT-Bench episode count [R19] |
Spatial VLM benchmark
| Metric | OmniSpatial |
|---|---|
| Identity | |
| Kind# | spatial-reasoning benchmark for vision-language models [R37] |
| First public date# | arXiv 2025-06-03 [R37] |
| Paper# | arXiv:2506.03135 [R37] |
| Scale | |
| Published scale# | 4 categories; 50 subcategories; 8.4K+ QA pairs [R37] |
| Medium# | image / VQA benchmark [R37] |
| Embodiment in the source# | — |
| Engine / renderer# | — |
| Primary metric family# | spatial QA accuracy in the paper [R37] |
| Language-conditioned# | yes — language questions over images [R37] |
| Access | |
| Public code# | yes — paper and project publish code [R37] |
| Public leaderboard# | — |
| Claim boundary# | not an embodied motor suite and not OpenEQA [R37] |
Embodied question answering
| Metric | OpenEQA |
|---|---|
| Identity | |
| Kind# | open-vocabulary embodied question answering benchmark [R21] |
| First public date# | CVPR 2024 [R21] |
| Paper# | — |
| Scale | |
| Published scale# | 1,600+ human-generated questions; 180+ real-world environments [R21] |
| Medium# | real environments (episodic memory and active exploration settings) [R21] |
| Embodiment in the source# | EQA agent as defined on the project page [R21] |
| Engine / renderer# | — |
| Primary metric family# | LLM-Match agreement with human answers on the project protocol [R21] |
| Language-conditioned# | yes — untemplated natural-language questions [R21] |
| Access | |
| Public code# | project page [R21] |
| Public leaderboard# | — |
| Claim boundary# | no arXiv id in the project bibtex. Not OmniSpatial and not ERQA’s 400 Gemini questions [R21] |
Humanoid motion
HumanTracker scores trackers on mocap. HumanoidBench is a MuJoCo whole-body task suite whose primary agent is Unitree H1 with two Shadow Hands. They do not share a task column.
HumanTracker
| Metric | HumanTracker |
|---|---|
| Identity | |
| Kind# | humanoid motion-tracking benchmark with a preference-aligned metric [R36] |
| First public date# | arXiv 2026-08-24 [R36] |
| Paper# | arXiv:2608.13555 [R36] |
| Scale | |
| Published scale# | ~153 hours optical mocap; 4 motion families; ~25K clips; HumanScore trained on 12K pairs / 24K motions [R36] |
| Medium# | optical motion capture retargeted to a simulated humanoid [R36] |
| Embodiment in the source# | 29-DoF humanoid qpos representation in the paper’s evaluator [R36] |
| Engine / renderer# | MuJoCo evaluation entry point in the paper [R36] |
| Primary metric family# | completion, MPJPE, and HumanScore (0–100) in the paper [R36] |
| Language-conditioned# | text labels on clips; not a household instruction suite [R36] |
| Access | |
| Public code# | yes — paper lists official code [R36] |
| Public leaderboard# | — |
| Claim boundary# | Humanoid-GPT is one evaluated tracker, not the benchmark body. This is not HumanoidBench and not a Galbot product column [R36] |
HumanoidBench
| Metric | HumanoidBench |
|---|---|
| Identity | |
| Kind# | simulated whole-body locomotion and manipulation benchmark [R22] |
| First public date# | arXiv 2024-03-15 [R22] |
| Paper# | arXiv:2403.10506 [R22] |
| Scale | |
| Published scale# | 15 manipulation + 12 locomotion tasks (27 tasks) [R22] |
| Medium# | simulation [R22] |
| Embodiment in the source# | Unitree H1 with two Shadow Hands as the primary agent [R22] |
| Engine / renderer# | MuJoCo [R22] |
| Primary metric family# | RL task return / success on the paper’s 27-task set [R22] |
| Language-conditioned# | — |
| Access | |
| Public code# | yes — official project publishes code [R23] |
| Public leaderboard# | — |
| Claim boundary# | optional other simulator models in the paper (including a Unitree G1 model) are not this column and are not Galbot G1. Not HumanTracker [R22] |
Explore worlds assembles twenty official XML fragments [R36]: seven locomotion/athletic lanes, three joint-range courses and ten object/room scenes.
Capability labels
Preview, full-scene and recorded-replay capabilities are labeled separately. Task names stay out of this suite-identity table; preview is not evaluation.
Training corpora
These rows are pretraining mixtures, not evaluation leaderboards. Do not add hours to trajectories or frames.
| Metric | Open X-Embodiment | DROID | BridgeData V2 | EI-30k | SynGrasp-1B |
|---|---|---|---|---|---|
| Identity | |||||
| Kind# | multi-embodiment robot-learning dataset pool [R24] | in-the-wild robot manipulation dataset [R26] | multi-environment robot demonstration dataset [R28] | mixed human and robot embodied pretraining corpus used with LDA-1B [R9] | synthetic VLA pretraining frames used with GraspVLA [R17] |
| First public date# | arXiv 2023-10-13 [R24] | arXiv 2024-03-19 [R26] | arXiv 2023-08-24 [R28] | arXiv 2026-02-16 [R9] | arXiv 2025-05-06 [R17] |
| Paper# | arXiv:2310.08864 [R24] | arXiv:2403.12945 [R26] | arXiv:2308.12952 [R28] | arXiv:2602.12215 [R9] | arXiv:2505.03233 [R17] |
| Scale | |||||
| Published scale# | 1M+ trajectories; 22 embodiments; 60 datasets; 34 labs; 527 skills; 160266 tasks (body). Abstract: 22 robots, 21 institutions, 527 skills, 160266 tasks [R24] | 76k trajectories; 350 hours; 564 scenes; 86 tasks [R26] | 60,096 trajectories; 24 environments [R28] | 30k+ hours mixed-quality human and robot data [R9] | 1 billion synthetic VLA frames [R17] |
| Medium# | real robot trajectories pooled from many datasets [R24] | real teleoperated trajectories [R26] | real demonstrations [R28] | mixed real, sim, labeled, and unlabeled human and robot data [R9] | synthetic [R17] |
| Embodiment in the source# | 22 embodiments: single arms, bi-manual robots, and quadrupeds [R24] | Franka Panda stack [R26] | WidowX-class Bridge robots in the paper [R28] | human video plus robot trajectories; LDA-1B real figures use Galbot G1 and Unitree G1 variants [R9] | synthetic grasping VLA frames; not a named robot SKU [R17] |
| Engine / renderer# | RLDS / OXE distribution [R25] | DROID distribution [R27] | BridgeData V2 distribution [R29] | LeRobot-format corpus described in the LDA paper [R9] | — |
| Primary metric family# | pretraining corpus; not an eval leaderboard [R24] | pretraining corpus; not an eval leaderboard [R26] | pretraining corpus; not an eval leaderboard [R28] | pretraining corpus for a world-action model [R9] | pretraining frames for GraspVLA [R17] |
| Language-conditioned# | language annotations vary by constituent dataset [R24] | language-annotated tasks in the paper [R26] | language-annotated demonstrations in the paper [R28] | mixed labeled and unlabeled [R9] | synthetic VLA frames with action data in the paper [R17] |
| Access | |||||
| Public code# | yes — official project publishes the dataset [R25] | yes — official project publishes the dataset [R27] | yes — official project publishes the dataset [R29] | described with LDA-1B; public dump as stated in that paper [R9] | promised with GraspVLA; check the project page for release [R17] |
| Public leaderboard# | — | — | — | — | — |
| Claim boundary# | abstract uses 21 institutions; body uses 34 labs / 60 datasets. Both stay attributed. Not an evaluation suite [R24] | 76k trajectories and 350 hours are the same corpus in two units, not two datasets. Abstract collector counts are not reconciled here [R26] | not SIMPLER’s WidowX eval column [R28] | hours are not trajectories. Not EVT-Bench [R9] | frames are not hours or trajectories. Not DexGraspNet’s 1.32 million grasps [R17] |
Lab-owned special-env pins
Scale for HumanTracker, OmniSpatial, Open6DOR, DexGraspNet, GAPartNet, EVT-Bench, and EI-30k lives in the tables above.
Research pins and UrbanVLA
These pins stay on Research so existing permalinks keep working. UrbanVLA’s outdoor eval is the paper’s own suite, not a separately named public benchmark.
HumanTracker pin
ECCV 2026 motion-tracking pin with official media.
Open the research pin → OmniSpatial pinICLR 2026 spatial VQA pin.
Open the research pin → Open6DOR pinIROS 2024 Oral project pin.
Open the research pin → DexGraspNet pinICRA 2023 dataset pin.
Open the research pin → DexGraspNet 2.0 pinCoRL 2024 cluttered-grasp pin.
Open the research pin → GAPartNet pinCVPR 2023 articulated-part pin.
Open the research pin → EVT-Bench pinNamed in TrackVLA. Episode counts are on this atlas.
Open the research pin → UrbanVLA outdoor evalPaper’s own special-env figure, not a public suite name.
Open the eval pin → EI-30k pinLDA-1B corpus pin. Hours stay on the corpora sheet.
Open the research pin → DREDSTransparent / specular depth restoration set.
Open the research pin → ASGraspTransparent reconstruction and grasp from active stereo.
Open the research pin → STOPNetMultiview suction on transparent objects.
Open the research pin → BODexBilevel dexterous grasp synthesis.
Open the research pin → UniDexGrasp++Generalist dex grasp lineage into GraspVLA.
Open the research pin → NaVid-4DEgocentric RGB-D VLN special env.
Open the research pin → ERQA (Gemini)400 embodied-reasoning questions live on the policy page.
Open the Gemini cell →Simulation platforms
A renderer is not a VLN dataset. Habitat numbers stay on this card so they are not mixed into Room-to-Room.
Habitat-Sim
Platform paper: Habitat-Sim is a high-performance 3D simulator. On a Matterport3D scene it reports several thousand frames per second single-threaded, and over 10,000 fps multi-process on one GPU.
Habitat-API defines tasks such as navigation, instruction following, and question answering on top of that simulator. That is not the Room-to-Room instruction count. [R30] [R31]
arXiv:1904.01201 · aihabitat.org · first public date arXiv 2019-04-02
Competitions and field trials
Official organisation pages only. Status and identity, not scores, payloads, or invented participant counts.
RoboCup
Ongoing robot-soccer and related leagues run by the RoboCup Federation. [M1]
DARPA Robotics Challenge
Completed disaster-response trial series. Historical programme identity only. [M2]
ANA Avatar XPRIZE
Completed avatar / telepresence prize. [M3]
CYBATHLON
ETH Zurich assistive-device championship. [M4]
EUROBENCH
EU benchmarking framework for biped systems; organisation page, not a spec table. [M5]
How this atlas connects
Search indexes this page once the landscape export is rebuilt. Compare can open a single table; it must not join “task” rows across classes. Research keeps media pins. Foundation-model pages keep policy scores.
Search a suite
?q=EVT-Bench#search should land on the episode-count row, not TrackVLA’s 1.7 million training mixture.
Pick columns from one table. Do not join RLBench tasks with BEHAVIOR activities or OXE hours.
Open compare → Cite the scale row#sim--published-scale is six different “task” definitions. The row is the warning.
The explorer fetches official HumanoidBench MJCF. Cabinet sliders move the five joint ranges the XML publishes; they are not RL returns.
Open the cabinet course → Open lab pinsOfficial figures and videos stay on Research. Scale for those named suites is here.
Open Research benchmarks → Policy scores stay putOpenVLA-OFT’s LIBERO percentages and Gemini ERQA’s 400 questions are policy results, not this atlas.
Open the policy cell → Named sim tasksIsaac Lab Galbot cube-stack IDs are NVIDIA environment names, not a public benchmark table.
Open Isaac Lab → Editorial boundaryThe site is not a performance ranking. This page is suite identity with empty cells where the source is silent.
Read methodology → Source registryNew markers are benchmarks-r* and benchmarks-m*. Lab papers reuse research-systems-r*.