Entity dossier

Bot Landscape / Ecosystem / Entity dossiers / VLABench

Benchmark benchmark--vlabench

VLABench

Language-conditioned simulation benchmark for VLA manipulation.

Reviewed relationships 1

  1. Verified

    VLABench uses simulator MuJoCo Simulator

    Scope benchmark execution

    Boundary The benchmark states MuJoCo plus dm_control.

    First observed · reviewed · [R32] · Show in graph

Canonical evidence claims 16

Claims are projected from authored tables. A dash remains an explicit missing value; it is never filled from a neighboring product or name match.

Robotics benchmarks — suite identity, not a ranking Vla Manipulation Eval 16 rows

MetricValue and evidence
Asset library2,164 objects across 163 categories [R32]
Claim boundary100 task categories are not RoboTwin’s bimanual tasks; performance tables stay in the paper [R32]
Embodiment in the sourcestandard evaluation: 7-DoF Franka Emika Panda with parallel gripper; framework also supports other forms [R32]
Engine / rendererMuJoCo + dm_control [R32]
Evaluation scope6 capability dimensions: mesh / texture, spatial, common sense / knowledge, semantic instruction, physical laws, and long-horizon reasoning [R32]
First public datearXiv 2024-12-24 [R32]
Kindlanguage-conditioned manipulation benchmark for VLAs, foundation-model workflows, and VLMs [R32]
Language-conditionedyes — natural instructions include implicit intent, common sense, and multi-step reasoning [R32]
Mediumsimulation [R32]
PaperarXiv:2412.18194; ICCV 2025 [R32] [R33]
Primary metric familyinteractive task success + graduated Progress Score; non-interactive VLM action-sequence matching [R32]
Public codeyes — MIT-licensed official repository [R33]
Public datayes — evaluation episodes and fine-tuning datasets are linked from the official project [R33]
Public leaderboardincomplete — the official repository still lists standard VLA / VLM leaderboard integration as open [R33]
Released trajectoriesstandard evaluation episodes and primitive-task fine-tuning data are public; no one suite-wide trajectory count is stated [R33]
Task structure100 task categories: 60 primitive + 40 composite [R32]

Source ledger 2

  1. [R32]ResearchVLABench — “VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks”. Type: research paper. ID: arXiv:2412.18194; ICCV 2025. Retrieved: 2026-09-04.Retrieved
  2. [R33]ResearchVLABench — official project site and linked MIT-licensed repository. Type: research project page. Retrieved: 2026-09-04.Retrieved

What is not inferred

No fuzzy entity matching, adjacent-column borrowing, universal compatibility, commercial maturity, or checkpoint-to-product identity. Absence of a relation means unreviewed or unsupported here—not proven incompatibility.