In brief
- Hand-authored HTML is canonical; JSON, search, and feeds are projections validated against it. Scope
- Products are compared only within one physical class and one row definition. Classes
- Every quantitative cell links to a graded source record with a direct URL and retrieval date. Evidence · Claims
—means the inspected primary source published no comparable value. It never means zero. Queries- Scores are projections that print their formula, weights, inputs, and coverage; unknown inputs are never estimated. Scores
- Corrections are evidence changes and need a direct primary source. Corrections
Scope and editorial boundary
Bot Landscape is a reference. It is not a buying guide, safety certification, or proof of commercial availability.
- It covers robot bodies, components, software, foundation models, evaluation-suite identity, market records, and their sources.
- Rankings and scores are published as transparent projections; see Scores.
- Benchmarks catalogues named suites; it does not copy policy success rates.
Hand-authored HTML is canonical. Generated JSON, search, sitemap, CSV, and build metadata are projections validated against it.
Vendor manuals and product tables supply hardware specifications. Filings supply market facts. Papers supply experimental results only.
— means the inspected primary source did not publish a comparable value. It never means zero.
Classification before comparison
Physical form controls specification comparability. Application controls market and deployment grouping. A new physical class receives a new table rather than a wider payload column.
Wheeled mobile manipulators, bipeds, quadrupeds, fixed arms, aerial systems, marine systems, wearables, and end-effectors do not share a payload row.
Ingress codes, named machine-safety claims, and energy-service continuity live on the safety class page, never as extra payload columns.
Rated and peak load, loaded and unloaded endurance, standing and walking payload, and charge time and runtime remain different metrics.
Galbot G1, Unitree G1, and AgiBot G1 are different machines. Product, paper, model, and evaluation-body identities are never inferred from a shared name.
data-class and the same row definition. A blank intersection is better than a false field.Units and completeness
Difference inspection compares authored state and text; it does not normalize units. Completeness counts describe coverage, not quality.
tops profile. Missing precision stays in the cell or as —.Cross-table profiles
A reviewed profile is a maintainer-audited intersection of row definitions from two or more source tables in one physical class.
Each metric names its comparison boundary and points back to canonical HTML. The browser does not join labels, convert units, or copy values into profile configuration.
Conditions kept apart
Rated vs peak load, different duty cycles, combined subsystem counts, and different eval bodies stay out of one another's profiles.
Temporal ecosystem graph
The ecosystem explorer is a reviewed identity-and-relation projection over the canonical claims. It does not replace source tables or infer universal compatibility.
It answers who publishes an artifact, which body a paper evaluates, and where evidence stops.
Namespaced ids distinguish organizations, robots, components, models, datasets, benchmarks, software, simulators, compute, standards, deployments, and research artifacts. Catalogue products retain their existing ids.
Every relation resolves to at least one stable claim id or source id and carries status, scope, boundary note, review date, and optional validity dates.
not-proven-on is a first-class edge. It prevents research lineage, shared names, or adjacent product claims from becoming false deployment or checkpoint joins.
Coverage, evidence posture, access, deployment, and interoperability are literal counts. They are never collapsed into a maturity, quality, or winner score.
Edges that never join
Boundary and superseded edges never enter a path. Organizations are adjacent evidence and non-transitive: a shared publisher or paper affiliation cannot join a robot to an unproven model.
The authored bridge contains identity, bindings, and evidence references only; claim values remain canonical in HTML. Published projections:
- ecosystem-graph.json publishes the complete relation projection.
- events.ndjson provides line-delimited data-release and relation-observation events.
- Each graph entity has generated HTML and JSON dossiers under
entities/.
Evidence grades and precedence
M ManualVendor manual, datasheet, product specification, or official engineering documentation. Preferred for hardware.
F FilingExchange- or regulator-published filing. Preferred for audited money, ownership, and shipment disclosures; never a hardware specification.
R ResearchPaper, model card, official project page, or released code. Supports experiments and artifacts, not an unmentioned product body.
P PressDated company announcement or named article. Used for launches, funding, and status with attribution.
A AnalystNamed research house with a dated methodology. Used for market estimates and ranks, never product specifications.
A reseller sheet is labeled as a reseller source. It does not inherit vendor-manual status.
Later official documentation normally supersedes earlier marketing for the same claim, but both values remain printed when the conflict itself matters.
How a claim ships
- Resolve the entity. Confirm vendor, product, variant, physical class, and date.
- Read the primary source. Preserve the source unit, test condition, version, and qualification.
- Normalize only when safe. Derived conversions are labeled. Different definitions never become one row.
- Attach evidence. Every quantitative cell links to a source record with a direct URL and retrieval date.
- State conflicts and unknowns. Conflicting values are not averaged; missing values stay
—. - Validate the release. Structural checks reject broken ids, missing evidence, mixed classes, stale counts, and invalid exports.
Claim query grammar
Canonical claims can be filtered by class, metric, published state, and evidence grade. A query never fills a dash and never ranks incomparable classes.
This is a projection over the HTML export, not a second database and not SPARQL.
class, table, metric_id, metric, state=reported|missing, grade / evidence_grade (manual / filing / research / press / analyst), subject.
state=missing means the inspected source did not publish a comparable value. It is not zero. ET1 arm load and the WR 2025 industrial column are the fixtures.
Scenario views do not query this state; they only list sourced cells and document dashes.
class=biped cannot return Galbot G1. Galbot ET1, Unitree G1, and AgiBot G1 remain distinct subject ids.
Name a class plus explicit sourced constraints. A satisfy cell needs a canonical source id.
The view lists subjects that have those cells and subjects that fail because the cell is —. Ranked fit scores build on these lists.
Generated /claims/<entity>.json files are a filtered compare-index. They are not a second dataset and do not copy landscape.json.
Shared affiliation does not transfer a claim. Boundary and superseded edges stay out of stack walks. Distinct payload definitions stay distinct rows.
- Local filter:
python3 scripts/query_claims.py class=biped state=missing. - Scenario filter:
python3 scripts/scenario_views.py scenario=biped-arm-load-ip. - Design: claim query grammar · claim slices.
- 2027/2028 horizon: 2026-09-06-horizon.md.
Scores and rankings
Scores answer practical questions, such as which robots are ready to simulate today, from the same committed data as every table.
- Each score prints its formula and weights, links every input to the evidence it came from, and shows input coverage.
- Unknown inputs add nothing and lower coverage; they are never estimated.
- Weights are adjustable on the page and kept in the share URL.
- Scores are versioned with each data release.
Model access, model formats, licence openness, the sim worlds on this site that ship the robot, and recorded replay. Default weights 30 / 20 / 20 / 20 / 10.
Open the board → scores.jsonEvery score with its inputs, raw values, evidence links, coverage, overall rank, and rank within class. Contract: scores.schema.json.
Each authored cell is also parsed into typed values: number or range, unit, SI value, qualifier (rated, peak, max, typical, standby), bound, and approximation.
Compound units are never read as their leading unit: m/s² is an acceleration, not a length. The authored text stays canonical; currencies are never converted; unparsed text stays unparsed.
A list that names its items and shares one closing unit (“knee ~360, hip ~220 N·m”) is read item by item on a row that names the unit's dimension. Each value keeps its name in label.
A count that names what it counts (“76k trajectories”, “depth ×2”) has the dimension named-count, the noun or part name as its unit, and no SI value.
Trajectories, frames, and tasks are never put on one scale: two such counts compare only inside one row and only when their units are the same text. Rows labelled as a boundary are not parsed.
Contract: claims-numeric.schema.json.
A sim world credits a catalogue robot only through a reviewed link in catalogue/embodiment-links.json. A Franka Panda world does not credit Franka Research 3.
Evidence health
The evidence-health family in scores.json scores each robot's distinct catalogue citations: 40% source grade mix, 30% retrieval freshness, and 30% review/access state.
It describes those citations, not every specification or the truth of a claim.
- Grade: manual or filing 1, research 0.8, press 0.5, analyst 0.4, averaged over the cited sources.
- Freshness: average max(0, 1 − retrieval age in days / 365), as of the latest retrieval date in the committed registry. Retrieval is not verification; old documents can still be correct.
- Review/access: share with direct access and no explicit review event due by that audit date. An unscheduled review is not proof of correctness.
- Missing or unresolved citations make the inputs unknown, contribute zero and lower coverage. Every input links its source records; the formula and audit date travel with the score.
Quest fit and party meters
The Quest builder turns a job into requirement lines and ranks every table subject with a published value. Fit = Σ weight × satisfaction ÷ Σ weight, with unpublished lines counted as 0 and shown as coverage.
- Lines. Payload, reach, runtime, speed, mass, ingress, operating temperature, USD price, compute, battery energy.
- Met or short. A line is met when the deciding value reaches the target; a shortfall counts in proportion (8 kg against a 10 kg target counts 0.8).
- Deciding value. Rated, typical, nominal, and unqualified values decide first; peak or max values are used only when nothing else is published, and standby figures never decide.
- Bounds. A bounded value cannot prove the opposite bound: “<2 m/s” does not meet a 2 m/s minimum, and the ranking shows it as “bound only” rather than as unpublished.
- Multi-part cells. A cell that lists several values under a slash-separated row label is read part by part: on “Thigh+calf / arm length”, a reach line reads the arm length only.
- Capability: share of lines some member meets.
- Reach & envelope: payload, reach, and speed lines.
- Endurance: runtime and battery lines.
- Sim readiness and openness: the best member's readiness score and licence input from
scores.json. - Evidence strength: source grade of the deciding claims (maker and field records 1, reviewed 0.8, press 0.5, analyst 0.4).
A party clears when every line is met and every meter that applies is at least 60. A meter with no lines or no linked readiness entry shows n/a and does not block the clear.
When a line is missing, the verdict names the highest-ranked robot that would meet it.
The URL holds the quest, class filter, and party; the cartridge code is a deterministic hash of that state and the data release.
Subjects are matched to readiness entries by id, exact name, or a name suffix that matches exactly one entry.
Quest log and XP
The Quest log lists every gap the committed data shows, each with what finishing it unlocks. XP is a fixed rule, not a judgement:
- a missing cell is 10, plus 20 when its row feeds the Quest builder;
- a claim resting only on press or analyst sources is 15;
- a source with a scheduled review is 10 plus 2 per claim that cites it, capped at 50;
- a source that is not directly reachable is 20;
- a number in a numeric row that the parser cannot read is 5. A row is numeric when its parsed values share a scale, so a row of named counts alone is not; a number joined to a word, as in “4-mic array” or “3D”, is a name and not a number;
- an unknown sim-readiness input is 30;
- a sim-world robot with no catalogue entry is 40.
Review dates are compared with today's date in the reader's browser, so overdue rechecks rise to the top. The save file is the current data release from history/index.json; Changes lists every earlier save.
Local build: python3 scripts/scores.py rewrites scores.json; --check fails when it is stale.
Result intervals
Result intervals turns each cited success rate that publishes a trial count into a success count, a 95% interval, and a letter tier.
A result without a trial count keeps its value and gets no interval and no letter. Results are compared only within one board: the same benchmark, split, condition, metric, and protocol.
k = round(value × n). The count is marked exact only when k of n prints as the published rate at its published precision, and either the rate lands on k or no other count prints the same.
99.99% of 3 trials is not exact, because 3 of 3 prints as 100.00%. Other counts are marked derived; the source usually averaged over seeds or tasks.
The posterior is Beta(1 + k, 1 + n − k), from a uniform prior. For 0 < k < n the interval runs from its 2.5% to its 97.5% quantile.
At k = 0 the low bound is 0%, and at k = n the high bound is 100%; the other bound is then the one-sided 95% quantile.
The interval therefore always contains the published rate: 0 of 24 gives 0% to 11.3%, and 10 of 10 gives 76.2% to 100%.
For each pair on a board, the probability that one success rate is higher is computed exactly from the two posteriors, so it stays correct for large trial counts at 0% or 100%.
A pair is separated when that probability is above 1 − 0.05/(2m) or below 0.05/(2m), for m pairs on the board. Results that share a letter are not separated.
Letters run A to Z, then AA, AB, so no two tiers share a name.
Local build: python3 scripts/result_intervals.py rewrites catalogue/result-intervals.json; --check fails when it is stale. Contract: result-intervals.schema.json.
Benchmark health
Benchmark health draws one card per benchmark from the result intervals above. Every value is computed from cited rows; nothing is estimated.
Each card shows the best published value and headroom on each board, the policies that share the top tier, the drop under perturbation, and how simulation compares with real robots.
- Saturated: an unperturbed simulated board with at least 2 policies has a best published value of at least 95%.
- Open: no board's best value reaches 50%.
- Leaders can't be separated: more than one policy shares a letter with the best counted row.
- Trial counts mostly missing: fewer than 50% of the benchmark's rows publish a trial count.
Each perturbation board is compared with the in-distribution board that has the same split, metric, and protocol. The drop is the in-distribution value minus the perturbed value for each policy on both boards.
- Collapses: the mean drop on some perturbation board is at least 20 points, over at least 5 policies on both boards.
- With fewer policies the card shows every drop and says Too few policies to judge robustness.
- The worst case is the largest single drop on the card.
The 5-policy minimum is the same as sim against real: a mean over fewer says more about those few models than about the benchmark.
Five is the fewest for which even a drop shared by every policy is unlikely by chance (sign test, 1 in 32).
A simulated board and a real board are paired only when split, metric, and protocol match.
Over the policies on both, the card gives the Spearman rank correlation, MMRV (mean maximum rank violation, Li et al. 2024), and the mean gap.
A pair gives a verdict only with at least 5 shared policies and a real board that separates at least two of them:
- Sim tracks real when every such pair has ρ ≥ 0.8 and MMRV ≤ 10 points;
- Sim disagrees with real otherwise.
With fewer policies the card says Too few policies to judge; three policies in the same order is not evidence.
Two boards are joined by policy name, so a policy listed twice on a paired board stops the build.
The card's status is the first badge present in this order: saturated, sim disagrees with real, collapses, open, sim tracks real, real robot results, unvalidated vs real.
Local build: python3 scripts/benchmark_health.py rewrites catalogue/benchmark-health.json; --check fails when it is stale. Contract: benchmark-health.schema.json.
Corrections and source submissions
Corrections are evidence changes, not votes. Submit the exact page, row, or source record, the replacement primary URL, and the reason the current claim is wrong, stale, or incomplete.
Wrong value, identity, condition, source grade, or stale document.
Open correction form → Submit a primary sourceVendor manual, filing, paper, model card, or official engineering repository.
Open source form → Submit a relationshipExact entity ids, scoped predicate, evidence state, validity dates, and a boundary note.
Open relation form → Contribution rulesIdentifiers, evidence boundaries, validation, and pull-request requirements.
Read contributor guide →Aggregator specifications without a primary document are not accepted. A source blocked to automated probes may remain cited when the access state is recorded explicitly and a human inspected it.
Releases, freshness, and citation
Cite metrics by row anchor, values by claim id, and sources by evidence id.
- The homepage publishes the newest source retrieval date.
build.jsonidentifies the code deployment and its data digest.- What changed opens the last data release on value changes; a code deploy is not a data release.
Select a value cell or use its Metric and Subject controls. The non-modal inspector copies the authored value, qualification, evidence citation, and exact claim URL.
Row-level citation remains available from the row header.
Use sources.html#<topic>-<grade><counter>.
landscape.json preserves canonical claims; ecosystem-graph.json and per-entity dossiers preserve reviewed identities and relations.
A due review means the record has not been re-inspected against its URL since date D. It does not mean the cited number is false, and it never recolors a specification cell.
Share sources.html?fresh=due (alias of ?review=due). A passed review date is a maintainer queue item; CI does not fail because a date elapsed.
Last-release value changes omit unchanged claims. Evidence-id and source URL or retrieval are separate filters. Feeds remain the machine path.
source-observations.json retains source identity, retrieval outcome, content type, digest, observation time, and affected claim ids bound to the canonical snapshot at import.
Retrieval is not verification, and a digest change never rewrites a cell.
- Latest queues compare a successful digest with the previous successful digest through outages.
- A digest change or outage stays open until a human retain, revise, or retire decision; repeating the same bytes does not resolve it.
- Later evidence edits do not erase that trail.
- Claims no longer in the current export are marked historical.
- Mutable vendor product pages (SPA): 90 days after retrieval.
- Versioned PDF or manual: only when a newer version appears.
- Filings: next fiscal filing.
- Analyst reports: the named next report.
- Research and press: never from age alone.
Identifiers in the export
- Entities. Catalogue cards export as entities.
- Claims. Table values export as typed claims with class, metric, table and subject ids, evidence ids, and reported/missing state. Numeric-looking text is not coerced into an unpublished unit.
- Claim id. A claim id is page, table, row, and sorted subject ids — never a column coordinate.
- Claim URL. An exact claim URL stores the stable claim id in
?claim=and keeps the metric row fragment. Reloading restores and outlines the same value cell. - Inspector. The inspector loads the compact comparison index only on first use; it does not fetch the full archival export.
- Data release id. A data release id is the SHA-256 digest of sorted canonical claim and source state. Generated timestamps, commits, file order, row order, and column positions are excluded.
- Release records. The first record is an explicit baseline. Added, removed, and evidence-changed are descriptive labels, not quality judgments.
- Sources. Each exported source carries structured document identity, publication/version fields when authored, retrieval and access status, its direct source URL, any URL fragment, and a stable registry URL.
- Missing metadata. A missing optional field is
null; the exporter does not infer unpublished metadata. - Retrieval date. Retrieval date means “last inspected for the cited claim,” not a guarantee that every mutable vendor page is unchanged. A source refresh changes that date only after review.
Licensing and third-party material
Repository code is MIT-licensed. Original editorial text and the compiled landscape dataset are available under CC BY 4.0.
Vendor manuals, trademarks, photographs, videos, engineering models, quoted passages, and other third-party material remain under their owners’ terms.