Aerial native PID flight
Watch a simulated Crazyflie 2.X quadrotor follow a 0.3 m circle for twelve seconds in PyBullet, under the PID controller that ships with gym-pybullet-drones.
The flight is scored against a tracking-accuracy gate written down before the first trial. Qualification failed: the first trial measured 0.0894 m of tracking error against the 0.05 m limit.
The video shows a second trial, declared after that failure with one change to how the controller is called. It met the same gate at 0.0110 m and is not counted as a pass.
Recording · Native verification · Every tuning flight · Reproduction notes (repository access required)
Qualification result and recording
- Qualification failedDecided by the first trial: 0.0894 m against the 0.05 m limit.
- 0.0894 mFirst trial. Tracking RMSE over samples 49–576. Limit 0.05 m. Failed.
- 0.0110 mSecond trial, declared after the failure. Met the same gate; not counted as a pass.
- 0.0192 mSecond trial. Largest distance from the target in any scored sample. Limit 0.10 m.
Seeking is unavailable here: this video is being served without byte ranges, so the browser can only play it from the start. The recorded state still follows every frame as it plays.
- First trial: source example
- Second trial: reference velocity supplied
- Target path
- Start-up, not scored
Hover the right-hand chart for values. Where the video can be sought, click the chart to move the video to that sample.
Per-sample values for both trials are in tracking.json; the unrounded capture is in samples.jsonl.
Both trials, side by side
| Item | First trial | Second trial |
|---|---|---|
| Gate | Committed 14 September 2026, before any flight. | The same bytes: limits, metric, circle, scoring window, seed and horizon. |
| Result recorded | 17 September 2026. | 9 October 2026. The change was committed at 18:09 UTC and the flight ran at 18:15 UTC. |
| Controller input | Position target only, as the example ships. | Position target and its velocity. Gains unchanged. |
| Recording method | Queried link states with forward kinematics recomputed, which altered the flight. | Passive. An unobserved repeat of the flight is identical. |
| Tracking RMSE, limit 0.05 m | 0.0894 m: not met. | 0.0110 m: met. |
| Largest error, limit 0.10 m | 0.0994 m: met. | 0.0192 m: met. |
| Lowest altitude, at least 0.03 m | 0.0907 m: met. | 0.0908 m: met. |
| Drone contacts, limit zero | 0: met. | 0: met. |
| Unchanged example, passive method | 0.0910 m RMSE and 0.1062 m largest error: neither limit met. | Not applicable: this trial is the passive method. |
| Independent motor replay | Exact over 176,482 state values. | Exact over 92,817 state values. |
| Gate checks | Not all met: tracking RMSE. | All met. |
| Standing | The qualification trial. Result: failed. | Declared after the failure, with the protocol’s rule amended. Not counted as a pass. See what was fixed beforehand. |
The tuning flights that led from the first trial to the second are in the table below.
What the recording establishes
| Item | Recorded evidence |
|---|---|
| Example | gym-pybullet-drones 2.2.0 at commit 7ebad1ec, examples/pid.py with one CF2X drone, no obstacles, PyBullet physics at 240 Hz and control at 48 Hz for 12 s. Original controller gains and source bytes. The example’s default is three drones with obstacles; the one-drone variant is authored here. |
| Declared gate | Written before the first trial and unchanged since: tracking RMSE at most 0.05 m and no sample farther than 0.10 m from its target over samples 49–576; altitude at least 0.03 m and zero drone contacts at every one of 2,880 physics steps; the full horizon flown. The example itself defines no success score: its reward is a constant −1. |
| First trial | The example as selected. RMSE 0.0894 m, maximum 0.0994 m: the RMSE check fails. This is the qualification result. Its evidence is kept unchanged. |
| Second trial | Declared after that failure. The same controller also receives the velocity of the target. RMSE 0.0110 m, maximum 0.0192 m, zero contacts, lowest altitude 0.0908 m: every check is met. Flown once on this trajectory. Not counted as a pass. |
| Independent replay | A fresh native setup applies the 576 recorded motor commands without evaluating the controller or restoring recorded states. All 577 samples and 92,817 compared state values of the second trial match exactly, and its gate checks are met again. |
| Video | 577 native RGB frames, 640 × 480, 48 fps; 384,037-byte H.264 video lasting 12.02 s. Every decoded frame is compared with its input; minimum PSNR 40.05 dB. |
| Publication | One simulation recording with synchronized camera and state, of the second trial. Zero qualified tasks, additional browser world assemblies, learned checkpoints or reproduced evaluation suites. |
Why the first trial failed
Almost all of the first trial’s error is lag along the path. Its along-track error is 0.0888 m RMS, against 0.0098 m across the path and 0.0010 m in altitude: 98.8 % of the squared error.
The drone flies the right circle about 0.47 s behind its target.
The cause is in how the example calls its controller. The position loop computes a force from the position error, its integral, and the difference between a target velocity and the measured velocity.
The example passes no target velocity, so that term pulls toward standing still while the target moves at 0.188 m/s.
A linear model of this loop alone, with the attitude loop treated as ideal, predicts a steady distance of 0.0965 m and a lag of 0.50 s. The flight measures 0.0894 m and 0.47 s.
Other explanations were checked and ruled out:
- Motor limits: commands stay 27 % of their range away from either limit.
- Model mismatch: controller and simulator use the same mass, thrust coefficient and gravity, and altitude holds to 1 mm.
- Start-up: the first second is not scored, and the error is larger in the second half of the scored window (0.095 m and 0.092 m by quarter) than in the first (0.082 m and 0.088 m).
- Control rate: running the controller at 240 Hz instead of 48 Hz leaves the unchanged example at 0.0937 m.
One thing is saturated, and it is not the cause. At 48 Hz the attitude loop’s torque command sits on its ±3,200 clip on at least one axis in 525 of the 528 scored samples of the first trial and 524 of the second.
At 240 Hz it never clips, and the unchanged example’s tracking error is no better.
What changed for the second trial
The second trial supplies the missing input: the time derivative of the declared circle at the same waypoint, passed through the controller’s own target_vel argument.
Gains, seed, trajectory, horizon, limits, scoring window and source bytes are unchanged. A test pins the hash of every declared section of the first trial’s protocol.
The same linear model predicts 0.0082 m with the velocity supplied.
The change was chosen on three other trajectories, not on the scored circle. Candidates were compared by a rule written before those flights:
- acceptable only with no motor command at a limit, no contact, and error not above the unchanged example on any tuning trajectory;
- among acceptable candidates the lowest mean error wins;
- unchanged gains are preferred when within 10 % of the best.
| Candidate | Faster circle | Figure eight | Line with reversals | Outcome |
|---|---|---|---|---|
| Unchanged example | 0.1001 | 0.1219 | 0.0687 | Baseline |
| Reference velocity, exact derivative | 0.0147 | 0.0207 | 0.0164 | Chosen: lowest mean, gains unchanged |
| Reference velocity, finite difference of waypoints | 0.0140 | 0.0238 | 0.0167 | Acceptable; higher mean |
| Position gain doubled | 0.0490 | 0.0633 | 0.0369 | Acceptable; changes a gain and halves the lag only |
| Velocity gain halved | 0.0542 | 0.0706 | 0.0426 | Acceptable; changes a gain |
| Both gain changes | 0.4248 | 0.5690 | 0.3389 | Rejected: unstable, hits motor limits and the ground |
After the scored flight, the same settings were flown sixteen more times on the scored circle with things the gate does not vary:
- simulated mass and thrust 10 % high and low;
- a 5 cm offset at the start in each axis;
- the simulator’s drag model and its ground-effect, drag and downwash model;
- control at 240 Hz;
- 2 mm of position noise on five seeds.
All sixteen stay within the RMSE limit. The worst is 0.0213 m with thrust 10 % low, and the largest single error is 0.0280 m. The outer Python and NumPy seeds have no effect on the unperturbed flight.
What remains after the change is mostly across the path. The drone flies the circle 8.6 mm wide, because nothing supplies the centripetal acceleration and the controller has no input for it.
How the two trials compare
The two trials differ in two ways, and only one is a change to the flight.
Reference velocity is the intended change. The controller, its gains and the gate are the same; the controller is told how fast its target moves. This is why the second trial tracks better.
Passive observation is a correction to the recording method. It does not help the drone: the same example scores 0.0894 m under the first method and 0.0910 m under the second, so the corrected method is the stricter one.
The like-for-like baseline for the second trial is therefore 0.0910 m, not 0.0894 m. Both fail; the second trial’s 0.0110 m is measured with the stricter method.
What was fixed beforehand, and what was not
The gate was committed on 14 September 2026, before any flight. Its limits, metric, circle, scoring window, seed and horizon are byte-identical for both trials, and a test pins the hash of each section.
That protocol also carried this sentence: “Keep the first bounded trial and report failures. Do not change seed, controller, horizon, criteria or source bytes to select a passing outcome.”
The second trial changes what the controller is given. The sentence was amended on 9 October 2026 in commit 88ed2a30, after the first trial failed, so that the protocol admits it. The gate’s numbers did not move.
Amending the rule after a failure loosens the protocol. The qualification result is therefore the first trial: failed, at 0.0894 m against 0.05 m.
The second trial is shown as what it is: a later trial, with one changed controller input, that met the same gate at 0.0110 m. Nothing on this site counts it as a qualified pass.
Commit 88ed2a30 also holds the change and the capture code. It reached the repository at 18:09:55 UTC. The second flight ran from 18:14:59 to 18:15:54 UTC on exactly those files.
Before that flight, the scored circle was flown only by the unchanged example: six logged diagnostic flights and fourteen debugging flights that the tuning log does not list.
None of those supplied a reference velocity. The chosen change was compared on three other trajectories and then flown on the scored circle once, before its result was known.
The selection rule was first committed with the declaration, after the tuning flights, and the tuning log with the result. Their order rests on the log’s timestamps, not on commits.
A repository cannot show that a flight was not made. That the chosen change did not fly the scored circle earlier rests on the working record of the session, which was read again for this audit.
The diagnosis came from the first trial on the scored circle, and a linear model predicted 0.0082 m for the change before it flew. The change itself has no parameter that could be fitted to the circle.
The second trial was audited again on 9 October 2026 with code written apart from the capture scripts. Rescoring the published samples gives the same 0.0110 m and 0.0192 m.
Replaying the 576 published motor commands in a fresh simulator, with no controller and no capture code, returns all 11,520 observation values with a largest difference of zero.
Observation without interference
The first capture changed the flight it recorded. At every sample it asked PyBullet for link states with forward kinematics recomputed.
PyBullet then rewrites its cached link frames, and gym-pybullet-drones applies each rotor’s thrust in those frames.
Left alone, the cached frames are the base pose one physics step earlier; refreshed, they are current for one step in five.
The same example with no observer query at all measures 0.0910 m RMSE and 0.1062 m maximum error, not 0.0894 m and 0.0994 m. Both versions fail the gate; the unobserved one also fails the maximum-error check.
The capture now reads the cached frames without recomputing them and derives the five fixed link poses from the measured base pose.
After each capture it flies the same example again with no camera and no state query, and requires every command and every returned observation to be identical: 13,824 values, largest difference zero.
The coordinate audit checks 4,039 body transforms using 16,156 points. It compares the derived link poses with native forward kinematics in a separate simulator: largest difference 5.5e-8, tolerance 1e-6.
It also confirms that the cached frames equal the base pose one physics step earlier, up to 1.25 mm from the current pose.
The camera is not affected by that lag. The renderer draws the current base pose, and the capture checks it.
The centre of the drone’s silhouette stays within 0.40 px RMS of the projected base position of the same sample, against 1.36 px and 1.46 px for the samples one frame later and earlier.
Time, state and controls
Sample 0 follows setup, before the first command. Each later sample follows one control interval of five 1/240 s physics steps.
State and camera clocks are 48 Hz. Encoded frame i has presentation time i/48 s, and the final state at 12 s occupies the last frame.
The example computes each command after stepping, so the first applied command is zero RPM and applied step k uses waypoint k − 2.
Tracking error compares each state with the target of the command applied during that interval.
Native sample downloads retain the 20-value drone observation, base pose and velocity, the cached link frames, supplied and clipped motor RPM, and each command’s position and velocity targets.
The action is four motor RPM values in native prop0–prop3 order. Native right-handed z-up positions are preserved; xyzw quaternions are reordered to wxyz without renormalization.
The source loop ends after 576 control steps. The metadata marks a recording boundary, with recording_truncated=true only at the final sample.
No reward is published: the source’s constant −1 is kept per sample as native_reward, beside its false ending flags, as the placeholders they are.
Original inputs and native behaviour
The pinned gym-pybullet-drones tree supplies 51 verified files. The input receipt retains Git identities, modes, sizes and SHA-256 hashes.
The asset audit checks six active asset files against original Git objects and the PyBullet source distribution. It records one unresolved texture reference in the plane material that the pinned loader does not read.
The isolated runtime pins Python 3.12.3, PyBullet 3.2.7 built from source, NumPy 2.5.3, SciPy 1.18.1 and CPU PyTorch 2.14.0 with hash locks.
It ran on Ubuntu 24.04 with a Ryzen 9 7950X3D; physics and rendering use the CPU.
PyBullet compiles its build time into its extension, so a rebuild is never byte-identical.
The runtime rebuilt for the second trial reproduces the first trial’s samples, physics journal, commands and outcome byte for byte.
Attribution and reproduction
gym-pybullet-drones is MIT-licensed. Its PID controller’s source credits “work conducted at UTIAS’ DSL” and names six contributors. The CF2 mesh credits Lakshman Kumar.
PyBullet is distributed under the zlib licence. Original authors retain their rights; this site generated the rendered video and measured-state recording.
Original geometry and engine distributions remain in native storage.
The protocol states the arguments, clocks, camera, gate, replay tolerances, the declared change for the second trial and the first trial’s result.
Capture, replay, tuning and audit scripts are in this site’s private repository; recipe links require repository access.
One scripted flight in simulation does not establish real-flight performance or a benchmark score.
The gate is this site’s own: meeting it says a recorded flight tracked its target within the stated distance, nothing more.
The frontier sequence continues with the marine station hold.