Benchmarking

To see where V3 stood, I benchmarked it against MuJoCo 3.15 on the same arm scenes, converted from the same .arow files, at the same 2 ms step, single-threaded on my laptop (Intel Core Ultra 9 285H). Batched across SIMD lanes, Achilles steps a seven-link arm 3.3× faster than MuJoCo's best single-threaded mode, reaching 5.45 million robot-steps per second at 64 robots. In double precision, MuJoCo's own, Achilles is still 1.5–1.6× faster at most batch sizes, and its results match MuJoCo's to about one part in 10¹¹. The one loss is a single seven-link arm, where most SIMD lanes sit empty (0.97×).

A VLA can look great near its training setup and fall apart a few centimeters away, so the hard part is deciding what a fair test is. We picked eight tasks across six kinds of challenge and changed the objects' placement and the wording on each of ten rollouts. We randomized the placements within a set workspace before running any evaluation, and every method ran the same ones. If I ran it again, the biggest thing I'd change is the lighting: we didn't control it well, and the models performed noticeably differently early in the morning than late at night.

Success alone also hides a lot, so in the main benchmark each rollout was scored three ways: did it succeed, what fraction of the goals were met, and how far along a written rubric it got. Separately from that benchmark, we ran an ablation on wording alone: one task with the apple held still and only the instruction changed, fifteen phrasings run once each. π0.5 succeeded on "Grab the apple and set it on the pan" and failed on "Take the apple and place it onto the pan." It was sensitive to how specific the prompt was: given an underspecified one like "Put the apple away," the arm wandered without targeting the apple before eventually turning to the task, and an overly specified prompt degraded it the same way. It struggles when the task's intent is missing or buried.

MolmoAct2 was strong on the banana but struggled with the other pick-and-place tasks, which suggests it's fragile to object type, and on the harder tasks it often misread the prompt entirely, pressing the top of the cabinet instead of closing the drawer. It also disengaged early: in the two-object task it moved one object and withdrew, and in the cluttered box it picked the wrong object and never came back for the right one. Across all of it, the VLAs failed in the same few ways: sensitivity to how the prompt was worded, dependence on the object (π0 never got the baseball into the bucket), and poor spatial reasoning. They ignored vertical clearance, like the height of a barrier or how high to lift a cloth before setting it aside, and when a task called for a specific procedure they defaulted to another one that couldn't finish it, grabbing the drawer's lip instead of pushing it closed.

Every VLA ships its own server, and they don't agree on how to talk: openpi's π0 and π0.5 use WebSockets, GR00T uses ZMQ and MolmoAct2 uses HTTP. I wrote one harness that hides all of that behind a single interface: a new model, task or whole benchmark is a short YAML file, not new code.

On the robot, the harness asks the policy for a chunk of actions, runs eight of them open loop, then asks again, recording the scene and wrist cameras throughout. The operator scores each rollout, everything is saved as it goes so an interrupted run resumes, and one script turns the results into success rates per task and per prompt.

Over winter break I took SKIDADDLE apart into pieces that each do one job. Controllers now only ask for accelerations: one along the path, one back onto it, one for heading, each with a weight and a priority. A scheduler runs whichever are active, and a default controller brings the robot to a stop when none are. A new allocator turns the requests into motor space and hands out each motor's remaining headroom, highest priority first and shared by weight within a priority, so correcting off the path can come before speeding along it without any motor going past its limits. The look-ahead, before the most expensive part of the loop, became a precompute: when a path is made, one backward pass finds the fastest speed the robot can enter each turn at, so the loop only reads it.

Planning in motor space means a turn's sideways acceleration comes out of the same wheel budget as speeding up and slowing down. Road Runner limits acceleration only along the path, so on a tight hairpin at the same limits its plan asks for more grip than the wheels have, falls behind and overshoots the end. With its limit lowered to 30 in/s² it stays within 9 in; SKIDADDLE stays within 4.5 in and stops 0.7 s sooner. It isn't ahead everywhere: on sharp corners Road Runner with the lower limit tracks tighter, and the rework doesn't correct sideways error yet, so it can stop a few inches off the end point.

Our chassis followed paths with pure pursuit: aim at a point a fixed distance down the path and ramp the speed up toward it. The robot's accelerations stay implicit in that, so it has no idea how hard it can brake or how fast it can take a corner, and at speed it swings wide of turns and overshoots where it should stop. I proposed replacing it with path tracking that models those accelerations explicitly, and led the feature from idea to implementation: I wrote the design and gathered the resources behind it, from spline fitting and arc-length reparametrization to the Riccati equation and a study that found LQR tracks more accurately than pure pursuit. First, a cubic spline through the waypoints is reparametrized by arc length, the speed at each point is capped by its curvature, and a forward and a backward pass keep every speed-up and every stop within what the robot can deliver. Then an LQR tracks that profile on a world-frame model of the chassis, where its dynamics stay constant, and its force is rotated into the robot's frame. I designed it and built the framework; the comparison here runs the finished MVP against our pure pursuit in simulation.