OpenDubs

On October 17, OpenDubs went downtown for the ARCS Foundation's Night to Ignite Science, a fundraiser for STEM PhD fellowships at the UW and Washington State that raised over $900,000. Four of us from the lab presented it to donors. The robot danced and moved its arms to the Monster Mash: a routine others in the lab put together, running on the arm infrastructure I built.

I really enjoyed the chance to talk with and give back a little to the people who make my research at UW possible. I talked the visitors through the motivation behind OpenDubs, my work on the project, and my passion for robotics. It was great to practice pitching my work to people with an interest in the field, but without intense technical depth.

The afternoon teleop first ran on the robot, I led a demo for twenty to thirty executives from Winbond, the memory chip maker, shortly before Winbond's $12 million gift to the UW College of Engineering. I prepared and ran it: the robot drove around the room while I teleoperated both arms and explained how it works.

Two days earlier, Fauna Robotics' team visited the lab, and I gave them the same tour: driving the robot around and talking through how it works.

A two-arm plan is only safe if both arms run it together: it was checked with each arm where the other would be at the same instant. On hardware, though, mj_manipulator_ros, the lab's layer that runs those plans on the robot, ran multi-arm plans one arm after the other, which doubled the time and passed through states the planner never checked. Replaying the crossing plan that way in MuJoCo and checking every fourth of its states, the arms intersect in 97 of 1,400.

I rebuilt execution so every arm starts its trajectory on one shared start stamp: the trajectories must share their timestamps, one stamp a moment in the future is computed once, and each arm sends its goal from its own thread against that stamp, so the motors start together however long each goal takes to send. If any arm fails or is aborted, a shared event cancels the others mid-motion. Running two arms from two threads also meant making the shared code thread-safe: both threads stepped one MuJoCo model, and concurrent runs crashed until I serialized every access to it behind one lock. To bring it all up on the real robot safely, I built a hardware-in-the-loop bring-up protocol.

The lab plans with its own MuJoCo planning library, mj_manipulator, which planned one arm at a time. With two arms on one torso, two plans that are each safe can still hit each other. I added an ArmGroup to it that plans all arms' joints together, so every step is checked arm against arm. On the clip's goals, the two arms planned separately collide in 58 of 210 frames; ArmGroup's plan bends one gripper about 16 cm around the other and touches in none. The same group plans to joint goals, gripper poses and lockstep Cartesian paths, keeps 5 mm between all geometry, retimes the trajectory at 125 Hz, and re-checks it at every sample. I made single-arm planning a one-arm group, and ArmGroup is what plans both arms on the robot today.

For demonstrations and live driving, I built teleop with a Meta Quest. The apps I tried had no passthrough and left me dizzy, so I wrote my own: instead of an app on the headset, the robot serves a page the Quest's browser opens in AR, so the operator still sees the room, and the page streams both controllers' poses, grip and trigger to the robot over a websocket. Holding a grip engages its arm, and the trigger works the gripper. The headset's page goes dark once it enters AR, so the robot also broadcasts that live telemetry to any other browser pointed at it, and someone at a laptop can watch exactly what the operator is sending. It matches the old SDK's interface, so switching was a one-line change. I built it against the arms in MuJoCo first; at first the loop couldn't keep up, and the bottleneck turned out to be publishing the sim viewer over the network, not the control.

Teleop and planning both lean on ssik, the lab's analytical IK solver, which I didn't write, and on the OpenArm it kept failing. I traced it to a singularity in the solver itself: the OpenArm's joint axes meet at shared points, so its link lengths are zero and every twist is a right angle, and that exact geometry breaks the Raghavan–Roth elimination the solver uses. My fix nudges those zero lengths by a tiny fraction of the arm's size, so the solver works on an arm a hair off the real one, then, since forward kinematics is cheap, refines the answer against the true arm's FK, step after step, until it's an exact solution again. The maintainer confirmed the fix would have worked while landing the same one upstream.

Each arm is a 7-DOF OpenArm whose Damiao motors run their own impedance loop: the Thor sends position, velocity and torque targets with gains over CAN-FD, and each motor closes the loop itself. I measured the stock ROS 2 stack first. A goal is accepted in 2.55 ms, and with both arms holding one block, 47 ms passes between a command to one arm and the torque spike in the other (σ = 4.7 ms).

What I was after was axial compliance in Cartesian space: a gripper that holds firm along some axes and gives along others. I wrote three controllers with Pinocchio to get it (Cartesian, joint and null space). Pure Cartesian impedance gave that compliance, but it left the redundant elbow free to drift off the collision-checked path. The combined controller ran joint-space impedance and subtracted the Cartesian terms from it, keeping the axial compliance and control of the null space, but it was numerically unstable. The null-space controller worked, but I switched off it for the much lower latency of the motors' own impedance loops: in the end we gave up Cartesian compliance for speed.

What I settled on is simpler: the planned trajectory goes straight to the motors, plus gravity compensation at 500 Hz from Pinocchio's RNEA (the recursive Newton-Euler algorithm), with every goal checked against position and torque limits in both joint space and Cartesian space (through forward kinematics).

For localization and a map, the base carries two RPLIDAR A1 2D lidars, one at the front and one at the back. Bringing them up, with the team, was mostly fighting USB: the two units have identical USB IDs, so udev rules name each by the hub port it's plugged into, and they power up a few seconds apart so neither times out.

Mapping and Nav2 expect one scan, so I wrote a node that merges the two. The two scans never line up on their own: nothing guarantees the lidars spin at exactly the same rate or finish a revolution at the same moment, and each reaches the Thor with its own latency. So the node pairs scans by their timestamps (any two within 0.1 s), looks up each scan's transform into the robot's frame at that scan's own timestamp, resamples both to the finer of the two angular resolutions, and keeps the nearest return in each direction of one 360° scan, stamped halfway between the two and dropping hits on the robot itself. On top of it, I set up slam_toolbox and Nav2, which drove the robot point to point in simulation.

Before the base could navigate, it had to know where it was. The first source was optical flow: a sensor that watches the floor slide past. I brought it up over micro-ROS and fed it into ArduPilot's EKF alongside the IMU of the Pixhawk the base ran at the time. It only read sensibly within about 10 cm of the floor, so it sits low, lit by its own LED.

To tell whether it worked, I needed ground truth. The lab's motion capture tracks the robot more precisely than anything you could measure by hand, so I wrote the package that brings it into ROS: the EKF's estimate and the mocap pose land in one bag on one clock, and every run can be compared with the truth point by point. The first runs showed why that mattered: driving 5.5 m, the estimate stayed within 0.74 m of mocap, but spinning in place sent it 155 m off. Once teammates found the sensor's mounting missing from the config, I tuned the flow gain against mocap, run by run, until the estimate stayed within 5 mm of motion capture.

OpenDubs runs on an NVIDIA Jetson Thor, so the first job was getting it ready for the whole team. I flashed JetPack onto it, got its IO talking to the rest of the robot, and set up the Docker environment we all develop in: a ROS 2 image with the build tools, Makefile and VS Code tasks to build and debug inside it, and per-person git identities so four people can commit from one shared machine.

The first version of the base drove its four mecanum wheels through a Pixhawk running ArduPilot, over MAVROS; it has since dropped the Pixhawk. With another intern, I brought up that drive code, then added what the rest of the stack needed: a URDF and an SDF with collision geometry for RViz and Gazebo, and a service that records topics to rosbags. Every test after this was a bag I could replay.

The base has since been rebuilt by others in the lab (I didn't work on it), without the Pixhawk, and carries everything else: a 51.2 V LiFePO4 battery, an emergency stop on the deck and a status display, all wired into one aluminum frame, with the arms, the Thor and the lidars riding on top.