Fleet RL

In online RL an arm practises a task over and over, so after every try something has to put the peg back where the task starts. I built that reset for the peg-insertion task. A teammate and I designed the tagging together, and they wrote the code that reads it: ten AprilTags on the peg's faces, from which the scene camera solves the peg's full pose. I calibrated each camera with a ChArUco board, first its lens (the intrinsics), then its pose relative to the arm (the extrinsics). For the second, a calibration script has the arm set the tagged peg down at a grid of spots on the board and fits the arm's joint zeros and the camera's pose to where the arm thought the peg was against where the camera saw it. With that, the reset finds the peg anywhere in the workspace, works out how it is lying, and plans every waypoint of the pick and the place before the arm moves, choosing the grasp the arm can reach with the least travel within its joint limits. It then checks where the peg landed. This lets the teammate's automatic success detection score each try, so the arms can run online RL on their own. The reset has run on a real arm; the clip shows the same planner in simulation.