Shucheng Jia

Autonomous Dual-Arm Tic-Tac-Toe

Personal project · Duke University · Fall 2025

Two Kinova Gen3 Lite arms play a full game autonomously (16× speed).

Overview

I built a dual-arm robot that plays Tic-Tac-Toe on its own, using ROS 2, an RGB-D camera and MoveIt 2 on two Kinova Gen3 Lite arms. The system covers the full pipeline: it sees the board, decides each move, plans the motion and picks and places pieces on real hardware. The two arms play opposite colors and take turns, each one choosing its move from the current board state.

~90%
Task success in Gazebo
~50%
Task success on real arms
2 arms
Kinova Gen3 Lite, taking turns

System

Dual-arm Tic-Tac-Toe ROS 2 systemRGB-D cameraoverhead,color + depthPerceptiondetect pieces by colordepth → 3D centroidsTF → robot frameGame braintracks board statewin, else blockmove service/left_arm PlayNoderequests next moveplans with MoveIt 2→ Kinova Gen3 Lite/right_arm PlayNoderequests next moveplans with MoveIt 2→ Kinova Gen3 LiteMutexNodeone arm in theshared zone at a timemovemoveafter every move, the camera re-reads the board
ROS 2 system. Each arm runs in its own namespace; a mutex keeps the two arms out of the shared zone at the same time.
Camera view with two red pieces detected and their centroids marked
Piece detection on the camera image. Each blob is filtered by area, aspect ratio (0.9–1.15) and solidity before its centroid (blue) is used.

The same pipeline in Gazebo simulation (2× speed).

Workspace analysis

Before running anything on hardware, I checked in MATLAB that the shared workspace was both reachable and safe. I replicated the two arm placements, sampled the task space at all nine board cells, solved inverse kinematics for both arms, and measured the clearance between the active arm's target pose and the passive arm's reachable workspace.

3D plot of both arms' reachable workspaces over the board Grid showing all nine cells reachable by both arms
Left: the two arms' reachable workspaces (green) overlap over the board. Right: all nine cells are reachable by both arms.

Lessons learned

The system reached about 90% task success in Gazebo and about 50% on the real arms. On hardware, most failures came from camera calibration error, visual noise, and the table color interfering with piece detection. These errors shift the final grasp position, so a grasp can miss even when MoveIt 2 plans and executes correctly. Perception accuracy, not planning, was the bottleneck in going from simulation to the real robot, even though the real setup reused the exact same code, board layout and camera TF as the simulation.

The top-down grasp works because the pieces are regular blocks. For irregular objects I tried GraspNet, which gives full 6-DoF grasp poses; it ran in MuJoCo simulation, but integrating it with the ROS 2 stack is left for future work.