PiPER Hand-Eye Calibration

Piper Hand-Eye Calibration

Eye-in-Hand: The camera is mounted on the end effector of the robotic arm, with the calibration board fixed. The resulting transformation matrix is T_ee_cam (end effector to camera). For grasping applications: T_base_cam = T_base_ee × T_ee_cam

• Eye-in-Hand: The camera is mounted on the end effector of the robotic arm, with the calibration board fixed. The resulting transformation matrix is T_ee_cam (end effector to camera). For grasping applications: T_base_cam = T_base_ee × T_ee_cam


Eye-to-Hand: The calibration board is fixed on the end effector of the robotic arm, with the camera stationary. The resulting transformation matrix is T_base_cam (base coordinate system to camera). For positioning applications: T_ee_cam = T_ee_base × T_base_cam

• Eye-to-Hand: The calibration board is fixed on the end effector of the robotic arm, with the camera stationary. The resulting transformation matrix is T_base_cam (base coordinate system to camera). For positioning applications: T_ee_cam = T_ee_base × T_base_cam


Prerequisites

• Ubuntu 22.04
• ROS2 Humble
• Online Calibration Board Generator（recommended:Original ArUco dictionary）
• ArUco Marker Detection Package
• PiPER Robotic Arm Package
• Hand-Eye Calibration Package
• USB Camera Instrinsic Calibration（Optional）

Calibration Steps

Create a workspace and clone packages

mkdir -p ~/handeye/src
cd ~/handeye/src
git clone -b humble-devel https://github.com/pal-robotics/aruco_ros.git
git clone -b humble https://github.com/agilexrobotics/piper_ros.git
git clone -b humble https://github.com/agilexrobotics/handeye_calibration_ros.git

• Create a workspace and clone packages
  mkdir -p ~/handeye/src
cd ~/handeye/src
git clone -b humble-devel https://github.com/pal-robotics/aruco_ros.git
git clone -b humble https://github.com/agilexrobotics/piper_ros.git
git clone -b humble https://github.com/agilexrobotics/handeye_calibration_ros.git


Build the workspace

cd ~/handeye
colcon build

• Build the workspace
  cd ~/handeye
colcon build


Run the ArUco marker detection program

source ~/handeye/src/install/setup.sh

# marker_id and marker_size must match the actual calibration board specifications
ros2 launch aruco_ros single.launch.py eye:=left marker_id:=582 marker_size:=0.0677

• Run the ArUco marker detection program
  source ~/handeye/src/install/setup.sh

# marker_id and marker_size must match the actual calibration board specifications
ros2 launch aruco_ros single.launch.py eye:=left marker_id:=582 marker_size:=0.0677


Connect the camera, run the camera program, and publish camera image and camera info topics. This example uses an IntelRealSense camera.

# "left" corresponds to the eye parameter in Step 3
# Remap the original camera topics to the topics subscribed by the ArUco detection program
ros2 run realsense2_camera realsense2_camera_node --ros-args \
-p rgb_camera.color_profile:=640x480x60 \
--remap /camera/camera/color/image_raw:=/stereo/left/image_rect_color \
--remap /camera/camera/color/camera_info:=/stereo/left/camera_info

• Connect the camera, run the camera program, and publish camera image and camera info topics. This example uses an IntelRealSense camera.
  # "left" corresponds to the eye parameter in Step 3
# Remap the original camera topics to the topics subscribed by the ArUco detection program
ros2 run realsense2_camera realsense2_camera_node --ros-args \
-p rgb_camera.color_profile:=640x480x60 \
--remap /camera/camera/color/image_raw:=/stereo/left/image_rect_color \
--remap /camera/camera/color/camera_info:=/stereo/left/camera_info


Connect the Piper robotic arm, activate the CAN module, and run the robotic arm program

bash ~/handeye/src/piper_ros/can_activate.sh
source ~/handeye/src/install/setup.sh
ros2 launch piper start_single_piper.launch.py can_port:=can0

• Connect the Piper robotic arm, activate the CAN module, and run the robotic arm program
  bash ~/handeye/src/piper_ros/can_activate.sh
source ~/handeye/src/install/setup.sh
ros2 launch piper start_single_piper.launch.py can_port:=can0


Run the Piper hand-eye calibration program

# mode parameter: eye_in_hand or eye_to_hand (corresponding to eye-in-hand and eye-to-hand calibration modes)
ros2 run handeye_calibration_ros handeye_calibration --ros-args \
-p piper_topic:=/end_pose \
-p marker_topic:=/aruco_single/pose \
-p mode:=eye_in_hand

• Run the Piper hand-eye calibration program
  # mode parameter: eye_in_hand or eye_to_hand (corresponding to eye-in-hand and eye-to-hand calibration modes)
ros2 run handeye_calibration_ros handeye_calibration --ros-args \
-p piper_topic:=/end_pose \
-p marker_topic:=/aruco_single/pose \
-p mode:=eye_in_hand


Control the robotic arm and follow the terminal prompts from the calibration program. Two control methods are available:

• Teach Mode: Click the teach button and directly drag the robotic arm. This is MIT mode, with lower precision than position-velocity mode.
• Gamepad Control：Enables position-velocity mode for higher precision, requiring a gamepad teleoperation program.
• Control the robotic arm and follow the terminal prompts from the calibration program. Two control methods are available:
  Teach Mode: Click the teach button and directly drag the robotic arm. This is MIT mode, with lower precision than position-velocity mode.
  Gamepad Control：Enables position-velocity mode for higher precision, requiring a gamepad teleoperation program.
  

Additional Notes

View the image

ros2 run image_view image_view --ros-args --remap /image:=/aruco_single/result

• View the image
  ros2 run image_view image_view --ros-args --remap /image:=/aruco_single/result


View the ArUco marker pose

ros2 topic echo /aruco_single/pose

• View the ArUco marker pose
  ros2 topic echo /aruco_single/pose


View the robotic arm end effector pose

ros2 topic echo /end_pose

• View the robotic arm end effector pose
  ros2 topic echo /end_pose


The camera info topic contains intrinsic and distortion parameters, which are required by the ArUco marker detection program.

• The camera info topic contains intrinsic and distortion parameters, which are required by the ArUco marker detection program.
  

For other cameras (e.g., industrial cameras, regular USB cameras), additional intrinsic and distortion calibration may be required. These parameters need to be encapsulated into the camera info topic.

• For other cameras (e.g., industrial cameras, regular USB cameras), additional intrinsic and distortion calibration may be required. These parameters need to be encapsulated into the camera info topic.