Next: AI Robotic Arm Plays Connect 4, Who Will Win？

In the previous article, we delved into the creation of a cerebral apparatus capable of conducting the Connect4 game. We briefly introduced various gaming algorithms such as the minimax algorithm, Alpha-Beta pruning algorithm, among others. However, the most crucial algorithm nowadays is the neural network algorithm and deep learning. The neural network algorithm allows computers to think like humans by setting unique environments for learning chess. In this article, we will further explore how to implement the idea of a robotic arm playing chess. In other words, an AI robotic arm that can play the game of chess. If you are interested, I would recommend reading the previous article.

Introduction

The following content is mainly divided into four sections for introduction:

● Information acquisition: The chessboard information is obtained through the camera to conduct the game.

● Information processing: The acquired information is processed to identify the position of the chess pieces. Through the gaming algorithm, the next step of the chess piece is calculated.

● Robotic arm trajectory: The design of how the robotic arm can grasp the chess pieces and the path of placing the chess pieces.

● Integration of functions: The above three functions are combined to achieve AI robotic arm chess playing.

Let see together！

Process

Information acquisition

Environment: Python, OpenCV the lastest, numpy

To begin with, the acquisition of chessboard information is necessary, which includes the chessboard itself and the pieces on it. We utilize the OpenCV library's method cv2.aruco.ArucoDetector(dictionary, parameters) to detect the Aruco QR codes, which marks the position of the Aruco from an image and calculates the position and posture information. This will allow us to determine the entire chessboard's position, including the location of the four corners.

Code: The code uses the cv2.aruco.ArucoDetector(dictionary, parameters) method to determine the position of our board.

dictionary = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_6X6_250)
parameters = cv2.aruco.DetectorParameters()
detector = cv2.aruco.ArucoDetector(dictionary, parameters)
corners, ids, rejectedCandidates = detector.detectMarkers(bgr_data)
rvec, tvec, _ = cv2.aruco.estimatePoseSingleMarkers(corners, 0.05, self.mtx, self.dist)
if rvec is None or len(corners) != 4:
return None
# debug
if DEBUG:
debug_img = bgr_data.copy()
for i in range(rvec.shape[0]):
cv2.drawFrameAxes(debug_img, self.mtx, self.dist, rvec[i, :, :, ], tvec[i, :, :, ],
0.03)
# Draw a square around the marker.
cv2.aruco.drawDetectedMarkers(debug_img, corners)
cv2.imshow("debug1", debug_img)
# Sort the detected QR code corner points in the following order: top left, top right, bottom left, bottom right.
corners = np.mean(corners, axis=2)
corners = (np.ceil(corners)).astype(int)
corners = corners.reshape((4, 2))
cx, cy = (np.mean(corners[:, 0]), np.mean(corners[:, 1]))
res: list = [None for _ in range(4)]
for x, y in corners:
if x < cx and y < cy:
res[0] = (x, y)
elif x > cx and y < cy:
res[1] = (x, y)
elif x < cx and y > cy:
res[2] = (x, y)
else:
res[3] = (x, y)
res = np.array(res)
## debug code
if DEBUG:
debug_img = bgr_data.copy()
for p in res:
cv2.circle(debug_img, p, 3, BGR_GREEN, -1)
cv2.imshow("aruco", debug_img)
return res

Once the chessboard is determined, we can assign different colors to represent the chess pieces. For a clearer distinction, we will use two highly contrasting colors, red and yellow, and label them accordingly.

We will set a logical sequence wherein every time a new chess piece is added to the board, the current state of the chessboard data will be passed on to the gaming algorithm to determine the next move.

Information processing

As previously mentioned, we have acquired the data of the chessboard. The next step is to pass this information to the gaming algorithm to predict the location of the next chess piece.

Here is the pseudocode for the processing:

function model_predict(state, available_actions):
# Convert available_actions to numpy array
available_actions = np.array(available_actions)
# Extend the state to meet the input requirements of the ONNX model
state = np.expand_dims(np.expand_dims(np.array(state, dtype=np.float32), axis=0), axis=0)
# Build the input of the ONNX model
ort_inputs = {self.policy_net.get_inputs()[0].name: state}
# Make model predictions and get the predicted value for each available location
r_actions = self.policy_net.run(None, ort_inputs)[0][0, :]
# Select the optimal drop position according to the predicted value
state_action_values = np.array(
[r_actions[action] for action in available_actions])
argmax_action = np.argmax(state_action_values)
greedy_action = available_actions[argmax_action]
return greedy_action

The main logic of this method is to use ONNX models for model prediction and select the optimal chess piece location based on the predicted results. Firstly, the available locations, which are stored as available_actions, are converted into numpy arrays. Then, the current game state, state, is expanded to meet the input requirements of the ONNX model. The expanded state is then passed to the ONNX model for prediction, and the predicted results are stored in the r_actions variable. Afterwards, based on the predicted results and available move locations, the predicted value of each available location is calculated. The location with the highest predicted value is selected as the optimal chess piece location and returned.

Robotic Arm trajectory

Now that we have both the brain (gaming algorithm) and eyes (recognition algorithm), all we need is a hand to execute the actions. We will use the Python library pymycobot to control the robotic arm. Due to the nature of the chessboard, the chess pieces can only be dropped from above. By assigning a coordinate point for each chess piece in each column, we can plan the trajectory of the robotic arm. As the chessboard is relatively clean without any obstructions, we do not need to consider too many path factors.

Code:

# Initialize and define several coordinate points
# Set a list of length 7
self.chess_table = [None for _ in range(7)]
self.chess_table[0]-[6] = [J1,J2,J3,J4,J5,J6] # seven positions
self.angle_table = {
"recovery": [0, 0, 0, 0, 0, 0], #initial position
"observe": [-2.54, 135.0, -122.95, -32.34, 2.46, -90.35],#observation position
"stack-hover-1": [-47.19, -37.96, -58.53, 1.05, -1.93, -1.84],#Pick up the position of the pieces
}

Next, introduce some methods of controlling the robotic arm in pymycobot:

#Sending the angle to the robotic arm.
self.send_angles(self.chess_table[n], ARM_SPEED)
#Sending the Cartesian coordinates to the robotic arm.
self.send_coords(coord_list,ARM_SPEED)

Integration of functions

Before integrating the function points, we have to sort out the logic between them.

With the flow chart of playing chess, the next step is to combine the function points together.

This is the structure file of the program.

├── Agent.py The logic of the robotic arm's gameplay.

├── ArmCamera.py Camera control.

├── ArmInterface.py Robotic Arm control

├── Board.py Data structure of the chessboard and related judgments.

├── CameraDemo.py Small program for testing camera performance.

├── config.py Utilized to define specific configuration details.

├── Detection.py Machine vision recognition.

├── dqn.pt Neural network model file, used to implement gameplay logic.

├── main.py main program.

├── StateMachine.py A state machine.

The specific codes are not shown here one by one, let's watch a video to see how they perform.

Summary

Theoretically speaking, it is nearly impossible for anyone to win against the machine. This is because the algorithm used by the machine can predict several steps ahead and even more, while an average person can only predict two or three steps ahead at most. However, based on the videos we have seen, the AI has only won one game, and lost the other due to structural issues. The chessboard was supposed to be in one location, but was forced to another.

Do you find this project interesting? We will continue to improve the Connect4 game suite and make it available on our website. If you are interested, please follow us for updates.

Would you like to try implementing other chess games using the robotic arm, such as chess or Chinese chess? Keep in mind that different chess games use different algorithms. We welcome you to share your thoughts with us in the comments section.