MeArm TicTacToe

# TrisVision MeArm: Mixed Reality and Computer Vision

## Introduction

The project consists of the development of an interactive robotic system based on a 4 degrees of freedom (4DOF) arm with a parallelogram structure (MeArm). The system is capable of challenging a human to a game of Tic-Tac-Toe, identifying moves (X) on a sheet of paper via a webcam and physically responding on the board (O).

The software is written in Python and communicates with a microcontroller via a BT-24 Bluetooth Low Energy module.

---

## Software

The architecture is modular, separating the vision logic, artificial intelligence, and hardware communication:

Vision/Main (`trisvision.py`): Uses OpenCV to track the playing field, recognize symbols (X and O), and render graphics in Mixed Reality.

Logic (`tris_logica.py`): Contains the algorithm (Minimax) to calculate the robot's move.

Communication (`ble_link.py`, `config.py`): Interface using the `bleak` library for asynchronous sending of commands (servo motor angles) to the Arduino via Bluetooth.

---

## Calibration

To allow the robot to interact accurately with the real world, several calibration phases were necessary.

### Spatial Calibration

Since the camera frames the table, the image has perspective distortions. The software dynamically searches for the 4 sides of the white paper and applies perspective correction (`warpPerspective`). By pressing the 'B'*key (Memory Lock), the system freezes the transformation matrix, allowing the robot to enter the scene without disrupting the geometric recognition of the board.

### Color Calibration

Target objects were isolated from the environment by adjusting the Min/Max thresholds for Hue (H), Saturation (S), and Value (V). In this case, the color of a yellow circle glued to the gripper was calibrated (`calibra_EE.py`), which is essential for visually tracking the position of the end-effector.

### Electromechanical Calibration

The hardware (SG90 micro-servo motors) has significant mechanical tolerances. Instead of using complex kinematic calculations, the system uses a static mapping (`MAPPA_CASELLE`) where a safe combination of angles (Base, Shoulder, Elbow) has been saved for each of the 9 Tic-Tac-Toe squares. This decouples the mathematical model from the physical problem, offloading hardware imperfections onto a manually tunable configuration file (via `test_ble.py`).

---

## Problem Solving

### Mechanical Limits and Motor Strain

The parallelogram structure of the arm made it difficult to maintain a constant height, risking crashing the gripper into the table or stripping the motors' gears.

Solution (Mixed Reality): The robot does not physically manipulate objects. It travels at a safe height, hovering over the field. The webcam tracks the yellow dot and, once positioned, the computer draws a virtual 'O' on the screen, blending the physical movement with a digital response.

### Visual Occlusion and False Positives

The black acrylic of the arm, upon entering the frame, was falsely detected by the Otsu filter as an 'X'. Furthermore, shadows and folds in the paper created visual noise.

Solution (Sensor Muting): The software disables the search for 'X's during the robot's turn, ignoring the moving hardware.

---

## Conclusion

The project demonstrates the integration of an advanced computer vision system with low-cost hardware control. The inherent limitations of budget mechanics were entirely compensated for through rigorous software design and the creative use of Mixed Reality.