Hardware components | ||||||
| × | 1 | ||||
| × | 1 | ||||
| × | 1 | ||||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
| ||||||
Hand tools and fabrication machines | ||||||
 |
| |||||
Today I received a package containing the few Elephant Robotics products. The shipment is well packaged and fully protected from mechanical impacts during transport. The package contains the myCobot 280 M5 robot arm, myCobot Camera Flange 2.0 and myCobot Adaptive Gripper. There is also a G-shape Base 2.0 which serves to attach the robot arm to the base.
The main component in this kit is the Robotic Arm. Тhe box contains the Robotic Arm, power supply, a USB cable and wires for connecting external modules. There is also the basic instructions, certificate and links. From the very first glance, it is clear that it is solidly made and of high quality, and I will examine its functionality later.
MyCobot 280 is 6-DOF collaborative robot (Cobot) wich weighs 850g, has a payload of 250g, and an effective working radius of 280mm. It is small in size but powerful in function. It can be adapted to various application scenarios such as scientific research, education, smart home, and commercial research. As the name suggests, this model features an m5 stack, with three buttons as well as a large number of input/output ports and pins for controlling additional external devices. It supports multiple programming languages like Python, C++, Arduino, C# etc., and also the platforms Windows, Android, Mac OSX and Linux. The overall structure of the mechanical arm is simple and stylish and contains 6 high-performance servo motors with fast response, small inertia, and smooth rotation.
And before I start describing my custom project, I want to describe a simple way to "drag teaching". Drag Teaching is an intuitive method for programming the movements of a robotic arm, especially common in cobots and specialized industrial robots. Instead of using complex code, the operator can literally grab the robot arm and physically guide it through the desired path. The robot can then reproduce the recorded path with high precision at normal operating speed. In this way, we can most easily capture the performance of this Robotic Arm, such as accuracy, speed and smoothness of motion.
The following is a description of the main project that will demonstrate the functionality of all three components: the Robotic Arm, the camera, and the gripper. So I decided to develop a version of the well-known game Rock-Paper-Scissors in which each player simultaneously forms one of three shapes with an outstretched hand - Rock, Paper, And Scissors.
At a specific moment, which will be indicated on the display and with an audio signal, the human and the robot will show one of the three shapes and then, according to the rules of the game, the winner will be declared. Let me emphasize that the robot generates a shape completely randomly and at the same moment when it receives information that the human has displayed a shape.
Let's start with the Python part. Here we use myCobot Camera Flange 2.0 which is connected to the PC via a USB cable. On Win10 and Win11 OS the camera is installed automatically. First of all, a Python environment with appropriate support (libraries) should be installed on the PC. Python version 3.11 is installed from the given link and during installation the "Add Python to PATH" option should be checked.
The Python program can use multiple tools to perform the given task. First I created a code with the OpenCV library but the ability to determine the shape of the hand was relatively weak. Then I created a code with the TensorFlow platform which worked well but the whole installation procedure was very complex. In the end I found the simplest but functional solution using the MediaPipe framework from Google.
It is necessary to install several components by running these two lines in the CMD (command prompt) one by one:
python -m pip install --upgrade pip
python -m pip install "mediapipe==0.10.20" opencv-python pyserial
At the end of this text, the Python code named "MediaPIPE_Code" is given. We just need to start this Python script and in the cmd window we need to select the serial port on which the M5Stack microcontroller is installed. In my case, it is COM17 (meaning option 1). Now a window with the video camera appears, and now we need to show the appropriate shape in front of the camera. As you can see, the program detects the three shapes with great accuracy. In the command window, you can see the result that is sent to the serial port.
Next comes the Arduino part, which is significantly more complicated, so I will try to explain the installation method in detail. First, let's install support in Arduino IDE specifically for this M5Stack microcontroller.
Next, we need to install the support libraries.
Go to Sketch --> Include Library --> Manage Library, and In the search box, enter M5Stack. I will install the M5Stack library version 0.4.5
In the same way we install the "MyCobotBasic" library.
Now we will do some simple tests to check the communication between the Robotic Arm and the PC, as well as the functionality of the libraries. In Arduino, we select the appropriate port (in my case COM17) and upload the code named "First communication TEST".
At this stage, there is no need to connect the 12V power supply. If everything is OK, certain information will appear on the M5Stack display. We can also test the functionality of the buttons.
Next, we connect the 12v power supply as well as the gripper and upload the following code named "Systematic test Code". This code first sets all the servos (Joints) to the default position which is 0 Degrees, and then tests one by one all the servo motors as well as the Gripper.
If this part also works, then we are done with testing and we need to move on to the implementation of the specific task which is the Rock, Paper, Scissors interactive game Human vs Robot.
For this purpose, I attached the robotic arm to a solid base for stability, and placed the camera on a special stand, although I could have mounted it directly on the robotic arm in a certain way (for example, it could have been mounted directly above the gripper, but for this particular project, this is impossible because the camera and the matrix display would always be in the opposite direction.). I did this for practical reasons, to protect the camera from unwanted arm movements during testing, and also with this type of mounting the device is universal and ready for other projects.
So we need to connect the myCobot Camera Flange 2.0 camera to the PC USB port and activate the Python script. On the M5Stack microcontroller, we first need to upload the code named "Rock_Paper_Scissors_Game".
***************** Youtube video **************************..
Three options are displayed on the screen. At the beginning, the arm is in some arbitrary position. Now we press the B button, which gradually takes the arm into a position ready for the game. When it stabilizes, to start the game, we press the A button. Now the robot straightens up, and the matrix display lights up in blue, which indicates that the game has started. At the same moment, a countdown of 3, 2, 1 begins, after which we have to show a certain shape with our hand in front of the camera.
Until the human shows a shape, the robot stands still and waits for data from the human.
Upon receiving such data, the robot immediately randomly generates one of the three characters Rock, Paper or Scissors. To repeat, Rock is a closed Gripper, Paper is an open Gripper, and Scissors is an open-closed gripper animation. When both players show their shape, it is displayed on the display, on the upper half - human, and on the lower - robot. Now the winner is determined, if the Robot wins, the message "I WIN" appears on the TFT display on a green surface, the upper matrix display also lights up green and the robot tilts back. If the man wins, the message "YOU WIN" appears on a red background on the TFT display, the upper matrix display lights up red and the robot leans forward, that is, bows. If the result is a draw, the message "TIE" is displayed on a yellow background on the TFT display, the upper matrix display lights up yellow and the robot moves left and right.
The game is accompanied by appropriate audio effects throughout. After this, the game ends and the robot returns to the "ready to play position". The "C" button has the function of displaying the total score at any moment of the game. By pressing it, the score is displayed and automatically disappears after two seconds.
And Finally short conclusion. This project demonstrates a complete interactive game between human and robot — the classic Rock-Paper-Scissors — using the MyCobot 280 M5 robotic arm. The robot recognizes human hand gestures via a Python + MediaPipe vision system and responds with real-time motion, LED feedback, sounds, and on-screen results displayed on the M5Stack TFT display. It’s a perfect blend of hardware control, AI gesture detection, and embedded programming.
Comments