AgileX NERO Robotic Arm Control with OpenClaw

1.Introduction

The recent popularity of OpenClaw has sparked a wave of enthusiasm for building personal robotic assistants. In this tutorial, we’ll demonstrate how to write an OpenClaw Skill that enables simple control of the AgileX NERO robotic arm.

Before proceeding, make sure you have already installed and configured the OpenClaw environment. Please refer to the official OpenClaw documentation for installation instructions.

2. Writing an OpenClaw Skill

Inside your OpenClaw agent workspace, create the following files under the skills directory:

.
├── config
│   └── hands_ctrl.yaml
├── scripts
│   └── hands_ctrl.py
└── SKILL.md

In this example, we are using the Three Provinces and Six Ministries multi-agent architecture, so the skill is placed inside the workspace of the Crown Prince agent, as shown below.

Don’t worry if you are not using this multi-agent architecture. The following implementation works equally well for a standard OpenClaw workspace.

2.1 SKILL.md

The contents of SKILL.md are shown below:

---
name: hands_ctrl
description: Use when the user wants OpenClaw to perform physical hand gestures like shaking hands or waving, or to recover/reset the hardware task. Executes the corresponding Python script based on user intent and securely interrupts any ongoing gesture before starting a new one.
---

# Gesture Control for OpenClaw

Use this skill when the user wants OpenClaw to act as a gesture controller for the hardware or robotic system.

## Inputs

Accept natural language commands or explicit action requests, such as:

- "握手", "shake hands", "let's shake"
- "挥手", "wave at me", "say hello"
- "恢复", "恢复任务", "recover", "reset"

Derive the intended action (`shake`, `wave`, or `recove`) from the user's input before execution.

## Interruption Handling (Ctrl + C)

Hardware can only perform one gesture safely at a time to prevent motor conflicts. 
If the user requests a new gesture or a recovery command while a previous `hands_ctrl.py` process is still executing, you MUST interrupt the active process first. Send a `Ctrl + C` (SIGINT) to the running process to safely cancel the current hardware action before executing the new command. Ensure no orphaned background processes are left behind.

## Modes

### Handshake

Use when the user issues a handshake command.

Produce the following execution:

~~~bash
python3 skills/scripts/hands_ctrl.py --action shake
~~~

Implement this execution with:
- strict passing of the `--action shake` argument.
- capturing of standard output to confirm the hardware received the command.

### Wave

Use when the user issues a wave command.

Produce the following execution:

~~~bash
python3 skills/scripts/hands_ctrl.py --action wave
~~~

Implement this execution with:
- strict passing of the `--action wave` argument.
- capturing of standard output to confirm the hardware received the command.

### Recover

Use when the user issues a command to recover or reset the task.

Produce the following execution:

~~~bash
python3 skills/scripts/hands_ctrl.py --action recove
~~~

Implement this execution with:
- strict passing of the `--action recove` argument.
- capturing of standard output to confirm the hardware received the command.

## Backend Rules

Prefer executing the provided script over reimplementing the logic. Use `skills/scripts/hands_ctrl.py` as the sole backend interface for these gestures. Do not attempt to modify or rewrite the hardware control logic within the script unless explicitly asked to do so. Ensure process termination (SIGINT / Ctrl + C) is handled gracefully by the system.

## Packaging Rules

- The execution context must be at the root of the workspace so that the relative path `skills/scripts/hands_ctrl.py` is valid.
- Ensure the Python environment has the necessary dependencies installed to run the script.

## Workflow

1. Acquire and parse the user's intent from the prompt.
2. Analyze whether the intent maps to the Handshake, Wave, or Recover mode.
3. Check if there is an active `hands_ctrl.py` process currently running.
4. If a process is running, send a `Ctrl + C` (SIGINT) to terminate it and wait for it to stop completely.
5. Verify the existence of the `skills/scripts/hands_ctrl.py` file locally.
6. Execute the command corresponding to the matched mode.
7. Capture execution logs (`stdout` and `stderr`).
8. Update the user on the success or failure of the hardware action, clearly stating if a previous action was interrupted via Ctrl + C.

## Output Expectations

When reporting progress or final results, include:

- detected gesture intent (shake, wave, or recove)
- whether a previous process was interrupted via Ctrl + C
- the exact script command executed
- validation of execution (e.g., success message or error trace)
- open risks or hardware backend limitations

2.2 How the Skill Works

This Skill simply maps natural-language commands to different execution modes.

• When the user says “shake hands”, OpenClaw executes:

python3 skills/scripts/hands_ctrl.py --action shake

• When the user says “wave”, it executes:

python3 skills/scripts/hands_ctrl.py --action wave

• When the user requests “recover” or “reset”, it executes:

python3 skills/scripts/hands_ctrl.py --action recove

All three commands invoke the same backend script : hands_ctrl.py. The actual hardware control logic is implemented inside this Python script. The Skill itself is responsible only for invoking the script with the appropriate command-line argument.

2.3 hands_ctrl.py

The implementation of hands_ctrl.py is shown below.

import time
import argparse
import yaml
from pyAgxArm import create_agx_arm_config, AgxArmFactory


def wait_motion_done(robot, timeout: float = 5.0, poll_interval: float = 0.1) -> bool:
    """Wait until the robotic arm reaches the target position or the operation times out."""
    time.sleep(0.5)
    start_t = time.monotonic()

    while True:
        status = robot.get_arm_status()

        if status is not None and getattr(status.msg, "motion_status", None) == 0:
            return True

        if time.monotonic() - start_t > timeout:
            print(f"Timed out waiting for motion completion ({timeout:.1f}s)")
            return False

        time.sleep(poll_interval)


# Set the following three parameters to None by default so that the
# "recove" action can be executed independently without requiring them.
def main(action_name, pose_prepare=None, pose_left=None, pose_right=None):

    # Create the robotic arm configuration and establish the connection
    cfg = create_agx_arm_config(robot="nero", comm="can", channel="can0")
    robot = AgxArmFactory.create_arm(cfg)
    robot.connect()

    # Switch to Normal Mode and enable CAN communication
    print("Switching to Normal Mode and enabling CAN communication...")
    robot.set_normal_mode()
    time.sleep(1)

    # Enable the robotic arm
    print("Enabling the robotic arm...")
    while not robot.enable():
        time.sleep(0.01)

    print("Robotic arm enabled successfully.")

    # Set the motion speed percentage
    robot.set_speed_percent(80)

    # Center (safe) pose of the robotic arm
    pose_center = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

    if action_name == "recove":
        print("\nExecuting recovery action: moving the robotic arm to the safe position...")
        robot.move_j(pose_center)
        wait_motion_done(robot, timeout=8.0)
        time.sleep(1)
        print("The robotic arm has returned to the safe position. Program finished.")
        return

    print(f"Executing action: {action_name}")
    print(
        f"Motion parameters:\n"
        f"Preparation pose: {pose_prepare}\n"
        f"Left pose: {pose_left}\n"
        f"Right pose: {pose_right}"
    )

    try:
        print("Moving to the center pose...")
        robot.move_j(pose_center)
        wait_motion_done(robot, timeout=8.0)

        print("Moving to the preparation pose...")
        robot.move_j(pose_prepare)
        wait_motion_done(robot, timeout=8.0)

        print(f"Starting continuous '{action_name}' motion (Press Ctrl+C to stop)...")

        cycle_count = 0

        while True:
            cycle_count += 1

            print(f"Cycle {cycle_count} - Pose 1")
            robot.move_j(pose_left)
            wait_motion_done(robot)

            print(f"Cycle {cycle_count} - Pose 2")
            robot.move_j(pose_right)
            wait_motion_done(robot)

    except KeyboardInterrupt:
        print("\nMotion interrupted by the user. Returning to the center pose...")
        robot.move_j(pose_center)
        wait_motion_done(robot)

    finally:
        time.sleep(1)
        print("Program finished.")


if __name__ == "__main__":

    parser = argparse.ArgumentParser(
        description="Control the AgileX NERO robotic arm using predefined motions loaded from a YAML configuration file."
    )

    # Path to the YAML configuration file
    parser.add_argument(
        "--config",
        type=str,
        default="skills/config/hands_ctrl.yaml",
        help="Path to the YAML configuration file (default: skills/config/hands_ctrl.yaml)",
    )

    # Action to execute
    parser.add_argument(
        "--action",
        type=str,
        choices=["wave", "shake", "recove"],
        default="wave",
        help='Name of the action to execute. It must match a key in the YAML file. '
             'The "recove" action is built-in and does not require YAML parameters '
             "(default: wave).",
    )

    args = parser.parse_args()

    if args.action == "recove":
        main(args.action)
        exit(0)

    try:
        with open(args.config, "r", encoding="utf-8") as f:
            config_data = yaml.safe_load(f)

    except FileNotFoundError:
        print(f"Error: Configuration file '{args.config}' not found.")
        exit(1)

    except yaml.YAMLError as e:
        print(f"Error parsing the YAML configuration file: {e}")
        exit(1)

    # Retrieve the parameters for the selected action
    actions_dict = config_data.get("actions", {})

    if args.action not in actions_dict:
        print(f"Error: Action '{args.action}' was not found in the configuration file.")
        exit(1)

    selected_action = actions_dict[args.action]

    pose_prepare = selected_action.get("pose_prepare")
    pose_left = selected_action.get("pose_left")
    pose_right = selected_action.get("pose_right")

    if not all([pose_prepare, pose_left, pose_right]):
        print(
            f"Error: The action '{args.action}' is missing required parameters "
            "(pose_prepare, pose_left, pose_right)."
        )
        exit(1)

    main(args.action, pose_prepare, pose_left, pose_right)

3. Script Overview

This script controls the AgileX NERO robotic arm using predefined motion parameters stored in a YAML configuration file.

The YAML file defines each motion sequence using three joint-space poses:

• Preparation pose
• Left pose
• Right pose

When the program starts, it performs the following steps:

• Creates the robotic arm configuration and establishes the connection.
• Switches the robot into normal operating mode.
• Enables CAN communication.
• Enables the robotic arm.
• Sets the motion speed.
• Executes the requested action based on the command-line argument.

Three actions are currently supported:

• Wave
• Shake Hands
• Recover

For both the wave and shake actions, the robot first moves to a preparation pose and then continuously alternates between the left and right poses until the user interrupts the program with Ctrl+C.

When interrupted, the script automatically returns the robotic arm to its center (safe) position before exiting.

The recover action is a built-in recovery command that immediately moves the robotic arm back to its predefined safe position.

4.YAML Configuration

The motion parameters are stored in hands_ctrl.yaml.

actions:
  wave:
    pose_prepare: [0.8, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
    pose_left: [0.8, 0.0, 0.0, 0.6, -0.6, 0.0, 0.0]
    pose_right: [0.8, 0.0, 0.0, -0.6, -0.6, 0.0, 0.0]

  shake:
    pose_prepare: [0.0, 0.6, 0.0, 1.0, 1.57, 0.0, 0.0]
    pose_left: [0.0, 0.6, 0.0, 1.0, 1.57, 0.0, 0.0]
    pose_right: [0.0, 0.6, 0.0, 0.6, 1.57, 0.0, 0.0]

Each action consists of three joint-space poses:

• pose_prepare – The initial pose before starting the motion.
• pose_left – The first motion pose.
• pose_right – The second motion pose.

The Python script loads these poses at runtime and executes the corresponding motion sequence according to the selected action.

5.Demo

After completing the configuration above, you can control your OpenClaw-powered robotic assistant using natural language and command the AgileX NERO robotic arm to perform simple gestures such as waving, shaking hands, and recovering to a safe position.

The demonstration is shown below.

FAQ

Q1:Can OpenClaw Control Real Robots?

Yes. OpenClaw is responsible for task understanding and Skill execution. Through Skills, it can connect to external programs and hardware devices, enabling control of real robots.

Q2:What Is an OpenClaw Skill?

An OpenClaw Skill is a modular extension that expands the capabilities of an AI Agent. Developers can define trigger conditions and execution logic in a SKILL.md file, allowing the Agent to invoke the appropriate scripts based on natural language instructions.

Q3: How Do You Use OpenClaw to Control the NERO 7-DoF Robotic Arm?

The control workflow includes the following steps:

• Install and configure the OpenClaw environment.
• Create a robot control Skill.
• Develop a Python script for hardware control.
• Configure robot motion parameters using a YAML file.
• Trigger robotic arm actions through natural language commands.

Q4: Why Use a YAML Configuration File?

A YAML file stores the motion parameters of the robotic arm, separating motion data from the control logic. This design allows developers to create or modify robot actions simply by updating the pose parameters in the YAML file, without changing the underlying Python control code.

Q5:Is Controlling a Robot with OpenClaw Considered Embodied AI?

The combination of OpenClaw and physical robot hardware represents a practical approach to enabling AI Agents to interact with the physical world. It is one of the promising directions being explored in the field of Embodied AI.

📷 Have Question?

If you encounter any issues with environment installation, parameter configuration, or RL training, feel free to leave your questions for further discussion.