Spider bot

Big Bertha: On-Device RL Locomotion for a 12-DOF Quadruped

A $1, 600 legged robot problem, solved with a 450 KB neural network and one board.

Pitch: A 12-DOF quadruped that learns to walk entirely in simulation, then maps, plans, and navigates fully on-device on a single Arduino UNO Q, with no GPU and no tethered laptop.

This Is Big Bertha

Big Bertha is a 12-DOF quadruped (four legs, three joints each) that learned to walk entirely in NVIDIA Isaac Lab simulation and now runs that same learned gait on real hardware, off a single Arduino UNO Q. No GPU on the robot, no laptop tethered to it. It maps its environment with a 2D LiDAR, plans paths with Nav2, and turns those paths into footsteps through a neural network small enough to fit in 450 KB.

It's built from parts anyone can source: 3D-printed PLA+ legs and 12 hobby MG995 servos, the same class of hardware that usually limits cheap quadrupeds to pre-scripted walk cycles on flat tables. Big Bertha instead runs a policy trained with reinforcement learning, transferred from simulation straight onto the physical robot.

How It Works

Train → Export → Deploy, same neural network file the whole way through.

• Train: In NVIDIA Isaac Lab, thousands of parallel simulated robots train a PPO policy against 27 reward terms (velocity tracking, foot-air-time, torque/smoothness penalties) until a real trot emerges.
• Export: The policy exports to ONNX as a single 450 KB file.
• Run: A C++ ROS 2 Jazzy node built on ONNX Runtime reads a 52-D observation (/odom, /imu, /joint_states, /cmd_vel) at 50 Hz and outputs 12 joint targets, each clamped to the joint's real mechanical limit.

The key design choice: the policy listens on /cmd_vel, the standard ROS 2 velocity topic, so the stock SLAM + Nav2 navigation stack can drive it without knowing it's talking to a neural network. SLAM maps, Nav2 plans and sends velocity commands, and the policy turns those into footsteps.

Why Arduino UNO Q

The robot has two totally different compute jobs: irregular Linux-class work (ROS 2, SLAM, Nav2, inference) and rigid real-time work (12 servos needing precisely-timed PWM). The UNO Q has both (an arm64 Linux processor plus a real-time microcontroller) on one board, so the split doesn't cost us a second board, extra wiring, or latency.

There's also no GPU on the robot, which is why the policy is ONNX (not PyTorch), the runtime is C++ (not Python), and the model is 450 KB, not tens of megabytes. It all shares one modest CPU, and it works.

Build

• Frame & legs: 3D-printed PLA, 12-DOF (hip/thigh/calf × 4), driven through the PCA9685 over I2C.
• Sensing: YDLidar X2 for 2D scanning, BNO055 IMU fused with odometry for pose.
• Power: 3S LiFePO4 + LiPo feeding two buck converters (light rail + 200W/20A servo rail), isolating electronics from servo current spikes.
• Compute: Arduino UNO Q runs the entire ROS 2 stack and the ONNX policy node, no companion computer.

Training Phase

Getting a reward function that produced a survivable gait took several failed attempts before it converged. Two problems kept showing up:

The policy found a trot when we were rewarding a crawl. A crawl_gait term was meant to hold exactly one foot in the air at a time, but early versions gated it incorrectly, and the policy discovered it could still collect reward while running 2–3 feet airborne at once, which is fast in simulation and destructive on a real MG995 servo.

Direction bugs zeroed out reverse without us noticing at first. Several terms were originally written against +x instead of along the commanded direction, so commanding the robot backward earned nothing on those terms, and the policy simply avoided reversing rather than learning to do it well.

# Crawl-style reinforcement: exactly one sustained swing while moving.
feet_air_c = self._contact_sensor.data.current_air_time[:, self._feet_ids]
n_swing = (feet_air_c > 0.06).float().sum(dim=1)
lead_air = torch.clamp(feet_air_c.max(dim=1).values, max=0.45) / 0.45
single = (n_swing == 1.0).float()

# Gate on progress ALONG the command, not along +x, or reverse gates
# the crawl shaping to zero.
fwd_gate = torch.clamp(fwd_vel * torch.sign(self._commands[:, 0]) / 0.10, 0.0, 1.0)
crawl_gait_reward = lead_air * single * fwd_gate

# Squared, not linear: linear only doubles the cost from 2 airborne to 3,
# which never bought a tripod. Squaring makes it 4x.
multi_swing_pen = torch.square(torch.clamp(n_swing - 1.0, min=0.0))

Even with this in place, the footfall analysis in Testing below shows the learned behavior still keeps ~2.2 feet loaded on average and is statically unstable 75% of the time, closer to a trot than a true crawl. The reward term shapes the gait; it doesn't fully dictate it.

Software Stack

LiDAR/IMU → state estimation → SLAM/AMCL → Nav2 (/cmd_vel) → PPO policy (ONNX) → 12 joint targets → servo PWM

Each joint group gets its own scale (sized to real mechanical margin), and every target gets a final hard clamp, so a bad inference can't send a servo past its limit.The real apply-action code from policy_controller_node.cpp, clamping every action before it ever reaches a servo:

for (int i = 0; i < bbpc::kActionDim; ++i) {
  double a = std::clamp(action[i], -action_clip_, action_clip_);
  const double scale = (i < 4)   ? action_scale_        // coxa
                       : (i < 8) ? thigh_action_scale_   // thigh
                                 : shin_action_scale_;   // shin
  double t = scale * a + obs_.default_joint_pos[i];
  target_pos_[i] = std::clamp(t, -joint_limit_, joint_limit_);
}

Each joint group gets its own scale (sized to real mechanical margin), and every target gets a final hard clamp — a bad inference can't send a servo past its limit.

Teleop

Because the policy only ever consumes /cmd_vel, it doesn't matter who's publishing to it. We drove Big Bertha manually with keyboard teleop over the same topic Nav2 uses autonomously: no separate manual-control code path, same policy, same 450 KB file.

Testing

• Isaac Lab: 98% commanded-velocity tracking. Footfall analysis showed the gait is closer to a trot than a crawl: only ~2.2 feet loaded at once, statically unstable 75% of the time, peak contact force 144 N.
• Gazebo Harmonic: Full stack validated (SLAM mapping, AMCL localization, Nav2 planning), monitored in RViz/PlotJuggler.
• Hardware: Same ONNX file, same configs, real robot; only the sensor drivers and servo bridge changed.

Challenges & Next Steps

Reward shaping (getting a survivable crawl instead of a hardware-destroying trot) and the sim-to-real gap were the hard parts: 98% tracking in Isaac vs. 35% in Gazebo. Next: domain randomization for servo latency/friction, a proper torque-speed actuator model, and closed-loop foot-contact sensing

Team Big B: Jjateen Gundesha (Lead), Jayesh Thakare (Dev), Mohit Talwar (Support). IIIT Nagpur ·