Magic Marcus
Published © GPL3+

Animatronic Velociraptor by MM

Animatronic Velociraptor - my dreamed animatronic dinosaur from my favourite universum Jurassic World.

AdvancedFull instructions providedOver 4 days54
Animatronic Velociraptor by MM

Things used in this project

Hardware components

Raspberry Pi 3 Model B
Raspberry Pi 3 Model B
×1
16 channel servo controller - Adafruit or simillar
×1
servo 40kg
×5
servo MG996R
×2
PET-G
×2
Waveshare USB to audio
https://www.waveshare.com/wiki/USB_TO_AUDIO#Working_with_Raspberry_Pi
×1
Screws, nuts, M8, M5, M4, M3, M2
×100
Plywood 4mm
×1
MDF 10mm
×1
Threaded screw 8mm
×2
SG90 Micro-servo motor
SG90 Micro-servo motor
×3

Software apps and online services

depthai operating system
Thonny - IDE

Hand tools and fabrication machines

Multitool, Screwdriver
Multitool, Screwdriver
Laser cutter Ikier-k1-ultra-36w
https://eu.ikier.com/products/ikier-k1-ultra-36w
3d printer Artillery Sidewinder X2

Story

Read more

Custom parts and enclosures

Head to print

You can import to Blender, save as stl and print.

STEP file for import to Fusion360

Unpack and import to Fusion360

Raptor

FBX format.

Schematics

Schematic

Code

Code

Python
This is all of my code for Animatronic Velociraptor.
import asyncio                                     # Imports the asyncio library for asynchronous programming
import depthai as dai                              # Imports the main module for interacting with the OAK-D camera
from adafruit_servokit import ServoKit             # Imports the driver for the PCA9685 PWM servo controller
from pathlib import Path                           # Imports the Path object for handling cross-platform file paths
import cv2                                         # Imports the OpenCV library for image processing and visualization
import time                                        # Imports the time module for tracking target timeouts
from playsound import playsound                    # Imports the audio library

# --- BLOCK 1: CONFIGURATION ---
SERVO_CONFIG = {
    0: (60, 120, 0.04, 90),  # eyeL
    1: (60, 120, 0.04, 90),  # eyeR
    2: (50, 130, 0.05, 95),  # PAN
    11: (45, 150, 0.2, 150), # armR
    12: (45, 150, 0.2, 150), # armR
    15: (90, 180, 0.5, 90),  # TILT                        # Channel 15 (TILT): Head up/down rotation limits and speed
}

TARGET_TIMEOUT = 2.0                               # Defines seconds to wait after losing target before returning to center

kit = ServoKit(channels=16)                        # Initializes the PCA9685 driver for a 16-channel servo board

# Dynamic initialization of arrays based on SERVO_CONFIG (Single Source of Truth)
target_positions = [SERVO_CONFIG.get(i, (0, 180, 0.5, 90))[3] for i in range(16)] # Creates a list of 16 target angles
current_positions = list(target_positions)         # Creates a copy of the list to track current physical servo positions
playsound('roar.mp3')                                 # You can play sound where you need in code


# --- BLOCK 2: SERVO CONTROLLER (Movement Engine) ---
async def servo_updater():                         # Defines the asynchronous function responsible for moving all servos
    #Handles smooth movement of all servos in a single background loop
    while True:                                    # Starts an infinite loop for continuous background servo updates
        for channel, config in SERVO_CONFIG.items(): # Iterates over only the configured and active servo channels
            min_a, max_a, smoothing, _ = config    # Unpacks the configuration tuple for the current channel
            target = max(min_a, min(max_a, target_positions[channel])) # Clamps the target angle within hardware limits
            current = current_positions[channel]   # Retrieves the last known physical position of the servo
            diff = target - current                # Calculates the difference between target and current position
            
            if abs(diff) > 0.5:                    # Applies a deadzone of 0.5 degrees to prevent mechanical jittering
                new_pos = current + (diff * smoothing) # Calculates the next intermediate position using linear interpolation
                current_positions[channel] = new_pos # Updates the software state with the new physical position
                kit.servo[channel].angle = new_pos # Sends the actual I2C command to the hardware to move the servo
                
        await asyncio.sleep(0.02)                  # Pauses the coroutine for 20ms, yielding control and running at ~50Hz


# --- BLOCK 3: OAK-D PROCESSOR & BLOCK 4: DECISION ENGINE ---
async def oakd_tracker():                          # Defines the asynchronous function for vision processing and decision making
    img_count = 0                                       # No. image to write
    pipeline = dai.Pipeline()                      # Instantiates a new DepthAI pipeline object
    nnPath = str((Path(__file__).parent / Path('mobilenet-ssd_openvino_2021.4_6shave.blob')).resolve().absolute()) # Resolves path to neural network
    
    camRgb = pipeline.create(dai.node.ColorCamera) # Creates a Color Camera node in the pipeline
    camRgb.setPreviewSize(1000, 500)                # Sets the preview frame resolution to 300x300 (required by MobileNet)
    camRgb.setInterleaved(False)                   # Configures the image format to planar (CHW)
    camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR) # Sets color format to BGR (standard for OpenCV)
    
    detectionNetwork = pipeline.create(dai.node.MobileNetDetectionNetwork) # Creates a MobileNet object detection node
    detectionNetwork.setBlobPath(nnPath)           # Loads the previously resolved neural network weights
    detectionNetwork.setConfidenceThreshold(0.5)   # Sets the minimum confidence score to 50% to filter false positives
    
    objectTracker = pipeline.create(dai.node.ObjectTracker) # Creates an Object Tracker node to follow bounding boxes
    objectTracker.setDetectionLabelsToTrack([15])  # Tells the tracker to only follow objects with class ID 15 (Person)
    objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM) # Sets the zero-term tracking algorithm
    
    xout_rgb = pipeline.create(dai.node.XLinkOut)  # Creates an XLink output node for sending the video stream to the host
    xout_rgb.setStreamName("rgb")                  # Names the video stream "rgb" for later retrieval
    xout_tracker = pipeline.create(dai.node.XLinkOut) # Creates an XLink output node for sending tracker metadata
    xout_tracker.setStreamName("tracklets")        # Names the metadata stream "tracklets"

    # Linking the Pipeline
    camRgb.preview.link(detectionNetwork.input)    # Sends camera preview frames directly to the neural network
    detectionNetwork.passthrough.link(objectTracker.inputTrackerFrame) # Sends the video frame to the tracker module
    detectionNetwork.passthrough.link(objectTracker.inputDetectionFrame) # Sends the video frame aligned with detections
    detectionNetwork.out.link(objectTracker.inputDetections) # Sends the bounding box data to the tracker
    objectTracker.out.link(xout_tracker.input)     # Sends the calculated tracking data out to the host
    camRgb.preview.link(xout_rgb.input)            # Sends the raw video frame out to the host for visualization

    last_target_time = time.time()                 # Initializes the timer with the current system time
    is_centered = True                             # Sets the initial state flag indicating the robot is in a neutral pose

    with dai.Device(pipeline) as device:           # Boots up the OAK-D camera and uploads the defined pipeline
        q_rgb = device.getOutputQueue("rgb", 8, False) # Gets the video output queue, max size 4, non-blocking
        q_track = device.getOutputQueue("tracklets", 8, False) # Gets the tracker output queue, max size 4, non-blocking
        
        while True:                                # Starts the infinite host-side processing loop
            in_rgb = q_rgb.tryGet()                # Attempts to get a video frame without blocking the thread (Crucial)
            track = q_track.tryGet()               # Attempts to get tracking data without blocking the thread
            
            if in_rgb is not None and track is not None: # Checks if both video and tracking data successfully arrived
                frame = in_rgb.getCvFrame()        # Converts the DepthAI frame object into a standard OpenCV numpy array
                
                if len(track.tracklets) > 0:       # Checks if there is at least one tracked person in the frame
                    last_target_time = time.time() # Resets the idle timer since a target is actively seen
                    is_centered = False            # Marks the flag to false as the animatronic is now actively tracking
                    
                    # Target Lock Logic: Find person with highest confidence
                    best_tracklet = max(track.tracklets, key=lambda t: t.srcImgDetection.confidence) # Selects target with highest confidence
                    
                    for t in track.tracklets:      # Iterates through all currently tracked objects (persons)
                        roi = t.roi                # Gets the Region of Interest (bounding box) of the tracked object
                        rect = roi.denormalize(frame.shape[1], frame.shape[0]) # Converts normalized coordinates back to absolute pixels
                        p1 = (int(rect.topLeft().x), int(rect.topLeft().y))    # Extracts top-left (X, Y) pixel coordinates
                        p2 = (int(rect.bottomRight().x), int(rect.bottomRight().y)) # Extracts bottom-right (X, Y) pixel coordinates
                        confidence = t.srcImgDetection.confidence * 100        # Converts the confidence score to a percentage
                        
                        if t.id == best_tracklet.id: # Process logic ONLY for the best target (Target Lock)
                            x = (roi.topLeft().x + roi.bottomRight().x) / 2    # Calculates normalized center X coordinate (0.0 to 1.0)
                            y = (roi.topLeft().y + roi.bottomRight().y) / 2    # Calculates normalized center Y coordinate (0.0 to 1.0)
                            
                            target_positions[2] = 180 - (x * 180)              # Maps normalized X to PAN servo angle (0-180), inverted
                            target_positions[15] = 180 - (y * 90) + 20         # Maps normalized Y to TILT servo angle, inverted + offset
                            
                            cv2.rectangle(frame, p1, p2, (0, 255, 0), 3)       # Draws a thick GREEN bounding box around the active target
                            cv2.putText(frame, f"TARGET ({confidence:.1f}%)", (p1[0], p1[1]-10), # Prepares the text label. You can add target id: add this ID:{t.id} after TARGET
                                        cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2) # Renders the text in GREEN above the box
                        else:                      # Logic for all other ignored background people
                            cv2.rectangle(frame, p1, p2, (100, 100, 100), 1)   # Draws a thin GRAY bounding box around ignored targets
                            cv2.putText(frame, f"Ignored", (p1[0], p1[1]-10), # Prepares the text label for ignored targets  ID:{t.id}
                                        cv2.FONT_HERSHEY_SIMPLEX, 0.4, (100, 100, 100), 1) # Renders the text in GRAY
                
                else:                              # Execution path if NO people are currently detected
                    time_since_last_seen = time.time() - last_target_time      # Calculates how many seconds have passed since target loss
                    
                    if not is_centered and time_since_last_seen > TARGET_TIMEOUT: # Checks if timeout threshold is exceeded
                        print("Target lost. Returning to neutral position...") # Prints a debug message to the console
                        for channel, config in SERVO_CONFIG.items():           # Iterates over all active servos
                            target_positions[channel] = config[3]              # Resets the target array values to the starting (neutral) angles
                        is_centered = True                                     # Sets the flag to True to prevent redundant array overwriting
                        
                    if is_centered:                # Execution path if the robot is resting in the neutral position
                        cv2.putText(frame, "STANDBY MODE - SEARCHING FOR TARGET...", (10, 30), # Prepares the standby status text
                                    cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 165, 255), 2) # Renders the ORANGE status text on the screen
            
                cv2.imshow("Raptor View", frame)    # Renders the finalized OpenCV frame to a desktop window
                #cv2.imwrite(f"./img/img_{img_count}.jpg", frame) # Save frame if you need
                img_count += 1
                
                if cv2.waitKey(1) == ord('q'): break # Checks if the 'q' key was pressed for 1ms, breaking the loop to shut down
            
            await asyncio.sleep(0.005)             # Yields thread control for 5ms to let the servo_updater run smoothly


# --- BLOCK 5: MAIN SECTION ---
async def main():                                  # Defines the main asyncio entry point
    tasks = [servo_updater(), oakd_tracker()]      # Creates a list containing the servo task and the vision task
    await asyncio.gather(*tasks)                   # Runs all collected tasks concurrently in the event loop


# --- BLOCK 6: ENTRY POINT & FAIL-SAFE ---
if __name__ == "__main__":                         # Checks if the script is being run directly
    try: 
        asyncio.run(main())                        # Starts the asyncio event loop and executes the main function
    except KeyboardInterrupt:                      # Catches the manual interrupt signal (Ctrl+C) from the terminal
        for i in range(16):                        # Iterates through all 16 potential servo channels on the PCA9685
            kit.servo[i].fraction = None           # Cuts the PWM signal, disabling holding torque to prevent motor burnout

Credits

Magic Marcus
1 project • 1 follower

Comments

Related channels and tags
  • Adafruit
  • Raspberry Pi
  • Robotics
  • Robotics