Isaac Nason
Published © CC BY-NC-ND

Aegis Control: AI-Powered Sentry Turret & Drone System

Forge your own autonomous guardian. The Aegis Control is an AI-powered sentry turret with drone integration.

AdvancedFull instructions providedOver 1 day155
Aegis Control: AI-Powered Sentry Turret & Drone System

Things used in this project

Hardware components

Raspberry Pi 4 Model B
Raspberry Pi 4 Model B
4GB Recommended
×1
Raspberry Pi 4 Camera Module OV5647
×1
PCA9685 16 CH 12Bit PWM Servo Motor Driver
×1
Servo Motor High Torque Coreless Metal Gear 35KG
×2
Stainless Steel Dowel Pin Cylindrical Assortment Kit
×1
Set Screw Collars 5.05mm Bore Shaft Collars 10mm OD, 5mm Width
×1
20V Battery
×1
Relay RF Control Switch
×1
Regulator Buck Converter Constant Current Power Supply Module
×2
High Power MOS Tube FET Trigger Switch Driver Module
×1
Stainless Steel M3 M4 M5 M6 Phillips Flat Head Machine Screws
×1
25ft - 1/2 inch PET Expandable Braided Sleeving
×1
16 Gauge 33 Feet Tinned Copper Electrical Wire
×1
Breadboard Jumper Ribbon Cables Kit
×1
XT60 Parallel Y Splitter Battery Connector
×1
XT60 to Type-C Fast Charging Cable
×1
5 Pairs XT60 Pigtail,XT60 Male and Female Plug
×1
Y Connector 3 Pin Plug Splitters Servo Extension Male to Female Servo Extension Cord
×1
SpeedyBee F7 Mini Flight Controller Stack
×1
Espressif ESP32-C5-DevKitC-1-N8R4
×1
BN-220 GPS Module
×1
Happymodel 0.42g Tiny 2.4G ExpressLRS Receiver
×1
iFlight 4pcs XING2 1404 3000KV Unibell Motors
×4
Gemfan Hurricane 4024 2-Blade Props
×1
11.1V 2800mAh LiPo Battery 3S 30C
×2
2 Packs Power Wheel Adapter for Dewalt 20V Battery Adapter
×1
Flex Ribbon Extension Cable for Raspberry Pi Camera - Black 1m/3.28ft Long
×1
Servo Horn 25T
×1
6mm D-Shaft (Stainless Steel, 20mm Length)
×1
PLA Filament 1.75mm Black
×1
Rigid Flange Shaft Coupling (3/4/5/6mm (Mixed), 8)
×1
6mm D-Shaft Servo Hub 25T
×1

Software apps and online services

Raspbian
Raspberry Pi Raspbian
Bookworm headless
Roboflow
Arduino IDE
Arduino IDE

Story

Read more

Schematics

Aegis Control Turret Schematic

Code

turret_controller.py

Python
This Python script is the central brain of the Aegis Control system. I've designed it with a multi-threaded architecture to handle the complex tasks of video streaming, AI processing, and real-time motion control simultaneously.

The code uses separate, managed threads for the camera feed, servo movements, and AI inference. This ensures that even while the AI is processing a frame, the web interface and manual controls remain perfectly responsive. Features like motion easing for smooth servo movement and a safe shutdown sequence are also built-in.

While the core functionality is robust, there are a couple of important points to be aware of when implementing this code:

1. (CRITICAL) WiFi Setup & Security:
The script currently contains experimental routes (/setup, /api/scan-wifi, /save-credentials) for setting up WiFi via the web interface. These routes rely on sudo commands for network management.

Warning: This method is NOT RECOMMENDED for a final build as it poses a security risk. For a secure and reliable headless setup, I strongly advise removing these routes from the Flask app and instead implementing a system-level script (like the one in the main guide) that automatically handles switching between a known WiFi network and an Access Point mode on boot.

2. AI Model & Tracking:

Model Output: The AI tracking logic is written to parse the output of a specific YOLOv8 TFLite model. If you train your own custom model, you may need to adjust the code in the ai_processing_thread to correctly parse the output tensor for bounding box coordinates and confidence scores.

Tuning: The TRACKING_P_GAIN constant in the script acts as a sensitivity setting for the auto-tracking. You will likely need to tune this value to get the smoothest and most responsive tracking performance with your specific servos.

This project is a complex and rewarding challenge, and I welcome any feedback or suggestions for improving the code!

Happy building!
import io
import time
import threading
import secrets
import subprocess
import cv2
import numpy as np
from tflite_runtime.interpreter import Interpreter
from flask import Flask, Response, request, send_from_directory, jsonify
from smbus2 import SMBus
from picamera2 import Picamera2
from gpiozero import LED
import json
import os
import NetworkManager

# --- Configurations ---
BUS_NUMBER = 0
BOARD_ADDRESS = 0x40
SERVO_FREQ = 60
PAN_CHANNEL = 0
TILT_CHANNEL = 1
SERVO_POWER_PIN = 17
PULSE_MIN = 150
PULSE_MAX = 600
PULSE_CENTER = (PULSE_MAX + PULSE_MIN) // 2

# --- AI & Motion Configuration ---
MODEL_PATH = 'best_int8.tflite'
LABELS_PATH = 'labels.txt'
CONFIDENCE_THRESHOLD = 0.4
IM_WIDTH = 1280
IM_HEIGHT = 720
TRACKING_P_GAIN = 0.08
current_speed = 'MEDIUM'
speed_settings = {'SLOW': 2, 'MEDIUM': 8, 'FAST': 20}
SLOWDOWN_DISTANCE = 120
MIN_SPEED_STEP = 1

# --- Global State Variables ---
current_token = None
motion_thread_lock = threading.Lock()
frame_lock = threading.Lock()
shared_frame = None
current_pan_pulse = PULSE_CENTER
current_tilt_pulse = PULSE_CENTER
target_pan_pulse = PULSE_CENTER
target_tilt_pulse = PULSE_CENTER
current_mode = 'MANUAL'

# --- Hardware Initialization ---
servo_power = LED(SERVO_POWER_PIN)
servo_power.off()
bus = SMBus(BUS_NUMBER)
bus.write_byte_data(BOARD_ADDRESS, 0x00, 0x00)
prescale = int(25000000.0 / (4096 * SERVO_FREQ) - 1.0)
old_mode = bus.read_byte_data(BOARD_ADDRESS, 0x00)
new_mode = (old_mode & 0x7F) | 0x10
bus.write_byte_data(BOARD_ADDRESS, 0x00, new_mode)
bus.write_byte_data(BOARD_ADDRESS, 0xFE, prescale)
bus.write_byte_data(BOARD_ADDRESS, 0x00, old_mode)
time.sleep(0.005)
bus.write_byte_data(BOARD_ADDRESS, 0x00, old_mode | 0xA1)
print("PCA9685 initialized.")

# --- Single, Global Camera Object ---
picam2 = Picamera2()
picam2.configure(picam2.create_video_configuration(main={"size": (IM_WIDTH, IM_HEIGHT)}))
print("Camera configured.")

# --- Threads ---
def camera_capture_thread():
    global shared_frame
    picam2.start()
    print("Camera capture thread started.")
    while True:
        frame = picam2.capture_array()
        with frame_lock:
            shared_frame = frame

def motion_control_loop():
    global current_pan_pulse, current_tilt_pulse
    while True:
        with motion_thread_lock:
            max_speed_step = speed_settings.get(current_speed, 8)
            pan_error = target_pan_pulse - current_pan_pulse
            pan_distance = abs(pan_error)
            pan_speed_step = max_speed_step
            if pan_distance < SLOWDOWN_DISTANCE:
                slowdown_factor = pan_distance / SLOWDOWN_DISTANCE
                pan_speed_step = max(MIN_SPEED_STEP, max_speed_step * slowdown_factor)
            if pan_distance <= pan_speed_step:
                current_pan_pulse = target_pan_pulse
            elif pan_error > 0:
                current_pan_pulse += pan_speed_step
            else:
                current_pan_pulse -= pan_speed_step
            tilt_error = target_tilt_pulse - current_tilt_pulse
            tilt_distance = abs(tilt_error)
            tilt_speed_step = max_speed_step
            if tilt_distance < SLOWDOWN_DISTANCE:
                slowdown_factor = tilt_distance / SLOWDOWN_DISTANCE
                tilt_speed_step = max(MIN_SPEED_STEP, max_speed_step * slowdown_factor)
            if tilt_distance <= tilt_speed_step:
                current_tilt_pulse = target_tilt_pulse
            elif tilt_error > 0:
                current_tilt_pulse += tilt_speed_step
            else:
                current_tilt_pulse -= tilt_speed_step
            set_servo_pulse(PAN_CHANNEL, int(current_pan_pulse))
            set_servo_pulse(TILT_CHANNEL, int(current_tilt_pulse))
        time.sleep(0.015)

def load_labels(path):
    with open(path, 'r') as f:
        return {i: line.strip() for i, line in enumerate(f.readlines())}

def ai_processing_thread():
    global target_pan_pulse, target_tilt_pulse
    labels = load_labels(LABELS_PATH)
    interpreter = Interpreter(model_path=MODEL_PATH)
    interpreter.allocate_tensors()
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    height = input_details[0]['shape'][1]
    width = input_details[0]['shape'][2]
    
    while True:
        if current_mode == 'AUTO':
            with frame_lock:
                if shared_frame is None:
                    time.sleep(0.05)
                    continue
                frame = shared_frame.copy()
            frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            frame_resized = cv2.resize(frame_rgb, (width, height))
            input_data = np.expand_dims(frame_resized, axis=0)
            if input_details[0]['dtype'] == np.float32:
                input_data = (np.float32(input_data) - 127.5) / 127.5
            interpreter.set_tensor(input_details[0]['index'], input_data)
            interpreter.invoke()
            output = interpreter.get_tensor(output_details[0]['index'])[0]
            detections = output.T
            highest_confidence = 0
            best_detection = None
            for detection in detections:
                confidence = detection[4]
                if confidence > highest_confidence:
                    highest_confidence = confidence
                    best_detection = detection
            if best_detection is not None and highest_confidence > CONFIDENCE_THRESHOLD:
                x_center, y_center = best_detection[0], best_detection[1]
                error_pan = x_center - 0.5
                error_tilt = y_center - 0.5
                with motion_thread_lock:
                    pan_adjustment = int(error_pan * (PULSE_MAX - PULSE_MIN) * TRACKING_P_GAIN)
                    target_pan_pulse = int(max(PULSE_MIN, min(PULSE_MAX, target_pan_pulse - pan_adjustment)))
                    tilt_adjustment = int(error_tilt * (PULSE_MAX - PULSE_MIN) * TRACKING_P_GAIN)
                    target_tilt_pulse = int(max(PULSE_MIN, min(PULSE_MAX, target_tilt_pulse + tilt_adjustment)))
        else:
            time.sleep(0.5)

# --- Flask App & Helper Functions ---
app = Flask(__name__)

def set_servo_pulse(channel, pulse):
    led_on_reg = 0x06 + (4 * channel)
    led_off_reg = 0x08 + (4 * channel)
    bus.write_word_data(BOARD_ADDRESS, led_on_reg, 0)
    bus.write_word_data(BOARD_ADDRESS, led_off_reg, pulse)

def joystick_to_pulse(joystick_value):
    travel_range = (PULSE_MAX - PULSE_MIN) / 2
    pulse = PULSE_CENTER + (joystick_value * travel_range)
    return int(max(PULSE_MIN, min(PULSE_MAX, pulse)))

def check_token():
    auth_header = request.headers.get('Authorization')
    token = auth_header.split(" ")[1] if auth_header and auth_header.startswith('Bearer ') else None
    return token is not None and token == current_token

def generate_video_frames():
    while True:
        with frame_lock:
            if shared_frame is None:
                time.sleep(0.05)
                continue
            frame = shared_frame.copy()
        (flag, encodedImage) = cv2.imencode(".jpg", frame)
        if not flag:
            continue
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + bytearray(encodedImage) + b'\r\n')

# --- Flask Routes ---
@app.route('/')
def index():
    return send_from_directory('.', 'index.html')

@app.route('/video_feed')
def video_feed():
    return Response(generate_video_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route('/api/connect', methods=['POST'])
def api_connect():
    global current_token
    if current_token is None:
        current_token = secrets.token_hex(16)
        return jsonify({"status": "connected", "token": current_token}), 200
    else:
        return jsonify({"status": "error", "message": "Turret is already in use."}), 409

@app.route('/api/set_speed', methods=['POST'])
def set_speed():
    global current_speed
    if not check_token(): return "Unauthorized", 401
    speed = request.get_json().get('speed')
    if speed in speed_settings:
        with motion_thread_lock:
            current_speed = speed
        return jsonify({"status": "ok"}), 200
    return jsonify({"status": "error", "message": "Invalid speed"}), 400

@app.route('/control', methods=['POST'])
def control_turret():
    global target_pan_pulse, target_tilt_pulse
    if not check_token(): return "Unauthorized", 401
    data = request.get_json()
    with motion_thread_lock:
        if current_mode == 'MANUAL':
            target_pan_pulse = joystick_to_pulse(data.get('x', 0.0) * -1)
            target_tilt_pulse = joystick_to_pulse(data.get('y', 0.0))
    return "OK", 200

@app.route('/center', methods=['POST'])
def center_turret():
    global target_pan_pulse, target_tilt_pulse
    if not check_token(): return "Unauthorized", 401
    with motion_thread_lock:
        target_pan_pulse = PULSE_CENTER
        target_tilt_pulse = PULSE_CENTER
    return "OK", 200

@app.route('/api/fire', methods=['POST'])
def fire_command():
    if not check_token(): return "Unauthorized", 401
    print("PEW PEW! Firing command received.")
    return "OK", 200

@app.route('/setup')
def setup_page():
    return send_from_directory('.', 'wifi_setup.html')

@app.route('/api/scan-wifi')
def scan_wifi():
    try:
        ssids = set()
        for dev in NetworkManager.NetworkManager.GetDevices():
            if dev.DeviceType == NetworkManager.NM_DEVICE_TYPE_WIFI:
                dev.RequestScan({})
                for ap in dev.GetAccessPoints():
                    ssid = ap.Ssid.decode('utf-8')
                    if ssid:
                       ssids.add(ssid)
        return jsonify(sorted(list(ssids)))
    except Exception as e:
        print(f"Error getting networks via NetworkManager: {e}")
        return jsonify([]), 500

@app.route('/save-credentials', methods=['POST'])
def save_credentials():
    try:
        ssid = request.form['ssid']
        password = request.form['password']
        if not ssid or not password:
            return "SSID and password are required.", 400
        subprocess.run(['sudo', 'nmcli', 'dev', 'wifi', 'connect', ssid, 'password', password], check=True)
        threading.Timer(5.0, lambda: subprocess.run(['sudo', 'reboot'])).start()
        return "Successfully connected! The device will now reboot."
    except Exception as e:
        return f"Failed to connect: {e}", 500

@app.route('/api/set_mode', methods=['POST'])
def set_mode():
    global current_mode, target_pan_pulse, target_tilt_pulse
    if not check_token(): return "Unauthorized", 401
    mode = request.get_json().get('mode')
    with motion_thread_lock:
        current_mode = mode
        if current_mode == 'MANUAL':
            target_pan_pulse = PULSE_CENTER
            target_tilt_pulse = PULSE_CENTER
    print(f"Mode changed to: {current_mode}")
    return jsonify({"status": "ok"}), 200

# --- Main Execution Block ---
if __name__ == '__main__':
    camera_thread = threading.Thread(target=camera_capture_thread)
    camera_thread.daemon = True
    camera_thread.start()
    
    motion_thread = threading.Thread(target=motion_control_loop)
    motion_thread.daemon = True
    motion_thread.start()
    
    ai_thread = threading.Thread(target=ai_processing_thread)
    ai_thread.daemon = True
    ai_thread.start()

    try:
        time.sleep(3)
        servo_power.on()
        app.run(host='0.0.0.0', port=5000, threaded=True)
    finally:
        servo_power.off()
        set_servo_pulse(PAN_CHANNEL, 0)
        set_servo_pulse(TILT_CHANNEL, 0)

wifi_setup.html

HTML
This is the one time initial setup to get onto your wifi
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Sentry Turret WiFi Setup</title>
    <style>
        body { font-family: sans-serif; background-color: #1a1a1a; color: #e0e0e0; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; }
        .container { background-color: #2a2a2a; padding: 2em; border-radius: 8px; box-shadow: 0 4px 8px rgba(0,0,0,0.5); width: 90%; max-width: 400px; }
        h1 { text-align: center; color: #4CAF50; }
        label { display: block; margin-top: 1em; }
        input, select, button { width: 100%; padding: 0.8em; margin-top: 0.5em; border-radius: 4px; border: 1px solid #555; background-color: #333; color: #e0e0e0; box-sizing: border-box; }
        button { background-color: #4CAF50; cursor: pointer; font-weight: bold; }
        button:hover { background-color: #45a049; }
        #message { margin-top: 1em; text-align: center; }
    </style>
</head>
<body>
    <div class="container">
        <h1>WiFi Setup</h1>
        <p>Connect your Sentry Turret to a new WiFi network.</p>
        <button id="scan-btn" type="button">Scan for Networks</button>
        <form id="wifi-form" action="/save-credentials" method="POST">
            <label for="ssid-select">Select a Network:</label>
            <select id="ssid-select" name="ssid">
                <option value="">-- Please Scan --</option>
            </select>
            <label for="password">Password:</label>
            <input type="password" id="password" name="password" required>
            <button type="submit">Connect & Reboot</button>
        </form>
        <div id="message"></div>
    </div>
<script>
        const scanBtn = document.getElementById('scan-btn');
        const ssidSelect = document.getElementById('ssid-select');
        const wifiForm = document.getElementById('wifi-form');
        const messageDiv = document.getElementById('message');

        scanBtn.addEventListener('click', async () => {
            scanBtn.textContent = 'Scanning...';
            scanBtn.disabled = true;
            try {
                const response = await fetch('/api/scan-wifi');
                const networks = await response.json();
                
                ssidSelect.innerHTML = '<option value="">-- Select a Network --</option>'; // Clear previous results
                networks.forEach(net => {
                    const option = document.createElement('option');
                    option.value = net;
                    option.textContent = net;
                    ssidSelect.appendChild(option);
                });

            } catch (error) {
                console.error('Error scanning for networks:', error);
                messageDiv.textContent = 'Failed to scan for networks.';
            } finally {
                scanBtn.textContent = 'Rescan for Networks';
                scanBtn.disabled = false;
            }
        });

        wifiForm.addEventListener('submit', () => {
            messageDiv.textContent = 'Connecting... The turret will reboot if successful. This page will stop working.';
        });
    </script>
</body>
</html>

index.html

HTML
This is the HUD interface for PC, phone, tablet, etc
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">
    <title>Aegis Control</title>
<style>
        html, body {
            height: 100%;
            margin: 0;
            overflow: hidden;
            background-color: #000;
            font-family: sans-serif;
            touch-action: none;
        }
        #video-container {
            position: absolute;
            top: 0;
            left: 0;
            width: 100%;
            height: 100%;
        }
        #video-stream {
            width: 100%;
            height: 100%;
            object-fit: cover; /* This is the key change to fill the screen */
        }
        #status-overlay {
            position: absolute;
            top: 10px;
            left: 10px;
            padding: 5px 10px;
            border-radius: 5px;
            font-weight: bold;
            opacity: 0.8;
            z-index: 10;
        }
        .status-disconnected { background-color: #d32f2f; color: white; }
        .status-connected { background-color: #388e3c; color: white; }
        
        /* Controls are now positioned individually */
        #joystick-container {
            position: absolute;
            left: 20px;
            bottom: 20px;
            width: 150px;
            height: 150px;
            opacity: 0.7;
            z-index: 10;
        }
        #joystickCanvas {
            width: 100%;
            height: 100%;
            border-radius: 50%;
            background: radial-gradient(circle, #555 0%, #222 100%);
        }
	#fireButton {
	    position: absolute;
	    right: 20px;
	    bottom: 20px;
	    width: 100px;
	    height: 100px;
	    
	    /* Apply the semi-transparent background directly to the button */
	    background-color: rgba(51, 51, 51, 0.7); /* This is the same as your original button background */
	    border: 2px solid #555;                 /* Re-add the border if you want it to match the joystick more */
	    border-radius: 15px;                    /* Apply the rounded corners */
	    
	    display: flex;                          /* Use flexbox to center the image */
	    justify-content: center;                /* Center horizontally */
	    align-items: center;                    /* Center vertically */
	    padding: 0;
	    cursor: pointer;
	    z-index: 10;
	}

	#fireButton img {
	    /* The image itself should be fully opaque, its parent (the button) controls overall opacity */
	    width: 80%; /* Adjust size as needed, e.g., 80% to give some padding */
	    height: 80%; /* Adjust size as needed */
	    object-fit: contain; /* Ensures the image scales nicely within the button */
	    opacity: 1; /* Ensure the image itself is not transparent */
	}
        #centerButton:disabled {
            background-color: rgba(34, 34, 34, 0.5);
            color: #555;
            border-color: #444;
        }
	#speedButton {
            position: absolute;
            top: 10px;
            right: 10px;
            padding: 5px 10px;
            font-weight: bold;
            background-color: rgba(51, 51, 51, 0.7);
            border: 2px solid #555;
            color: #e0e0e0;
            border-radius: 5px;
            z-index: 10;
        }
	#crosshair {
            position: absolute;
            top: 50%;
            left: 50%;
            width: 200px;  /* Increased size */
            height: 200px; /* Increased size */
            transform: translate(-50%, -50%);
            background-image: url('/static/images/crosshair2.png'); /* Ensure this filename is correct */
            background-size: contain;
            background-repeat: no-repeat;
            pointer-events: none;
            z-index: 5; /* Ensure it's above the video but below UI elements */
        }
	#modeButton {
	    position: absolute;
	    top: 50px; /* Position it below the speed button */
	    right: 10px;
	    padding: 5px 10px;
	    font-weight: bold;
	    background-color: rgba(51, 51, 51, 0.7);
	    border: 2px solid #555;
	    color: #e0e0e0;
	    border-radius: 5px;
	    z-index: 10;
	}
    </style>
</head>
<body>

    <div id="video-container">
        <img id="video-stream" alt="Video stream failed to load. Is the turret on?"/>
        <div id="status-overlay" class="status-disconnected">DISCONNECTED</div>
	<div id="crosshair"></div>
    </div>

    <div id="controls-container">
        <div id="joystick-container">
            <canvas id="joystickCanvas"></canvas>
        </div>
        <button id="speedButton">SPEED: >></button> 
        <button id="modeButton">MODE: MANUAL</button>       
        <button id="fireButton">
    		<img src="/static/images/ammunition.png" alt="Fire Button">
	</button>
 </div>
    </div>
<script>
        // --- CONFIGURATION ---
        const PI_ADDRESS = ""; 
        let authToken = null;

        // --- ELEMENT SETUP ---
        const videoStream = document.getElementById('video-stream');
	const modeButton = document.getElementById('modeButton');
        const canvas = document.getElementById('joystickCanvas');
        const fireButton = document.getElementById('fireButton'); // <-- CORRECTED ID
        const speedButton = document.getElementById('speedButton');
        const statusOverlay = document.getElementById('status-overlay');
        const ctx = canvas.getContext('2d');
        
        let centerX, centerY, radius, knobRadius;
        let isDragging = false;

        // --- SPEED CONTROL LOGIC ---
        const speeds = ['MEDIUM', 'FAST', 'SLOW'];
        const speedSymbols = ['>>', '>>>', '>'];
        let currentSpeedIndex = 0;

        // --- CORE FUNCTIONS ---
        async function connectToServer() {
            try {
                const response = await fetch(`${PI_ADDRESS}/api/connect`, { method: 'POST' });
                const data = await response.json();
                if (data.token) {
                    authToken = data.token;
                    statusOverlay.textContent = 'CONNECTED';
                    statusOverlay.className = 'status-connected';
                    fireButton.disabled = false; // <-- CORRECTED VARIABLE
                    console.log("Connected! Token:", authToken);
                    setSpeed(speeds[currentSpeedIndex]); // Set default speed on connect
                } else {
                    statusOverlay.textContent = 'IN USE';
                    console.error("Failed to connect:", data.message);
                }
            } catch (error) {
                statusOverlay.textContent = 'OFFLINE';
                console.error("Connection failed:", error);
            }
        }
        
        async function sendCommand(endpoint, body = {}) {
            if (!authToken) return;
            try {
                await fetch(`${PI_ADDRESS}${endpoint}`, {
                    method: 'POST',
                    headers: { 
                        'Content-Type': 'application/json',
                        'Authorization': `Bearer ${authToken}`
                    },
                    body: JSON.stringify(body)
                });
            } catch (error) {
                console.error(`Failed to send command to ${endpoint}:`, error);
            }
        }

        async function setSpeed(speed) {
            await sendCommand('/api/set_speed', { speed: speed });
        }

        speedButton.addEventListener('click', () => {
            currentSpeedIndex = (currentSpeedIndex + 1) % speeds.length;
            const newSpeed = speeds[currentSpeedIndex];
            const newSymbol = speedSymbols[currentSpeedIndex];
            speedButton.textContent = `SPEED: ${newSymbol}`;
            setSpeed(newSpeed);
        });

        // --- JOYSTICK FUNCTIONS ---
        function setCanvasSize() {
            const size = 150;
            canvas.width = size;
            canvas.height = size;
            centerX = canvas.width / 2;
            centerY = canvas.height / 2;
            radius = size / 2;
            knobRadius = radius * 0.3;
            drawJoystick(centerX, centerY);
        }

        function drawJoystick(x, y) {
            ctx.clearRect(0, 0, canvas.width, canvas.height);
            ctx.beginPath();
            ctx.arc(x, y, knobRadius, 0, Math.PI * 2, true);
            ctx.fillStyle = fireButton.disabled ? '#444' : '#666'; // <-- CORRECTED VARIABLE
            ctx.fill();
            ctx.strokeStyle = fireButton.disabled ? '#555' : '#888'; // <-- CORRECTED VARIABLE
            ctx.lineWidth = 4;
            ctx.stroke();
        }

        // --- EVENT LISTENERS ---
        function handleStart(e) {
            if (fireButton.disabled) return; // <-- CORRECTED VARIABLE
            e.preventDefault();
            isDragging = true;
        }

        function handleEnd(e) {
            if (!isDragging) return;
            e.preventDefault();
            isDragging = false;
            drawJoystick(centerX, centerY);
            sendCommand('/center');
        }

        function handleMove(e) {
            if (!isDragging) return;
            e.preventDefault();
            const rect = canvas.getBoundingClientRect();
            const touch = e.touches ? e.touches[0] : e;
            
            let x = touch.clientX - rect.left;
            let y = touch.clientY - rect.top;

            const distance = Math.sqrt(Math.pow(x - centerX, 2) + Math.pow(y - centerY, 2));
            if (distance > radius - knobRadius) {
                const angle = Math.atan2(y - centerY, x - centerX);
                x = centerX + (radius - knobRadius) * Math.cos(angle);
                y = centerY + (radius - knobRadius) * Math.sin(angle);
            }
            drawJoystick(x, y);
            const normalizedX = (x - centerX) / (radius - knobRadius);
            const normalizedY = ((y - centerY) / (radius - knobRadius)) * -1;
            sendCommand('/control', { x: normalizedX, y: normalizedY });
        }

        // --- INITIALIZATION ---
        window.addEventListener('resize', setCanvasSize);
        canvas.addEventListener('mousedown', handleStart);
        document.addEventListener('mouseup', handleEnd);
        document.addEventListener('mousemove', handleMove);
        canvas.addEventListener('touchstart', handleStart, { passive: false });
        document.addEventListener('touchend', handleEnd);
        document.addEventListener('touchmove', handleMove, { passive: false });
        fireButton.addEventListener('click', () => sendCommand('/api/fire')); // <-- CORRECTED VARIABLE & ENDPOINT

        videoStream.src = `${PI_ADDRESS}/video_feed`;
        setCanvasSize();
        connectToServer();

	let currentMode = 'MANUAL';

	async function setMode(mode) {
	    await sendCommand('/api/set_mode', { mode: mode });
	}

	modeButton.addEventListener('click', () => {
	    if (currentMode === 'MANUAL') {
	        currentMode = 'AUTO';
	        modeButton.textContent = 'MODE: AUTO SCAN';
	    } else {
	        currentMode = 'MANUAL';
	        modeButton.textContent = 'MODE: MANUAL';
	    }
	    setMode(currentMode);
	});

    </script>
</body>
</html>

requirements.txt

Python
You can save this list as a file named requirements.txt and then run pip install -r requirements.txt to install them all.
# Web Framework
Flask

# I2C Communication
smbus2

# Raspberry Pi Camera
picamera2

# GPIO Control
gpiozero

# Numerical & Image Processing
numpy
opencv-python

# Networking (for the experimental web setup)
python-networkmanager

turret_service.txt

Python
Be sure to change all <<< >>> noted variables with your setup
[Unit]
Description=Aegis Control Turret Service
Wants=network-online.target
After=network-online.target

[Service]
User=<<<username>>>
WorkingDirectory=/home/<<<username>>>/sentry_turret
ExecStart=/usr/bin/python3 /home/<<<username>>>/sentry_turret/turret_controller.py
Restart=always
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Essential Debugging Commands

BatchFile
If you run into issues with performance or hardware detection, these commands are your best friends. Run them in the SSH terminal:

Check for Throttling (Crucial!): This command tells you if your Pi is reducing its speed due to low voltage or overheating.
Bash
vcgencmd get_throttled

If the output is anything other than throttled=0x0, you need to check the power supply or cooling.

Monitor Temperature: See how hot the CPU is running, especially if overclocking.
Bash
vcgencmd measure_temp

Verify I2C Connection: This command scans the I2C bus and checks if the PCA9685 servo driver is connected and detected at address 0x40.
Bash
i2cdetect -y 0

If you don't see "40" in the grid, there is a wiring problem.

Monitor Live Performance: This tool shows you a live view of your CPU usage, letting you see how hard the Python script and its threads are working.
Bash
htop

ESP32 listener code

C/C++
This code listens for a command from the Raspberry Pi on Serial2 and then would trigger the MSP command to the flight controller.
// --- ESP32 DroneBridge Command Listener ---

#define PI_SERIAL Serial2 // Use UART2 for communication with the Pi
#define PI_BAUD_RATE 115200

String incomingCommand = ""; // A string to hold incoming data

void setup() {
  // Start the main serial port for debugging to the computer
  Serial.begin(115200);
  Serial.println("ESP32 Command Listener Booting...");

  // Start the second serial port to listen for commands from the Raspberry Pi
  PI_SERIAL.begin(PI_BAUD_RATE);
  
  // Initialize communication with the flight controller (DroneBridge/MSP) here
  // ...
}

void loop() {
  // Check if there's data available from the Raspberry Pi
  if (PI_SERIAL.available() > 0) {
    char receivedChar = PI_SERIAL.read();

    // If we receive a newline character, the command is complete
    if (receivedChar == '\n') {
      Serial.print("Received command from Pi: ");
      Serial.println(incomingCommand);

      // Check if the received command is "RTH"
      if (incomingCommand == "RTH") {
        Serial.println("RTH Command confirmed! Triggering drone's Return to Home.");
        
        //
        // --- YOUR MSP LOGIC HERE ---
        // This is where you would construct and send the appropriate
        // MSP command to the flight controller to activate RTH mode.
        //
      }

      // Clear the string for the next command
      incomingCommand = "";
    } else {
      // Add the character to our command string
      incomingCommand += receivedChar;
    }
  }
  
  // Other DroneBridge code would run here
  // ...
}

Raspberry Pi to ESP32 Communication Instructions

Python
Adding Pi-to-Drone Communication
Reference this section when you are ready to implement the communication link between the Sentry Turret and the Endurance Drone. This guide provides the necessary hardware setup and all the required Python code additions.

1. Enable the Hardware Serial Port on the Raspberry Pi
First, you must enable the Pi's hardware UART, which is often disabled by default.

Open a terminal and run sudo raspi-config.

Navigate to 3 Interface Options.

Navigate to I6 Serial Port.

When asked, "Would you like a login shell to be accessible over serial?" select NO.

When asked, "Would you like the serial port hardware to be enabled?" select YES.

Finish and reboot the Pi when prompted.

Wiring: Connect the Raspberry Pi's TXD (GPIO 14) pin to the ESP32's RX pin, and the Pi's RXD (GPIO 15) pin to the ESP32's TX pin. Crucially, also connect a ground (GND) pin between the Pi and the ESP32.

Modify the code in turret_controller.py to the code below:
import serial

# --- Drone Communication Configuration ---
DRONE_SERIAL_PORT = '/dev/ttyS0'  # Hardware UART on Raspberry Pi
DRONE_BAUD_RATE = 115200
RTH_COMMAND = b'RTH\n' # The command must be a byte string, ending with a newline

# --- Drone Communication Initialization ---
drone_serial = None
try:
    # Initialize the serial connection
    drone_serial = serial.Serial(DRONE_SERIAL_PORT, DRONE_BAUD_RATE, timeout=1)
    print(f"Successfully opened serial port {DRONE_SERIAL_PORT} for drone communication.")
except serial.SerialException as e:
    print(f"ERROR: Could not open serial port {DRONE_SERIAL_PORT}: {e}")
    print("Drone communication will be disabled for this session.")
    
    
    
# --- Drone Communication Function ---
def send_rth_command_to_drone():
    """Sends the pre-defined RTH command to the ESP32 over serial."""
    if drone_serial and drone_serial.is_open:
        try:
            print(f"Sending RTH command to drone via serial...")
            drone_serial.write(RTH_COMMAND)
            return True
        except Exception as e:
            print(f"ERROR: Failed to send command to drone: {e}")
            return False
    else:
        print("WARNING: Drone serial port not available. Cannot send RTH command.")
        return False
        

# ---    Find your existing fire_command() function and replace it entirely with this modified version ---

@app.route('/api/fire', methods=['POST'])
def fire_command():
    if not check_token(): return "Unauthorized", 401
    print("PEW PEW! Firing command received.")

    # In a full V2 implementation, you would first check if a target is locked on.
    # For now, we'll just call the function directly when the fire button is pressed.
    success = send_rth_command_to_drone()
    
    if success:
        return "OK - RTH command sent.", 200
    else:
        # Still return a success status to the web UI, but note the failure in the log.
        return "OK - Fired, but failed to send RTH command.", 200

Credits

Isaac Nason
1 project • 3 followers
Applied Robotics designer with 15 yrs exp. I bring intelligent systems to life, blending AI, custom electronics, and advanced 3D fabrication

Comments

Related channels and tags
  • 3D Printing
  • 3D Printing
  • AI & Machine Learning
  • AI & Machine Learning
  • Drones
  • Drones
  • Espressif
  • Python on Hardware
  • Python on Hardware
  • Raspberry Pi
  • Robotics
  • Robotics