Scaring People with AI: Raspberry Pi, Ai Camera, SONY IMX500

In a quiet suburban neighborhood, October had arrived, and the air buzzed with Halloween excitement. At the same time, a new Raspberry Pi AI camera came into my hands, and that means it's time to scare someone. People often fear AI because they believe it will take away jobs, but this time, it will help me create a new kind of fear.

Traditionally, the whole setup in the video:

The Problem

I bought a reaper that makes some scary noises.

But it activates with a button. A button, Carl!

This is the most useless feature I’ve ever seen. Some reapers use sound activation, but they’re just as ineffective because no one’s going to clap or press a button to get scared.

Did the engineers really think someone would clap or press the button, hoping to be suddenly scared?

Of course, I could have bought a motion sensor and used it, but this is the era of AI, not motion sensors.

The Solution

The concept is simple: a reaper that only comes to life when someone walks by, thanks to object detection powered by the new AI camera. Here's how the system works:

• An ESP32 and a relay are connected to the reaper, and the ESP32 is linked to a Wi-Fi network.
• The reaper is placed near the sidewalk.
• The AI camera and Raspberry Pi are hidden nearby, facing the sidewalk, and connected to the same Wi-Fi network.
• The AI detection runs continuously, and when the Raspberry Pi detects people, it sends a UDP broadcast message, 'BOO!'
• The ESP32 receives the message and activates the reaper, scaring passersby.

Raspberry PI & AI Camera (Sony IMX500) setup

I used a Raspberry Pi with a standard Debian Bookworm system and some predefined Wi-Fi settings in Pi Imager.

To keep the AI module discreet, I designed an analog of a wildlife camera holder.

Then I put a Raspberry PI and an AI Camera inside and closed it with a cover.

The next step was to install the software. I have a repository that has all the required scripts:

https://github.com/Nerdy-Things/raspberry-pi-sony-imx500-halloween-project

For the Raspberry Pi, I needed to install the following dependencies. It is located in the /raspberry/install.sh

#!/bin/bash

# https://www.raspberrypi.com/documentation/accessories/ai-camera.html

# Required - installing dependencies
sudo apt update && sudo apt full-upgrade

sudo apt install -y imx500-all

sudo apt install -y python3-opencv python3-munkres

#Optional
pip install model_compression_toolkit --break-system-packages
pip install imx500-converter[pt] --break-system-packages

# Download models from https://github.com/raspberrypi/imx500-models
git submodule init
git submodule update

sudo reboot now

The script will work only if it runs inside the cloned repository. Otherwise, all models can be found here.

AI Camera Sony IMX500 Object detection

After installation, I ran the recognition.py script, which is similar to the AI Camera example. But with the critical changes at the bottom:

while True:
        last_results = parse_detections(picam2.capture_metadata())
        # Record file to SD card
        data_folder = f"../data/images/{DateUtils.get_date()}/"
        try:
            picam2.capture_file(f"{data_folder}/{DateUtils.get_time()}.jpg")
        except:
            FileUtils.create_folders(data_folder)

        if (len(last_results) > 0):
            for result in last_results:
                if result.category == 0: 
                    print("Person detected, sending BOO!")
                    send_udp_message(MESSAGE, PORT)

The script uses an imx500_network_ssd_mobilenetv2_fpnlite_320x320_pp model and gets detection results from the AI Camera, saves an image to an SD card, and if a person is detected (result.category == 0), it sends a UDP broadcast message:

interfaces = socket.getaddrinfo(host=socket.gethostname(), port=None, family=socket.AF_INET)
    allips = [ip[-1][0] for ip in interfaces]
    
    # Create the socket once
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
    
    for ip in allips:
        try:
            print(f'sending on {ip}')
            sock.sendto(message, ("255.255.255.255", port))
        except Exception as e:
            print(f"Error sending message on {ip}: {e}")
    sock.close()

The remaining part was to place the hidden camera somewhere.

And run the script.

ESP32 & a Relay

I want to keep my relay wireless, so I came up with the following schema:

Three AA batteries are connected to a voltage converter that stabilizes the output to 5V.

The converter is connected to the ESP32's 5V and ground pins, and the same wires are connected to the relay’s DC+ and DC-.

The rest was just replace the button with a relay and hang the reaper:

Troubleshooting

There was a problem: the reaper needed to scare people at night, and as we all know, cameras typically struggle to capture good images in low-light conditions. In other words, the AI camera needed adjustments.

The solution was to tweak the camera settings by extending the exposure time and increasing the brightness, gain, and contrast.

# Night settings
controls = {
"Brightness": 0.25,
"Contrast": 1.0,   
"ExposureTime": 150000,
"AnalogueGain": 32.0,
}

config = picam2.create_preview_configuration(
controls = controls,
buffer_count=12
)

The Results

Day mode:

Night mode:

In action:

Happy Halloween!

Cheers!