Person Detection with Raspberry Pi AI Camera & OpenPLC

This project demonstrates how to implement privacy-focused person detection using the Raspberry Pi AI Camera integrated with OpenPLC for real-time visual notifications via a stack light or signal tower light.

Person detection AI has numerous applications including crowd counting, intrusion detection, and congestion monitoring. Traditional approaches required transmitting captured images to external PCs or cloud services for AI inference, raising significant privacy concerns as images could be stored or accessed by third parties.

The Raspberry Pi AI Camera solves this privacy challenge by processing AI inference locally on-device. Only detection metadata (status and position coordinates) is transmitted, and no actual images leave the camera. This architecture makes it ideal for privacy-sensitive environments.

System Overview: This project creates an automated notification system where:

• The Raspberry Pi AI Camera performs real-time person detection
• Detection status is communicated to a PLC (Programmable Logic Controller)
• A stack light provides immediate visual notification when persons are detected

PLCs are industrial-grade computers designed for sequence control in manufacturing environments. While various notification methods exist, this project prioritizes visual clarity by activating a stack light, making detection status instantly recognizable from a distance, as demonstrated in the video below.

What You Can Do with This Project

This project enables you to:

• Deploy a person detection system using the Raspberry Pi AI Camera
• Complete source code is available in the project repository
• Integrate AI inference with industrial control systems by connecting the Raspberry Pi AI Camera to OpenPLC for automated device control
• A stack light is used as the notification device in this implementation
• The system can be tested and verified without physical hardware. Detection status can be monitored through the application interface alone

Required Equipment

The list of required equipment is as follows.

1. Raspberry Pi AI Camera (required): Purchase information is available from Raspberry Pi)

2. Raspberry Pi × 2 (required): In this project, one Raspberry Pi 4 and one Raspberry Pi 5 are used. Device prior to Raspberry Pi 4 is required for OpenPLC use. Check the list of supported models for Raspberry Pi AI Camera)

3. Type-C cable (required): Power connection

4. Micro SD card (required): 32GB or more recommended

5. Mouse: Only if you need it for development

6. Monitor: Only if you need it for development

7. Keyboard: Only if you need it for development

8. HDMI cable: Only if you need it for development

9. Stack light: Used in this project, but can be changed according to the application's requirements. The device used in this project can be found here.

10. Switching power supply: For power supply to the stack light. The device used in this project can be found here.

11. Relay module: For connecting the stack light to the Raspberry Pi. The device used in this project can be found here.

12. GPIO Pin: For connecting the stack light to the Raspberry Pi. The device used in this project can be found here.

13. Heat sink & case: Only if needed (it is recommended to use a heat sink). The device used in this project can be found here.

14. PC: For OpenPLC Editor installation. Compatible with Windows / MacOS / Linux. Installation is here.

Setup

Writing the Raspberry Pi Image

Write the operating system to your SD card using Raspberry Pi Imager (download here). The tool is available on Windows, macOS, and Ubuntu.

For step-by-step instructions, refer to the official installation guide.

AI Camera Setup

The Raspberry Pi AI Camera package includes the following components:

• AI Camera
• 15-pin MIPI CSI cable (Standard-Standard 20cm)
• Use this cable when connecting to the Raspberry Pi 4 series.
• 22-pin MIPI CSI cable (Standard-Mini 20cm)
• Use this cable when connecting to the Raspberry Pi 5 or Pi Zero series.
• Small focus adjustment tool
• The tool can be inserted and turned to adjust the focus.

Connecting the Camera:

Attach the Raspberry Pi AI Camera to your Raspberry Pi as shown below:

NOTE:FPC cables have specific orientations. Ensure correct alignment during connection. The contacts should face the proper direction.

Configuration:

After physical connection, complete the setup process following the official AI Camera documentation.

Verification Test:

Confirm successful installation by running the object detection sample:

rpicam-hello -t 0s --post-process-file /usr/share/rpi-camera-assets/imx500_mobilenet_ssd.json --viewfinder-width 1920 --viewfinder-height 1080 --framerate 30

Expected Result:

The camera should display real-time object detection. In the example below, an apple is correctly identified:

Implementation Details

In this project, we will perform person detection using an AI model for object detection. The detection results will be monitored by OpenPLC, and the light status of the stack light will be controlled according to the results. Specifically, when a person is detected, red LEDs will be lit, and when no person is detected, green LEDs will be lit.

The system configuration is as follows. Please connect the two Raspberry Pi devices within the same LAN.

OpenPLC

OpenPLC is open-source PLC software. You can check the official site here. It complies with the international standard IEC61131-3 for PLCs and supports five types of PLC programming languages. Real PLC devices have high introductory costs and are difficult to procure, thus OpenPLC serves as an alternative solution for learning and demo purposes. In this project, we will create a PLC control program using ladder language, one of the five supported PLC programming languages.

Implementation

AI Camera Capture Script

For person detection, we will use nanodet_plus_416×416.rpk, one of the sample models for object recognition installed during the AI Camera setup. The list of models can be checked in the /usr/share/imx500-models directory.

This model is trained on the COCO dataset, which includes the label person. Using this label, we will implement person detection.

We will refer to the sample script for object recognition published by the Raspberry Pi Foundation for the person detection code. First, clone the script with the following command.

git clone https://github.com/raspberrypi/picamera2.git

You can verify its operation using the following command.

python picamera2/examples/imx500_object_detection_demo.py --model=/usr/share/imx500-models/imx500_network_nanodet_plus_416x416.rpk

The execution result is as follows. It can be confirmed that the detection of the person is performed correctly. (Mosaic processing is implemented separately)

Additionally, detection of hands only is also possible. (Mosaic processing is implemented separately)

Analyzing Detection Results

Before implementation, let's confirm the output of object detection from the Raspberry Pi AI Camera. The output from the camera is received in the np_outputs of the parse_detections function, and the results are stored in last_detections.

last_detections stores data for each detection sample as an object of the Detection class. The data members of the Detection class are as follows.

• category: Category of the detection sample
• conf: Confidence score of the detection result
• box: Position information of the detection result (x, y, w, h)

Thus, person detection information can be determined by the presence of category = person.

Pymodbus Implementation

We will use the Modbus TCP protocol for communication between the Raspberry Pi with the AI Camera and the Raspberry Pi running OpenPLC. This is a standard communication protocol for industrial equipment and is adopted by OpenPLC as well.

The Pymodbus library will be used for implementation in Python.

sudo pip install pymodbus # This time we are using pymodbus==2.5.3

Note

Port 502 used for Modbus communication is a privileged port, so execution with administrative rights is required.

Here is the basic setup for the necessary Python script to use the Modbus Server.

# pymodbus module
from pymodbus.server.asynchronous import StartTcpServer
from pymodbus.datastore import ModbusSequentialDataBlock, ModbusSlaveContext, ModbusServerContext
from pymodbus.device import ModbusDeviceIdentification

# Setting up the Modbus Server
store = ModbusSlaveContext(
        di=ModbusSequentialDataBlock(0, [0, 0, 0]) # Initialize di
    )
context = ModbusServerContext(slaves={0x01: store}, single=False) # 0x0n: slave device number n

def modbus_server(context):
    identity = ModbusDeviceIdentification()
    identity.VendorName = 'RaspberryPi'
    identity.ProductCode = 'RPi'
    identity.VendorUrl = 'http://www.raspberrypi.org/'
    identity.ProductName = 'RPi Modbus Server'
    identity.ModelName = 'RPi Model'
    identity.MajorMinorRevision = '1.0'

    StartTcpServer(context, identity, ("", 502))

Next, we'll implement the part where the values of the Modbus Server are changed based on the output from the AI Camera. The check_including_person function will call update_di to dynamically change the values of the contacts depending on whether a person is detected. The implementation is as follows.

# Setting the Detection Area
is_detection_area = False # ON = True / OFF = False

class Detection_Area: 
    def __init__(self, x, y, w, h):
        self.x = x # 0 <= x <  640
        self.y = y # 0 <= y <  480
        self.w = w # 1 <  w <= 640 - x
        self.h = h # 1 <  h <= 480 - y

if is_detection_area:
    detection_area = Detection_Area(0, 0, 320, 480)

# Check for person in detection area
def check_including_person(request):    
    global context
    detections = last_detections
    labels = get_labels()
    if detections == []:
        update_di(context, in_person=False)
        return 
    else:
        for detection in detections:
            if labels[int(detection.category)] == "person":
                if is_detection_area:
                    p_x, p_y, p_w, p_h = detection.box # detection area of person
                    if not((detection_area.x + detection_area.w <= p_x) or (p_x + p_w <= detection_area.x) or (detection_area.y + detection_area.h <= p_y) or (p_y + p_h <= detection_area.y)):
                        update_di(context, in_person=True)
                        return 
                else:
                    update_di(context, in_person=True)
                    return
        update_di(context, in_person=False)
        return 

# Update di based on detection results
def update_di(context, in_person):
    if in_person:
        values = [1, 0, 0]
        context[0x01].setValues(2, 0, values)
    else:
        values = [0, 1, 0]
        context[0x01].setValues(2, 0, values)

The object recognition model of the Raspberry Pi AI Camera performs AI inference over the entire screen by default. However, in practical applications, there may be cases where you only want to detect people in specific areas. Therefore, we have implemented it such that the stack light status switches only when a person is detected within the specified detection_area parameter.

Finally, we are ready to go by starting the Modbus Server as a thread. Please check the code for details. The entire code can be reviewed here.

OpenPLC Editor

Now let's move on to the OpenPLC implementation. This time, the ladder program used for PLC control implements only a simple function to control the stack light according to the person detection status, making it a very simple structure. The OpenPLC Editor was used to create the ladder program. Please refer to the official site for installation. We are using it installed on a Windows PC.

Note

A simpler structure like the one below can also implement the same functionality.

The created ladder program needs to be converted to.st (Structured Text) format to be used in OpenPLC Runtime. The conversion can be executed from the down arrow icon at the top of the Editor.

OpenPLC Runtime

Next, we will set up the OpenPLC Runtime. Here, we will select the executable program, configure the environment, and set up the Slave Device (as referred to in OpenPLC Runtime and is expected in the prerequisites).

After starting OpenPLC Runtime, accessing <Raspberry Pi's IP address>:8080 will display the login screen. The initial credentials are as follows.

• username: openplc
• password: openplc

Setup Procedure

1.Hardware Configuration

Select the Hardware tab and choose Raspberry Pi in the OpenPLC Hardware Layer. This will set the GPIO pin assignments for the Raspberry Pi.

2.Uploading the Program

Select the Programs tab to choose the program to run. Select the.st file created earlier in the OpenPLC Editor and give it an appropriate name before executing Upload Programs. After successful compilation, you should see Go to Dashboard, indicating that the program upload is complete.

3.Slave Device Configuration

Select the Slave Devices tab to configure the Device. In this project, Raspberry Pi 5 (the AI Camera side) will act as the Slave Device. The configuration items are as follows. For details, refer to the official site.

• Slave ID: A unique identifier for each Slave Device
• IP Address: The IP address of the Slave Device
• IP Port: The port number for Modbus communication (usually 502)
• Discrete Inputs: Contacts of the Slave Device

[!NOTE]

Please ensure that the IP Port matches the Modbus Server Port in the Settings tab.

GPIO Configuration

Since the GPIO of the Raspberry Pi provides low voltage output of 3.3V or 5V, additional power and relay circuits are necessary to control devices requiring higher voltage. The stack light to be used this time is driven by 15V-24V; thus, a 24V relay circuit will be constructed, using an AC-DC switching power supply (set to 24V). The wiring is as shown in the following diagram.

Execution Results

We will now execute the created code to verify whether the stack light control according to person detection status functions correctly.

Execution Procedure

1.Start OpenPLC Runtime

Select the Dashboard tab and press Start PLC to begin PLC control.

2.Start AI Camera

Obtain the code and execute it using the following command.

sudo python app.py

[!NOTE]

Since port 502 used for Modbus communication is a privileged port, please run it in administrator mode.

Verification Results

1.When detection_area is not set

It is confirmed that the stack light status is switching based on the detection of people in the image.

2.When detection_area is set

When executing with detection_area set to the left half of the image, it can be confirmed that when a person is not within the detection_area, the stack light remains green, and it only switches to red when a person enters the detection_area.

3.Operation Confirmation within OpenPLC Runtime

The operation can be confirmed not only from the actual stack light’s illumination status but also from the Monitoring tab in OpenPLC Runtime.

• When a person is detected

• When no person is detected

In this demonstration, video output is performed for operation confirmation; however, the Raspberry Pi AI Camera also allows acquiring only metadata. This enables the construction of a privacy-conscious person detection system without saving image data.

Conclusion

In this project, we constructed a system connecting the RaspberryPi AI Camera and OpenPLC to notify person detection information with a stack light. PLCs are widely used for automation in factory equipment control, and can be applied to various tasks not just limited to person detection tasks. For instance, a system can also be built to emit alerts when abnormal products are detected in a similar setup.

When in Trouble

If you encounter any issues while reading the article, please feel free to comment on this article. Also, please check the support site below. Please note that it may take some time to respond to comments.

• Raspberry Pi Forums

Code

Find all the code on our Github