Jim EwingLucas E
Published © MIT

Big Brother @ Home - Wireless AI Security System

An intelligent wireless security system that recognizes friends or foes using the latest AI image recognition algorithms.

AdvancedFull instructions provided15 hours7,552
Big Brother @ Home - Wireless AI Security System

Things used in this project

Hardware components

Raspberry Pi Zero Wireless
Raspberry Pi Zero Wireless
×1
Camera Module V2
Raspberry Pi Camera Module V2
×1
SORACOM Air Global IoT SIM
SORACOM Air Global IoT SIM
×1
Huawei MS2131i-8 3G IoT Global 3G USB Dongle
×1
5v DC stepper motor with ULN2003 controller (optional)
(optional)
×1
PL2303 USB UART Cable
Optional but recommended. Version as cheap a $1.00 can be found on eBay but ship from China.
×1

Software apps and online services

OpenCV
OpenCV
Raspbian
Raspberry Pi Raspbian
Linux v5.1 kernel
SMS Messaging API
Twilio SMS Messaging API

Hand tools and fabrication machines

3D Printer (generic)
3D Printer (generic)

Story

Read more

Custom parts and enclosures

3D Case

3D printed case for Pi Zero W, Pi camera, and Huawei MS2131i-8 modem

3D Stepper Motor Base

3D printed base for mounting a stepper motor and pivot arm

3D Stepper Motor Pivor Arm

Pivot arm for the stepper motor

Schematics

System Components

Pi Zero W, Pi Camera v2.1 and Huawei MS2131i-8 cellular modem with Soracom SIM card

Code

AI Surveillance Script

Python
Script that will detect persons in front of the camera and send an email or SMS text message
# USAGE
# python pi_ai_camera.py --conf conf.json

# import the necessary packages
from pyimagesearch.tempimage import TempImage
from picamera.array import PiRGBArray
from picamera import PiCamera
import argparse
import warnings
import datetime
import twilio
import imutils
import json
import time
import cv2
import smtplib

from twilio.rest import Client
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-c", "--conf", required=True,
        help="path to the JSON configuration file")
args = vars(ap.parse_args())

# filter warnings, load the configuration and initialize
# client
warnings.filterwarnings("ignore")
conf = json.load(open(args["conf"]))
client = None

# check to see if email should be used
if conf["use_email"]:
        # connect to dropbox and start the session authorization process
        email_from = conf["email_from"]
        email_to = conf["email_to"]
        email_server = conf["email_server"]
else:
        twilio_account_sid = conf["twilio_account_sid"]
        twilio_auth_token = conf["twilio_auth_token"]
        twilio_from = conf["twilio_from"]
        twilio_to = conf["twilio_to"]

# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = tuple(conf["resolution"])
camera.framerate = conf["fps"]
rawCapture = PiRGBArray(camera, size=tuple(conf["resolution"]))

# allow the camera to warmup, then initialize the average frame, last
# uploaded timestamp, and frame motion counter
print("[INFO] warming up...")
time.sleep(conf["camera_warmup_time"])
avg = None
lastUploaded = datetime.datetime.now()
motionCounter = 0

# capture frames from the camera
for f in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
        # grab the raw NumPy array representing the image and initialize
        # the timestamp and occupied/unoccupied text
        frame = f.array
        timestamp = datetime.datetime.now()
        text = "Unoccupied"

        # resize the frame, convert it to grayscale, and blur it
        frame = imutils.resize(frame, width=500)
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        gray = cv2.GaussianBlur(gray, (21, 21), 0)

        # if the average frame is None, initialize it
        if avg is None:
                print("[INFO] starting background model...")
                avg = gray.copy().astype("float")
                rawCapture.truncate(0)
                continue

        # accumulate the weighted average between the current frame and
        # previous frames, then compute the difference between the current
        # frame and running average
        cv2.accumulateWeighted(gray, avg, 0.5)
        frameDelta = cv2.absdiff(gray, cv2.convertScaleAbs(avg))

        # threshold the delta image, dilate the thresholded image to fill
        # in holes, then find contours on thresholded image
        thresh = cv2.threshold(frameDelta, conf["delta_thresh"], 255,
                cv2.THRESH_BINARY)[1]
        thresh = cv2.dilate(thresh, None, iterations=2)
        cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
                cv2.CHAIN_APPROX_SIMPLE)
        cnts = imutils.grab_contours(cnts)

        # loop over the contours
        for c in cnts:
                # if the contour is too small, ignore it
                if cv2.contourArea(c) < conf["min_area"]:
                        continue

                # compute the bounding box for the contour, draw it on the frame,
                # and update the text
                (x, y, w, h) = cv2.boundingRect(c)
                cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
                text = "Occupied"

        # draw the text and timestamp on the frame
        ts = timestamp.strftime("%A %d %B %Y %I:%M:%S%p")
        cv2.putText(frame, "Room Status: {}".format(text), (10, 20),
                cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)
        cv2.putText(frame, ts, (10, frame.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX,
                0.35, (0, 0, 255), 1)

        # check to see if the room is occupied
        if text == "Occupied":
                # check to see if enough time has passed between uploads
                if (timestamp - lastUploaded).seconds >= conf["min_upload_seconds"]:
                        # increment the motion counter
                        motionCounter += 1

                        # check to see if the number of frames with consistent motion is
                        # high enough
                        if motionCounter >= conf["min_motion_frames"]:
                                # check to see if dropbox sohuld be used
                                if conf["use_email"]:
                                        # write the image to temporary file
                                        t = TempImage()
                                        cv2.imwrite(t.path, frame)

                                        # Create the container (outer) email message.
                                        msg = MIMEMultipart()
                                        msg['Subject'] = 'Intruder Alert!'
                                        msg['From'] = email_from
                                        msg['To'] = email_to
                                        msg.preamble = 'Person detected in camera view'
                                        # Open the files in binary mode.  Let the MIMEImage class automatically
                                        # guess the specific image type.
                                        fp = open(t, 'rb')
                                        img = MIMEImage(fp.read())
                                        fp.close()
                                        msg.attach(img)

                                        # Send the email via our own SMTP server.
                                        s = smtplib.SMTP(email_server)
                                        s.sendmail(email_from, email_to, msg.as_string())
                                else:
                                        # send SMS message via Twilio
                                        client = Client(twilio_account_sid, twilio_auth_token)

                                        message = client.messages \
                                            .create(
                                                 body='Person detected in camera view',
                                                 from_=twilio_from,
                                                 to=twilio_to
                                             )

                                        
                                # update the last uploaded timestamp and reset the motion
                                # counter
                                lastUploaded = timestamp
                                motionCounter = 0

        # otherwise, the room is not occupied
        else:
                motionCounter = 0

        # check to see if the frames should be displayed to screen
        if conf["show_video"]:
                # display the security feed
                cv2.imshow("Security Feed", frame)
                key = cv2.waitKey(1) & 0xFF

                # if the `q` key is pressed, break from the lop
                if key == ord("q"):
                        break

        # clear the stream in preparation for the next frame
        rawCapture.truncate(0)

Ai Surveillance JSON Configuration File

JSON
Default configuration file for AI surveillance system
{
        "show_video": false,
        "use_email": true,
        "min_upload_seconds": 3.0,
        "min_motion_frames": 8,
        "camera_warmup_time": 2.5,
        "delta_thresh": 5,
        "resolution": [640, 480],
        "fps": 16,
        "min_area": 5000,
        "phone": "16179028287",
        "email_from": "ai_camera@email_address.com",
        "email_to": "your@email_address.com",
        "email_server": "localhost",
        "twilio_account_sid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
        "twilio_auth_token" = "your_twilio_auth_token",
        "twilio_from": "+18005551212",
        "twilio_to": "+18005551212"
}

OpenCL for the VC4 GPU

OpenCL API for the Videocore IV CPU on the Pi Zero

OpenCV

OpenCV AI vision framework

Credits

Jim Ewing

Jim Ewing

3 projects • 16 followers
Embedded systems developer. AI developer. Blockchain developer. Expertise in wireless communication, solar inverters, handheld instruments.
Lucas E

Lucas E

1 project • 0 followers
Harvey Mudd College, Computational Biology Major.

Comments