Jdetect AI Security System Powered by TensorFlow Lite

Jdetect AI security system powered by Tensorflow Lite

Jdetect is an AI security system, powered by Tensorflow Lite.

What does this black box do?

To understand Jdetect, let's look at a user story. The typical experience starts from when you decide you need a home security camera system.

When choosing an IP security camera, there are so many things to consider.

Shape. Bullet is too big, dome has reflection problem, eyeball is a good choice.

Weather proof is important as the cameras will be installed outside.

There are many image resolution options to choose from.

Lens decides how wide and how far the camera can see.

There are many cameras with wifi function, but I reckon wired installation is the best way and trouble free.

Mobile app is important.

Apart from just being a camera, some cameras have extra functions like built in audio, siren and flashlight.

If you search on taobao

Or ebay.

You will realise the best cameras come from Hikvision

and Dahua.

In my case, I picked Ezviz, it's a sub-brand of Hikvision. Their cameras do not have very high pixels but with extra features of 2 way audio and siren built in. I picked the base model for around 300 yuan,

They also have models with AI function, the price is much higher,

The price difference become more noticeable especially when you are buying multiple.

Now the camera is installed.

This is the Ezviz app, and everything is set up and working. We block out the image for privacy reason.

You will want the security camera system to work like this. Camera detect the intruder, and send alert to owner.

The owner receive the alert, and is able to react to the satiation, right at the time.

You can check live camera view,

talk to the person,

sound the siren or trigger other home automation.

Last resort, you will call law enforcement.

It is a perfect plan, until you come across the problem of false alarm.

If you got tree and sun shade in the camera view, give it a bit of wind, it will cause so much movement and totally confuse the motion detection on the camera. This is where outdoor installation is different to indoor installation. The motion detection on the camera just don't know how to handle these.

When you get hundreds of false alarm, the alarm function become useless, and it basically defeats the whole purpose of having a security camera system.

At this point, I wanted to throw them away.

Then the solution come along.

Jdetect is the solution.

Jdetect is a human detector.

Jdetect adds AI to IP cameras and completes the system, makes it useful again.

Talking about human detection, we have to start from Tensorflow object detection.

Tensorflow object detection provides a complete set of API, pre-trained models, and methods to deploy to mobile devices.

On Tensorflow github website, you can find out how to use the API.

The model zoo provides many models already trained on COCO dataset. Person is one of the 80 classes of the COCO dataset, so we can use these pre-trained models for human detection off the shelf, straight away.

Different models have different trade-off between accuracy and run time.

We also refer to Tensorflow for Poet tutorial for how to run Tensorflow inference on mobile device. There are 2 ways.

Tensorflow mobile, which is marked as deprecated, but still useable for now.

Most of the models from the model zoo can run on Tensorflow mobile, but just they will run slow.

Tensorflow lite is the new way, and the optimised way for inference on mobile device, meaning it's much faster, but it has its own limitation. Not all Tensorflow operations are supported yet, meaning not all the models from the model zoo will work on Tensorflow lite. So far, SSD is the only one works from our test.

This is how we convert Tensorflow trained pb model to Tensorflow lite model.

And this is how we use the Tensorflow lite model on Android for object detection inference.

We tried and compared a few models from the model zoo.

Faster rcnn model has good accuracy, but it can only run on Tensorflow mobile, and the speed is low.

We tried ssd fpn model, it has even higher accuracy than faster rcnn, but again, it's even slower on Tensorflow mobile. At around 30 seconds per frame, it's not workable for real time human detection.

Then we tested ssdlite model, it works on Tensorflow lite, it takes 1.6 second, which we think is the right balance between accuracy and speed.

After all the test and comparison, ssdlite Tensorflow lite model is the one we selected for Jdetect application.

There are different ways to add AI to home security system.

You can have AI on cloud, good server hardware means you can run more powerful model, but then you create a dependency with the cloud which means cost and network traffic, and probably delays.

You can also have AI on edge,

Normally you will not be able to do much with the camera, the camera is a manufactured product, and the firmware is controlled by the manufacturer. You can however throw the existing cameras away and buy the new AI camera, but that means additional cost, and you will again be at the leniency of the vendor, in terms of when or if they will update their firmware.

Or you can add AI to the local network,

AI can run on a NVR, the network video recorder, or on a local PC, but I do not want another equipment, the power consumption and noise of another equipment put me off.

Jdetect is the best solution.

We let cameras do what they do best, and leave AI thinking to a dedicated device, Jdetect.

With Jdetect, we re-purpose a common Android TV box to do human detection. This is the spec of the box I use, but any current model Android TV box will be able to run Jdetect.

If you search on ebay, you will find they are normally priced around $50.

Now we have the Jdetect box.

From this box, Jdetect connects to IP cameras on the local network by RTSP, real time stream protocol.

Most of current cameras can support main stream and sub stream simultaneously, and sub-stream is the best for Jdetect, because first, Tensorflow object detection does not require very high image resolution,

And second, sub-stream will minimise the load on the local network.

I did not have to make any change to the setting on the camera, as long as you have the correct URL for the sub-stream, it will work straight away.

On the Jdetect box, we install the Jdetect software, Jdetect is actually made up of 3 applications,

Jmotion detects motion on camera image,

Jperson detects if there is any person on image,

Jemail sends alert message,

All 3 run as service in the background, which mean the box is still usable as a TV box.

Jdetect is placed on the TV stand and connected to the router.

Jdetect use SMTP protocol for sending emails,

You will get instant alert, with no false alarm, and

Jdetect can detect human even when there is only a partial view of human on the image.

This is an example email from Jdetect. Can you see there is someone at the door way?

This diagram summarise Jdetect system.

Multiple cameras (say up to 8) and Jdetect box are on the same local network.

Jdetect gets image from cameras and checks if there is any human on the image.

If yes, it will send alert email to the registered email address. There is no limit on the number of email address.

Jdetect is designed to be an always on system,

It will auto start when you power on the box.

It will handle the interruptions of power and network automatically.

And it keeps comprehensive log of its activities, so you can establish a complete time line of events.

Jdetect keeps all the images it receives,

A 64GB sd card can hold 10 years history for my system with 3 cameras.

And these are good data for re-training the Jdetect human detection model, to improve its accuracy.

With traditional home security system, when you go out, you key in a passcode on the control panel to arm the system, and when you come home, you key in a passcode to disarm the system.

Jdetect can sense whether you are home or away, and automatically arm and disarm.

This way, you further reduce unnecessary alert.

To use this function, you need to set fixed IP address on your mobile phones and register the IP address with Jdetect.

Please note this is an optional function, Jdetect is more than capable of working 247 non-stopping.

It's easy to test and see if Jdetect system is working properly.

When Jdetect is armed, show yourself to one of the cameras,

If Jdetect system works properly, you will get an instant alert email.

Jdetect re-purpose an Android TV box, but because everything is run as background service, the box is still completely useable as a TV box, you can use the box to watch TV, movie etc at the same time when Jdetect is running.

Ok, I hope you get the idea of Jdetect so far.

The benefits of Jdetect system can be summarised in 5 key selling points.

First, Jdetect add AI to security system, works with any IP cameras, and bring new life to existing investment.

Secondly Jdetect does it with no ongoing cost, no cloud subscription, and Jdetect has very low power consumption, low heat and no noise.

It's easy to use, after the initial configuration, it's basically set and forget, as long as it's plugged in, it will work and it will auto arm and disarm.

Jdetect box has a low profile, it is small and almost deceiving.

It does not look anything like a security system.

Other people will not know what it is even when they look at it.

It is a TV box, but not just a TV box.

Last, and the most important point about Jdetect, and what sets Jdetect apart from other products, is, it is getting better.

Jdetect is an evolving product, always tracking to the newest and latest development of AI technology,

Whenever there is a new and better object detection model published, and become supported on Tensorflow lite, you can plug it in on Jdetect.

Jdetect is tracking to the most dynamic and vibrant development of AI technology on object detection.

Jdetect can be re-trained with customer data, and can also be customised to suit specific requirement.

Thanks for watching.

We hope you find Jdetect interesting,

If you have any query, please feel free to contact us.