Hardware components | ||||||
 |
| × | 1 | |||
 |
| × | 1 | |||
Software apps and online services | ||||||
 |
| |||||
obniz identifies what is on camera and if it is a cat, it sounds a buzzer.
MaterialsHow to makeHardware connection
Connect the speaker to the obniz Board, referring to the library of speakers.
Software
In order to recognize a cat, use an object detection library called Coco SSD from TensorFlow.js.
In the bindPage() function, we're setting up and loading the webcamera.
After that, the detection() function detects the object based on the captured video.
Since the detection() function calls itself recursively, the detection is done continuously.
The judgCat() function in the detection() function determines whether the object is a cat or not.
Program <html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link
rel="stylesheet"
href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css"
/>
<link rel="stylesheet" href="/css/starter-sample.css" />
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/coco-ssd"></script>
<script src="https://obniz.io/js/jquery-3.2.1.min.js"></script>
<script
src="https://unpkg.com/obniz@3.x/obniz.js"
crossorigin="anonymous"
></script>
<style>
#canvas {
position: absolute;
top: 0;
left: 0;
pointer-events: none;
}
#wrapper {
position: relative;
}
</style>
</head>
<body>
<div id="wrapper">
<video id="video" width="800px" height="450px" autoplay="1"></video>
<canvas id="canvas" width="800px" height="450px"></canvas>
</div>
<script>
let canvas = document.getElementById("canvas");
let ctx = canvas.getContext("2d");
ctx.strokeStyle = "#00ff33";
ctx.font = "18px 'serif'";
const obniz = new Obniz("OBNIZ_ID_HERE");
obniz.onconnect = async function() {
let speaker = obniz.wired("Speaker", { signal: 0, gnd: 4 });
bindPage();
async function bindPage() {
let video;
try {
video = await loadVideo();
} catch (e) {
console.error(e);
return;
}
detection(video);
}
async function loadVideo() {
const video = await setupCamera();
video.play();
return video;
}
async function setupCamera() {
const video = document.getElementById("video");
if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
const stream = await navigator.mediaDevices.getUserMedia({
audio: false,
video: true
});
video.srcObject = stream;
return new Promise(resolve => {
video.onloadedmetadata = () => {
resolve(video);
};
});
} else {
const errorMessage =
"This browser does not support video capture, or this device does not have a camera";
alert(errorMessage);
return Promise.reject(errorMessage);
}
}
function detection(video) {
cocoSsd.load().then(model => {
function judgeCat() {
ctx.clearRect(0, 0, canvas.width, canvas.height);
model.detect(video).then(predictions => {
predictions.forEach(async element => {
draw(element);
console.log(element.class, element.score);
if (element.class == "cat") {
speaker.play(1000);
await obniz.wait(1000);
speaker.stop();
}
});
});
setTimeout(arguments.callee, 1000);
}
judgeCat();
});
}
function draw(element) {
ctx.fillStyle = "#00ff33";
ctx.strokeRect(
element.bbox[0],
element.bbox[1],
element.bbox[2] / 2,
element.bbox[3] / 2
);
ctx.fillRect(element.bbox[0], element.bbox[1] - 20, 100, 20);
ctx.fillStyle = "#ff0000";
ctx.fillText(element.class, element.bbox[0] + 5, element.bbox[1] - 5);
}
};
</script>
</body>
</html>Before we get into the project, let's look into obniz.
Here → obniz for DIY electronics
obniz is a cloud-connected IoT development board. You can program on the web browser of any smartphone or computer and the command is sent to obniz through the internet via obniz cloud. By connecting the obniz to the cloud through wifi, users can remotely control devices that are physically connected to obniz.
Thanks to this cloud based approach, you can program with Python, Javascript, or other languages you prefer and control hardware directly. You don't need to integrate firmware into the device side. Recording and analyzing data is also easy with obniz cloud service.
Want to control hardware things with your current Python or Javascript skill? Want to start IoT project but don't know where to start? Want to learn programming with languages you prefer?
obniz will help you broaden your viewpoint and develop both your SW and HW skills.
For more information, please visit our official website → Official Website
Where to get obniz board? → Amazon /Official Store
Comments