Reality Capture rooftop garden with IoT sensors

Intro

This project is an expanded project of the digital twin greenhouse with IoT sensors.

This time we created interactive 3D experiences of real-world spaces - a rooftop garden, crafted a complete walkthrough of this space in a 3-dimensional, interactive platform with a 360-degree virtual tour.

In the real world, sensors were deployed in corresponding positions. Within this 360 tour, tags were added for notating important features of those sensors, and the pop-up windows showed the real-time data which was collected from a real garden.

p.s. This project is now live on a Smart Village Solution Page. Visit here to experience it by yourself.

Full-screen experience: rooftop garden

LiDAR

"LiDAR (light detection and ranging) is a technology invented in 1960 that measures the time it takes for light to touch an object and reflect back to calculate the distance, similar to how radar works, except that a LiDAR scanner calculates distance and depth by emitting a 'laser beam' and timing its return speed to scan and create a 'depth map' of your space. This principle is similar to how bats perceive the distance of an object through 'ultrasonic echoes'.

Traditionally, LiDAR scanners are mounted to airborne or terrestrial platforms, and in recent years it has been used mainly in automotive electronics. LiDAR can enhance the experience by augmenting AR technology and may be widely used in consumer electronics in the future.

We now have the power of LiDAR in our own hands.

MATTERPORT

Matterport has developed a hardware device and software platform that can be used to present a touring effect similar to that of a surroundings website. Its most important feature is the ability to capture 3D spatial data and to obtain a 3D model of the interior in general. It is also possible to present the Doll House function and the advancing effect when switching between the surroundings.

Matterport for iOS is an app that makes it easy to create a 3D model of any space with your iPhone or iPad has released an update today to support the LiDAR sensor on the iPhone 12 Pro, 13 Pro, 14 Pro, and Pro Max and iPad Pro.

Here’s how Matterport describes its Capture app for iPhone and iPad:

Create cloud-based 3D models of real spaces to experience and share. The Matterport 3D Capture System provides a full end-to-end solution to quickly and consistently capture and create dimensionally-accurate and visually-realistic digital twins of your space. Matterport’s goal is to digitally transform the built world. That means every building, space, and place that you can physically experience - and there are more than 4 billion of them out there. In order to achieve this mission, we have to give everyone the power to capture in 3D with the devices in their pockets. We have to unlock this potential on iPhones, Androids, and other portable devices, including tablets.

After learning the above technology.

In this tutorial, I will show you how I build this project step-by-step.

1.Deploy devices

The engineers deployed a set of SenseCap sensors in the rooftop garden.

Hardware components

S2101 - Air Temperature and Humidity Sensor

S2102 - Light Intensity Sensor

S2103 - CO2, Temperature, and Humidity Sensor

S2104 - Soil Moisture and Temperature Sensor

S2105 - Soil Moisture, Temperature, and EC Sensor

SenseCAP Sensor Hub 4G Data Logger

Liquid Sensor Group:

Salinity Sensor; Dissolved Oxygen Sensor; pH Sensor; Liquid EC and TDS Sensor

SenseCAP S2120 8-in-1 LoRaWAN Weather Sensor

SenseCAP Outdoor Gateway - LoRaWAN

Note:

The Air Temperature and Humidity Sensor (S2101) and CO2, Temperature, and Humidity Sensor(S2103) should be installed with the probe facing downwards.

The Light Intensity Sensor (S2102) should be installed with the probe facing upwards, and the Soil Temperature and Humidity (conductivity) node (S2104 & S2105) should be installed with the probe inserted horizontally near the plant root.

2.3D Capture

By using Matterport App.

The first thing that you want to know is how to properly hold your smart device. The way you hold your phone is basically by having your elbows tucked in close to your side so you can firmly get a hold of your phone/iPad.

Now you want to lean back a little bit and I'll tell you exactly why. Most often it's very common to do this the same way you would take a picture of a panorama, where it's a vista point of this far away landscape. In this case, because all the surfaces are, relatively speaking, much, much closer to us, we can't do that. You can't have your body be the axis. What you need to do is have the phone be the axis. So to do that, again, just have your elbows tucked in close to your side to keep a firm grip on the phone.

And what you do is just tap the button right here to get the scan going. I'll tilt it so that the circle is aligned with the dot. It'll scan the first position and then I just rotate to the right and what I'm doing is just kind of pivoting around my big toe or the ball of my foot. And there you have that.

And just kind of keep going like this until I've done the complete 360.

So as soon as I'm done, all that information gets processed into the complete panorama and it uses Cortex to convert it into the 3D model.

I'm all done. As you saw on the screen, I did 20 scan positions for this rooftop garden.

Now I'm ready to upload. So to upload I can very simply just press the Upload button in the top right corner.

it'll ask you to sign in if you haven't yet signed into the account. Directly from the Capture app, you can either sign in or sign up for an account if you don't already have one.

3.Realtime Data - API

Matterport SDK https://matterport.github.io/showcase-sdk/docs/reference/current/index.html

SDK for Embeds https://matterport.github.io/showcase-sdk/sdk_home.html

Seeed Studio has a complete instruction document on how to make your first HTTP API call to SenseCAP HTTP API.

https://sensecap-docs.seeed.cc/httpapi_quickstart.html

GitHub

We open-sourced this project on GitHub.

Feel free to contact me & leave your comments 🧡！