Software apps and online services
Hand tools and fabrication machines
Community Supported Agriculture (CSA) schemes use greenhouses or polytunnels to grow seasonal organic vegetables. As they are using regenerative farming techniques without pesticides or herbicides, plant diseases and improving crop quality are big challenges. These greenhouses are also typically in areas without power or wired internet access.
What we want to do is to build a solution that can be easily be installed in a greenhouse, using image-based machine learning to perform plant disease detection.
Existing solutions focus mostly on crops like maize, wheat, rice and soybeans that are grown outside, using UAVs (Unmanned Aerial Vehicles) to gather images. Solutions created for greenhouses assume mains power availability and/or WiFi access.Our solution
The unit is mounted in the top of the greenhouse in such a way that the Spresense camera board can be used to take images of the plants in the greenhouse. The Sony Spresense and camera sensor allows us to capture images and perform embedded machine learning (ML) on a battery-powered microcontroller, without having to send the images to the cloud for processing. Instead, we can use the Spresense LTE extension board with either LTE-M or NB-IoT to just send the observation results of the ML algorithm. Results and photos are also saved to the on-board SD card.
This project used Edge Impulse to train and validate a classifier on almost 23 thousand (!) images of tomato plants in the New Plant Diseases Dataset. The classifier can detect 9 different plant diseases on tomato plants.Setting up the firmware
The firmware is based on the 4G LTE Connected Smart Wildlife Camera provided by Edge Impulse.
- Create an Edge Impulse account
- Install the Arduino IDE
- Install the Spresense Arduino Library
Spresensedevice, then under
- From GitHub, clone our greenhouse-ai repo
- From Edge Impulse, clone our plant-disease repository
- In your cloned Edge Impulse project, select
Arduino Libraryand click
build. Follow the instructions to add this library to your Arduino IDE
- In the Arduino IDE, click File->Open and then navigate to the greenhouse-ai repository folder you cloned earlier, and open the
plant-sketch.inofile in the
- Build the project to verify it compiles correctly
- In the
4g_camera.inofile, configure the APN with your SIM card details.
- Also in
4g_camera.ino,specify the web server details.
- Now, rebuild and upload the sketch.
- Solder a 2-pin JST-PH connector to the main board
- Remember to fit a shading seal to IC4 on the main board
- Insert the mini-spacers into the LTE extension board and attach the main board, making sure the board-to-board connector is properly seated
- Connect the camera module to the main board
- Insert an SD card into the SD card slot
- Insert an IoT SIM card into the SIM slot
- Attach the battery
- Push fit the Sony Spresense into the enclosure base, with the battery below the extension board
- Push fit the camera module into the enclosure lid, and push fit the lid into the enclosure base
- Attach the clamp to the enclosure base using a 25mm (1 inch) M5 bolt and nut
- Program the firmware on the Spresense using Arduino
The enclosure is designed so that everything press fits, with no screws required. A 25mm (1 inch) M5 bolt and nut is required to connect the clamp to the enclosure. The designs can be edited in OpenSCAD if required.
Printed on a Prusa i3 MK3 printer with the following settings:
- Filament: PETG
- 0.20mm SPEED
- Infill: 20%
- Supports: None
It takes about 2 hours to print the enclosure and 30 minutes for the clamp.
The current firmware sends the classification results as JSON using an HTTP POST request, using the following format:
I wrote a little RESTful API (based heavily on a tutorial by Rob Kendal) that takes the JSON object and stores it as a JSON file, so that you can run it on a basic web server without requiring a database.
I use DigitalOcean to host my server, but you can put it anywhere that has Node.js support. Follow the steps on DigitalOcean to set up a Node.js production environment on an Ubuntu VPS, and the clone the https://github.com/gniezen/greenhouse-ai repository on your server. Run
npm install to install all the dependencies, and
npm start to check if it works. If it's working, press
Ctrl-C to stop the server, and start it using PM2 instead by running
pm2 start server.js.
I installed the the unit at Summit Good, a local CSA scheme with a remote greenhouse/polytunnel on the outskirts of the city.
The unit has the following startup sequence:
- initialise the camera
- mount the SD card (to save photos of observations)
- start the GPS module (to get an accurate timestamp for the observations)
- connect the the LTE/NB-IoT network (to send the observation results)
- start the camera
When the ML algorithm detects a plant it gets classified on the device into one of the following when a plant is detected:
- Healthy plant
- Bacterial spot
- Early blight
- Late blight
- Leaf mold
- Leaf spot
- Mosaic virus
- Spider mites
- Target spot
- Yellow leaf curl virus
The results, together with the local date and time and battery level, are converted into JSON format. The observation results are then sent to the web server, and displayed in a table as follows:
Results will only be saved if the device got an accurate timestamp via GPS, and when the cellular network is not available it will still save the results to the SD card. After sending the results, the device will shut down the cellular and GPS modules and go into a deep sleep state for 3 hours to save battery power.Conclusion
In this project we implemented a battery-powered solution for remote sensing of tomato plant diseases in greenhouses without access to wired internet. ML classification occurs on the device and only the observation results are sent over an LPWAN network to a web server.
This project can easily be modified to recognise other types of plant disease, or multiple units can be used to cover a larger area. The unit could also be mounted above seedling trays to try and detect the number seedlings that have sprouted, using the new FOMO algorithm on Edge Impulse. Another idea would be to try and detect whether a seedling tray is wet or dry and needs watering.
I look forward to see how you modify and build on top of this design!