Crop Monitoring for Yield Optimiation

INTRODUCTION

Our team is very excited by this Hackster.io NXP Hovergames competition because optimizing food production positively impacts everyone.

By utilizing RGB-D camera data (NVDI, growth, pests, etc.), a JXCT 7-in-one soil sensor (NPK, pH, salinity, moisture, and temperature), GPS data (location and elevation), and Bosch BME688 sensor data (greenhouse gases, moisture, temperature) we measure NDVI plant health, soil health, and greenhouse gas improvements. This allows us to provide actionable recommendations for water, fertilization, and the best areas to plant/not plant.

The Normalized Difference Vegetation Index (NVDI) is a standard used with satellite imagery to identify vegetation health. NVDI (calculated as NDVI = (NIR - RED) / (NIR + RED)) yields a number between -1 and 1: -1..0 = dead or no plants, 0.0.33 = unhealthy plants, 0.33..0.66 = moderately healthy plants, and 0.66..1 = healthy plants. While this gives you an overall indicator of plant health, it doesn’t explain the problem or the action farmers should take to correct it. In addition, existing systems that collect this data are typically only updated once a week via satellite data. We can collect this data with drones daily to track crops more effectively. Our down-looking sensor (the Oak-D depth camera) can capture plant growth rates.

We can then direct the quadcopter to collect more information about underperforming areas. The standard for testing soil health looks at pH (6.5 is optimal for growth), nutrients (NPK = Nitrogen, Phosphorous, and Potassium), salinity (via electric conductivity), moisture, and temperature. To collect this data, the quadcopter will land in suspect areas inserting a five-pin probe into the ground to take a sample. We can also take close-up pictures of the crops to identify possible pests or diseases. We can provide detailed crop health data and actions to improve yields.

The 97% of greenhouse gases that cause global warming include carbon dioxide (CO2 79%), Methane (CH4 11%), and Nitrous Oxide (NO2 7%). The BME688 can detect all of these compounds in the air or on landing. By making crops healthier, we allow them to remove more C02 from the air and convert it into O2. By targeting fertilizers, we can reduce NO2 and CH4. In addition, the Bosch sensor will allow us to collect temperature, moisture, and pressure. We can use this data to show reductions in greenhouse gases. We intend to look for correlations between soil temperature and moisture by looking at the air readings at a specific height above the ground.

What we were able to achieve:

• Drone flight
• Collecting BME688 data.
• Collecting sample soil using a 7in1 sensor.
• Collecting OAK-D imagery data.
• Calculates NVDI data using Open Drone Map.

OVERVIEW

The next few sections will cover the hardware and software builds.

HARDWARE BUILD

This is a summary of the steps to build the hardware.

1. NXP KIT-HGDRONEK66 basic drone

2. 7 in 1 Soil Sensor RS485 adapter

3. TF-Luna Lidar

4. OAK-D Camera

5. BME688 Gas Sensor

STEP 1. NXP KIT-HGDRONEK66 basic drone using the directions here

STEP 2. 7-in-1 Soil Sensor RS485 adapter

STEP 3. TF-Luna Lidar

STEP 4: OAK-D Camera

STEP 5: BME688 Gas Sensor

SOFTWARE BUILD

Special thanks to all the open-source contributors at:

• NavQ+ [link](https://nxp.gitbook.io/8mpnavq/) and RudisLabs [link](https://github.com/rudislabs/navqplus-create3-images/releases/download/)
• PX4 [link](https://px4.io/) and NXP FMUK66v3 [link](https://www.nxp.com/design/designs/px4-robotic-drone-vehicle-flight-management-unit-vmu-fmu-rddrone-fmuk66:RDDRONE-FMUK66)
• BME688 [link](https://github.com/akchobby/bme680-python) and [Bosch](https://www.bosch-sensortec.com/products/environmental-sensors/gas-sensors/bme688/)
• Luxinos OakD [link](https://www.luxonis.com/)
• Open Drone Map [link](https://opendronemap.org/)
• OpenCV [link](https://opencv.org/)
• ROS 2 [link](https://www.ros.org/)

The source for this project is available [here](https://github.com/theguybieber/hovergames3). The README.md file provides the overall directions, but here is a summary of the top level steps.

1. NAVQ+ - follow directions in navq_plus

2. FMUK66v3 - follow directions in fmuk66v3

3. TFMini - follow directions in tfmini

4. Soil - follow directions in soil

5. BME688 - follow directions in bme688

6. OAKD - follow directions in oakd

7. ODM - follow directions in odm

USAGE

• Manually fly the drone over the target area.
• Run scripts to collect BME, Soil, and image data as needed.
• Post-process the imagery to get NDVI calculations.

Here is an example of NVDI data (runs on a mac):

Here is an example of the soil data (runs on a teensy 2.0 right now).

txRequest:  01 03 00 06 00 01 64 0B
rxResponse: 01 03 02 02 C8 B8 B2
txRequest:  01 03 00 12 00 01 24 0F
rxResponse: 01 03 02 00 00 B8 44
txRequest:  01 03 00 13 00 01 75 CF
rxResponse: 01 03 02 00 D8 B8 1E
txRequest:  01 03 00 15 00 01 95 CE
rxResponse: 01 03 02 00 00 B8 44
txRequest:  01 03 00 1E 00 01 E4 0C
rxResponse: 01 03 02 00 00 B8 44
txRequest:  01 03 00 1F 00 01 B5 CC
rxResponse: 01 03 02 00 00 B8 44
txRequest:  01 03 00 20 00 01 85 C0
rxResponse: 01 03 02 00 00 B8 44
Soil pH: 7.12 pH
Soil Moisture: 0.00 RH
Soil Temperature: 21.60 deg C
Soil Conductivity: 0.00 1us/cm
Soil Nitrogen: 0.00 mg/kg
Soil Phosphorus: 0.00 mg/kg
Soil Potassium: 0.00 mg/kg

Here is an example of the gas data (runs on the navq+).

Calibration data:
par_gh1: -68
par_gh2: -9013
par_gh3: 18
par_h1: 715
par_h2: 1022
par_h3: 0
par_h4: 45
par_h5: 20
par_h6: 120
par_h7: -100
par_p1: 36352
par_p10: 30
par_p2: -10371
par_p3: 88
par_p4: 8904
par_p5: -151
par_p6: 30
par_p7: 33
par_p8: -1621
par_p9: -3017
par_t1: 26009
par_t2: 26446
par_t3: 3
range_sw_err: 1
res_heat_range: 1
res_heat_val: 41
t_fine: 110833


Initial reading:
gas_index: 0
gas_resistance: 438919.8456922418
heat_stable: False
humidity: 28.215
meas_index: 0
pressure: 973.3
status: 32
temperature: 21.65


Polling:
21.65 C,973.29 hPa,28.21 %RH
21.68 C,973.30 hPa,28.22 %RH,5684.846331497602 Ohms
21.74 C,973.29 hPa,28.21 %RH,6090.597639893414 Ohms
21.80 C,973.31 hPa,28.17 %RH,6844.919786096257 Ohms
21.86 C,973.30 hPa,28.13 %RH,7722.007722007722 Ohms
21.91 C,973.29 hPa,28.06 %RH,8599.838753023381 Ohms
21.95 C,973.31 hPa,27.99 %RH,9370.424597364568 Ohms

LESSONS LEARNED

• The drone drifts a bit using just the GPS and IMU. Recommend moving to image stabilized flight.
• Adding a downward LIDAR helps with position/elevation hold.
• The FMU to NAVQ ethernet port comes up as the preferred route and messes up network connectivity.
• It is easy to overload the FMU if you do not limit the speed it sends data.
• Spare parts are great for crashes.
• The power connector for the NAVQ board also fits into some of the connectors on the FMU board. Watch out.

FUTURE WORK

• Deployment of soil sensor from the drone.
• Collect BME688 data to train models using BME AI Studio. We want to "sniff" healthy crops, greenhouse gases, and soil data.
• Video flight stabilization and offboard mission routing to cover large areas.
• Package all modules as ROS nodes.