R.A.D - Farming

Overview

The goal: to make farmers fields safer in countries that have experienced conflict by locating unexploded ordinance in their fields. I hope to accomplish this by designing and integrating multiple detection methods into a drone based platform, including a gas sensor, thermal imaging, image recognition, and a deployable 'metal' detector.

1. Introduction

When considering how to use a drone to help improve food security, I went through a lot of ideas, some a little silly.

• Use a drone to herd cattle and check the herd health.
• Use a drone to check on crop health
• Use a drone to watch for larger pests and scare them off (AKA the flying scarecrow)
• Mapping flower patches to help bee keepers place hives
• Survey pasture land for bare dirt at risk of erosion, then drop native seeds for something fast growing to try to limit soil loss
• Some other good ideas that were unrelated to farms entirely

It was then I realized how lucky I was. I am to live in country where I can walk into a field and my biggest worry is upsetting the farmer. In countries such as Ukraine, farmers have to choose between their crops and their safety because of unexploded ordinance left laying around.

This isn't fair or equitable so I propose a system to help clean up their fields. The solution will be based on an autonomous unmanned aerial vehicle (a Hovergames provided drone) to detect unwanted items in their fields

2. Main concept

The developed system will attempt to do this by scanning a field via drone in a search pattern, assessing and documenting anything it finds. Assessment will be on multiple indicators, allowing it to flag a location on the map as green/yellow/red and provide photos and sensor images

1. Visual scan - Soil disturbance (fresh dirt or anything shining)

2. Temperature variance with surrounding soil (metal will heat/cool at a different rate than the soil) via thermal camera

3. Attempt to detect if it is a metal object with a deployable sensor lowered while holding over the location - If conditions 1 or 2 indicate a potental risk

4. Use the Bosch Sensortech BME688 AI-enabled environmental gas sensor to register traces of explosives (For the purposes of testing I will use a plant-based substitute)

Photos will ideally be straight down, before lowering the sensor. The IR array will be scanning straight down to scan the ground. In crops, my hope is the prop wash will help clear the area of crop enough to allow clearer scans

3. System design

'Skyhook' mechanism options

1. Tape Roll (aka Cubesat Antenna style)

• Not directly usable, most designs all call for single use deployment and don't allow for rolling up
• Rewinding would require either retaining the whole spring mechanism to retract it or a more powerful (stepper?) motor

2. Scissor lift style

• use multiple hinged pivots to expand and lower platform like an extending grabber - https://www.instructables.com/Extending-Grabber/
• Drive with opposing servos attached to top arms
• Are two needed to minimize swaying and interference with flight? Will assembly droop without constantly running servo to maintain position.
• Bike spokes are a potential for a rigid and light choice

3. Fishing - use a continuous rotation servo/stepper motor and a spool/spools to lower on sensor on fishing line. May allow too much swinging of payload. *This may not be allowed under Canadian law so wasn't a great choice

4. Get flying now - Suspend sensors to UAV legs with elastic shock cord or equivalent (simplest but forced to use low flights to get close for sensitivity )

5. Combine 2 & 3 - Use fishing line and pulley (block and tackle style) to lower the force needed from servo/motor, use scissor mechanism for stability of payload

Final verdict - After a lot of research I stumbled onto the Pantograph, the design used for electric trains as a power contract with overhead lines. This mechanism has the advantage of keeping the head level at the same angle when retracted and extended. Please see the version 1 of this design in the objects below

Assumptions

• FMU will talk to the microcontrollers over UART/I2C/CAN to read data
• I will successfully train Machine Learning to tell when it’s a metal object in the thermal camera
• Multiple passes will be needed (Morning when objects are warming slower than the ground, or late day when they are cooling slower) and the IR array needs to see variance either direction as a red flag.
• Normal flight is expected to be in a clear field and will need to have a ‘fence’ set for the scan area so it doesn’t run off into a bush.
• Due to the nature of the intended objects, equivalents will need to be chosen for testing. Biodegradable substances such as Vineger for the gas sensor training and metal pipes painted black to for the thermal and magnetometer (Black should allow it to warm enough in sunlight to stand out from cold snow)
• This won’t be perfect, but hopefully it well help someone

Risks

• Drone is over 250g and will require a license to fly in Canada. This needs to be obtained prior to any outdoor test flights. - Completed
• Procurement for the sensor board will run over budget, or run into engineering issues[^1], or will hit supply chain delays and prevent flight before winter -Test Flights did run into weather issues
• Thermal camera may be too low in resolution or sensitivity to pick up anything small or partially buried - This was the case. Initial testing with the thermal camera modules did show possible uses however this level of sensitivity appears out of reach.
• Thermal camera may be too slow to refresh, forcing a low flight speed and lowering coverage per charge - Export compliance mandates a 9hz refresh for higher resolution thermal cameras however these weren't available to test anyways.
• If the sensor assembly is heavy, will holding the servos in place impact battery life? Do the servos need to be disabled and only powered up periodically? - Sensor assembly design was completed however is still awaiting the first test version to be 3d printed. Thiswill be a new project for this specifically
• If the sensor assembly is too 'loose' when retracted, will it impact flight stability as it swings? - To be determined however the final Pantograph design shows promise for folding away cleanly
• Finding appropriate locations to test fly - Wewere able to find some areas where flights were permitted
• My unfamiliarity with the PX4 platform & software will keep the drone grounded - Well...watch this and decide

• How sensitive is the balance on the drone and will unbalance impact battery life or stability?
• Winter weather will impact testing (Sub-Zero temperatures, snow and moisture buildup, lowered battery life) - This was correct. Snow also makes for much less graceful landing than expected.

Stretch goals (v3.0 maybe?)

• Way to drop a flag or paint to mark the location directly.
• Camera gimbal
• Access to a 3d printer is the main limiting factor for making fixtures
• Spray chalk or silly string? Likely something in a spray can, with a servo activator and a mount
• Ability to measure ground height to ensure sensor is lowered to right height
• Laser ‘scanning’ to look for a reflection of shiny metal as a verification step? (just a laser pointer mounted on the bottom plus looking for the reflection on video)
• Visual collision avoidance or something else to keep a clear flight path

Sensor system notes

• Needs to fit securely onto chassis
• Attach with Zip strips and rubber spacers to absorb vibration
• All components need to be secured (no pins that can work loose, secure sub-boards with hot glue to keep vibrations from damaging solder joints
• Needs I2c port to attach to FMU (research) and needs pullups on I2C ports
• FLIR module needs SPI & I2C
• Power source (research)
• Include cut-able solder trace and pads to isolate Deb board and allow FMU direct access to IR array
• Make sure any extra pins are broken out into solder pads for future use
• Panelize with the magnetic sensor board
• Add holes for mounting blocks or tie loops
• Add strain holes so it will break instead of dragging down the drone (hopefully)
• Lift mechanism needs prototyping to find best option. Reference sketchbook
• If possible design for future port for flag marker and laser pointer
• Consider how to power the laser diode (will a 5v diode be bright enough?)

References

Background

• In 2016 Ukraine reported 600 fatalities from landmines during the first 2 years of war. https://www.theguardian.com/global-development/2016/apr/04/ukraine-attempt-defuse-landmines-as-more-are-planted

Prior Art

• Magnetometer suspended from drone for scanning at 5-10M altitude in Ukraine https://empr.media/discover-ukraine/tech/ukrainians-created-a-drone-for-searching-for-unexploded-ordnance-objects/
• Drone mounted thermal cameras for locating mines made with less metal https://www.pix4d.com/blog/landmine-clearing-thermal-camera-drone
• Small drones can buried landmines in the desert using airborne IR Thermography https://commons.lib.jmu.edu/cgi/viewcontent.cgi?article=2921&context=cisr-journal
• Commercial solution using Fluxgate magnetometer mounted to UAV legs and laser altimeter https://integrated.ugcs.com/magnetometer-drone
• Commercial solution - Magnetometer suspended below UAV https://integrated.ugcs.com/magarrow-uas-deployable-magnetometer
• https://blogs.icrc.org/inspired/2020/06/16/drones-infrared-cameras-mines/
• Drones and thermal cameras used to detect Russian "butterfly" landmines
• https://www.sciencedaily.com/releases/2018/06/180619122448.htm
• Researchers from Binghamton found the most accuracy in early morning as the plastic warmed faster than ground/rocks
• Timothy S. de Smet, Alex Nikulin. **Catching “butterflies” in the morning: A new methodology for rapid detection of aerially deployed plastic land mines from UAVs.** The Leading Edge, 2018; 37 (5): 367 DOI: 10.1190/tle37050367.1

Concept and research

• https://www.ndt.net/article/wcndt00/papers/idn106/idn106.htm
• https://ieeexplore.ieee.org/document/6291520 (not available possibly) or https://opg.optica.org/ao/viewmedia.cfm?uri=ao-34-25-5809
• http://www.goldrefiners.com/blog/2017/6/16/inexpensive-new-thermal-cameras-let-treasure-hunters-see-through-walls

Metal Detector designs

• Pin point metal detector, Arduino based https://www.hackster.io/mircemk/diy-pin-pointer-metal-detector-1391ff
• Simple Arduino based metal detector https://www.hackster.io/mircemk/diy-very-simple-arduino-metal-detector-5a54f8
• Pulse induction metal detector https://www.hackster.io/mircemk/diy-arduino-based-pulse-induction-metal-detector-c6f244
• Induction Balance metal detector with dual coils https://www.hackster.io/mircemk/diy-sensitive-arduino-ib-metal-detector-d5e029
• Upgraded Induction Balance metal detector with discrimination (ferrous vs non-ferrous metals)<https://www.hackster.io/mircemk/diy-sensitive-arduino-ib-metal-detector-with-discrimination-b55e9d
• Magnetic sensor (may be too short range) https://www.hackster.io/Infineon_Team/ai-powered-joystick-907a1f

UAV Design References

• https://www.hovergames.com/technology
• https://community.nxp.com/t5/Mobile-Robotics-Drones-and/ct-p/mobile-robotics
• https://www.hackster.io/savagecabbage/smart-scan-delivery-drone-ed5c54

Old Contest entries

• https://www.hackster.io/mdobrea/a-hovergames-drone-to-sustain-a-quarantine-zone-23729a
• Highlighted as an excellent example of documenting for last year https://www.hackster.io/scarecrow/sentinel-conservation-via-aerial-reconnaissance-and-escort-1d37ac
• 1st place, Hovergames challenge 2 -https://www.hackster.io/matthew-hurley/connecting-a-raspberry-pi-zero-w-to-the-hovergames-drone-kit-e3acf1

[^1]:(aka Who let this guy design it)

#Update - Oct 23 2022 - Planned use of the Flir module was unavailable due to export problems. However another thermal array was substituted from Seeed Studios.

Progress log as of March 30, 2023

What went well:

• The physical build of the device.
• Learning the NavQPlus system and all the possibilities it opens up.
• I was really happy with the pantograph mechanism I settled on for the lowerable sensor arm. I look forward to testing it.
• Being able to contribute to the Hovergames documentation on Gitbook as clarifications came up.
• The wonderful community on Discord supporting each other's questions.
• I discovered I had a supply of cheap/free 5v Battery Eliminators in the form of old USB car adapters. The voltage regulator IC used can handle 7v to 20+ volts. With some creative modification, most of the weight can be stripped off leaving a small circuit perfect for adding to the drone.
• An associate I know is also a drone pilot doing commercial surveying and is currently being trained on a drone based Ground Penetrating Radar system that opened up some exciting discussions.

What ran into trouble:

• The learning curve of the PX4 FMU was high. There are still some issues with CPU usage I hope to sort out before moving to offboard control with ROS2. We did get it flying though.
• Creating a replacement camera arm that could be mounted to a servo, allowing for forward or downward camera use was developed. This was unnecessary but should improve usefulness of the drone. It was also meant to hold the thermal camera module so it could be used forward looking also.
• Research an 3d design of the sensor arm mechanism was definitely more time consuming that planned.
• Time investigating UAVCAN/Cyphal wasn't immediately useful to this project but should become more relevant later. This is mainly because ROS and Mavlink control both rely on Ethernet or UART for a connection.
• Cold weather put a hold on test flights which slowed momentum on the project.
• Early tests to use the CircuitPython Blinka libraries with the BME688 didn't work. This was because Blinka uses a library to recognize the board in use and that library couldn't identify the NavQPlus. There are instructions for adding to it which are in progress but ran into a bug we haven't resolved
• Thermal camera resolution was not sufficient yet to pick out much detail. While we hope to find other uses for it, a higher resolution image is needed to provide useful data
• Training of the Vision system was in progress however became stalled because collecting the required training images included lots of time reading news reports and Twitter. This definitely impacted other productivity.

What didn't make progress before the deadline

• Magnetometer design is still in the concept phase.
• 3d printing of the first versions of the NavQPlus case, Servo mount for a camera arm, and the sensor arm
• ROS2 and offboard control
• Scripting the data to output into a nice Google Earth heat map as planned.