HoverLive

Motivation

Most focus on a specific application to be carried out by our drones. Currently, there is an inherent gap between the underlying software architecture and the associated application of the drone. Our solution tries to reduce this gap as little as possible by defining a common framework that incorporates all the software elements with the hardware, to produce a simplified flight management application for autonomous missions. This could solve the problem of first responders and emergency and medical personnel to immediately plan and execute a mission with little to no effort.

Aim

Our solution develops a high-level framework, coined as "HoverLive" that incorporates the NXP FMU, NavQ companion computer, and the coral camera, into a common framework based on the PX4 flight stack, MAVSDK, ROS, and additional software elements (such as OpenCV, TF Lite, etc) that can be easily deployed into their Hovergames drone for easy management of autonomous missions via a dedicated application. The capabilities of the framework include autonomous takeoff, simplified waypoint navigation with active obstacle avoidance, and vision-based autonomous landing.

This is particularly helpful because it allows the user to quickly deploy a drone in case of an emergency such as during the ongoing pandemic situation.

Use-cases

Scenario 1 >> Delivering a sensitive package from the source to the requested destination, where it lands autonomously using marker detection and tracking.

Scenario 2 >> Emergency response capability where the drone needs to be deployed to the destination without any delay, once a panic/request is triggered.

Workflow

The framework revolves around the reference design set by the NXP FMU and its capabilities. The solution runs on the NavQ board which will manage the FMU via UART connection. In the extended version, it also manages the coral camera feed for visual inspection and marker-based autonomous landing, along with any additional hardware required.

Setting up NavQ

The 8MMNavQ is a small purpose-built experimental Linux computer based on the NXP i.MX 8M Mini SOC. It is focused on the common needs of Mobile Robotics systems. The system is built as a stack of boards, the top board is a SOM (system on module) containing the Processor, memory, and other components with strict layout requirements, and where the secondary boards are relatively inexpensive (often 4 layer boards) and allows for versions with customization to be easily built.

• The SD card included in the NavQ kit is preloaded with our HoverGames-Demo Linux distribution. The default username and password are:username: navq | password: navq
• Power on the NavQ using the included USB-C cable.
• Connect the Serial USB adapter to the UART2 port on the bottom port and connect to the laptop USB.
• Default NavQ terminal settings are: 115200 Baud, N, 8, 1 (no Parity, no flow control)

$ dmseg | tail # to check if USB serial connection established and port assigned.

$ minicom -D /dev/ttyUSB0 -b 115200 # connect to NavQ serial console.

Either u will get a login prompt. Else press enter.

Log in using navq:navq as the credentials.

• When the NavQ arrives, the Demo image will already be loaded to the SD card. This image does not take up the full amount of space on the SD card, so you'll need to expand the space in order to install more packages such as ROS or OpenCV. Follow the steps below to expand the filesystem.Download and run the following script resizeDisk.sh.

$ chmod a+x ./resizeDisk.sh
$ sudo ./resizeDisk.sh sd      # For SD card
$ sudo ./resizeDisk.sh eMMC    # For eMMC

• NavQ can be connected to the internet directly via an ethernet connection or via WiFi. To connect to WiFi, follow the steps below:A package named connman is included in the image to help you connect to WiFi through the command line. To connect to WiFi, run the following commands:

$ connmanctl
connmanctl> enable wifi
connmanctl> scan wifi
connmanctl> services
WIFI_SSID     wifi_name_managed_psk
connmanctl> agent on
connmanctl> connect wifi_name_managed_psk
<enter passphrase>
<wait for connection success message>
connmanctl> exit

• Check for a successful connection

$ ping google.com

• It is recommended to create a unique hostname to connect to the NavQ instead of using an IP address all the time.To change the hostname we need to modify /etc/hostname.

$ sudo echo navq > /etc/hostname

• For remote access, the navq needs to be connected to SSH.Follow the steps below.openssh-server is pre-installed.Find IP of navq or assign a static one in your router

$ ssh navq@<ip>

Type yes and this opens the terminal of navq on our laptop.

If you want to set up a desktop environment for navq, follow the guide here. Now the NavQ is setup for HoverLive.

HoverLive: Setting up

HoverLive is a high-level framework, that incorporates the NXP FMU, NavQ companion computer, and the coral camera, into a common framework based on the PX4 flight stack, MAVSDK, ROS, and additional software elements (such as OpenCV, TF Lite, etc) that can be easily deployed into their Hovergames drone for easy management of autonomous missions via a dedicated application. It gives complete freedom for the users to use their choice of drone API (currently supporting MAVSDK and MAVROS) in their choice of programming language (currently supporting C++ and Python). In the case of hardware-dependent parts, they are implemented as add-on modules for isolating the core package.

The capabilities of the framework include autonomous takeoff, simplified waypoint navigation with active obstacle avoidance, and vision-based autonomous landing.

• SSH into the navq and clone the HoverLive repository.

$ git clone https://github.com/crisdeodates/Hovergames-HoverLive.git
$ git checkout master

• Build the project

$ cd Hovergames-HoverLive/
$ catkin init
$ mkdir build
$ cd build/
$ cmake ..
$ make

Now that the project is set up, let's go through the structure of HoverLive and how it can be used in our projects.

HoverLive: Dependencies

Currently, HoverLive tries to implement high-level wrappers for MAVSDK and MAVROS. So both are dependencies for the package. In the case of firmware and SITL, PX4 is also a requirement. In the case of hardware-dependent parts, they are implemented as add-on modules for isolating the core package.

• Setting up PX4
• Setting up MAVSDK
• Setting up MAVROS

HoverLive: Walkthrough

HoverLive is structured as follows:

The API and Environment layers are extensive and could be expanded to include more drone APIs and environments in the future. This allows a plausible concept of expandability to the overall package. The shortcoming will be increased dependency on each of the extended APIs, although which cannot be neglected if integrated.

The basic implementation of HoverLive is as follows:

• Here the basic implementation revolves around abstracting the core API features in such a way that it won't be ambiguous to the user.
• The core features such as Arming the drone, Disarming, taking off, Landing, etc are extracted to user callable states to which the HoverDrone instance transits.

The main advantage is the ease of structured state callbacks that allows uniform implementation throughout each API and environment.

This implementation is inspired by the MAV fleet control by Julian Blanco.

HoverLive: Usage

Integrating and using HoverLive into a new or current project is quite easy.

Let's take the case of MAVSDK Python implementation of HoveLive.

• Import the HoverLive packages

$ from HoverDrone import Drone

• Import the required states

from HoverDrone.states import Arm
from HoverDrone.states import Disarm
from HoverDrone.states import Takeoff
from HoverDrone.states import Land

• Connect to the HoverDrone instance

my_drone = Drone.connect("myDrone", "udp://:14540")

• Configure and add each required states as individual threads

my_drone.start()
my_drone.add_state(Arm)
my_drone.add_state(Disarm)
my_drone.add_state(Takeoff(5.0))
my_drone.add_state(land)

• Execute the threads

my_drone.join()

IMPORTANT: Some features may not work as the package is still in development and modifications might be done without notice that may change the current implementation and topology.

HoverLive: Testing

A simple SITL mission with takeoff, Position request, and a land scenario is tested using the HoverLive package implemented on NavQ.

Due to the restriction of flying the hardware drone, a PX4 SITL with JMAVSim is run on the NavQ. Also it should be noted that in this season of Hovergames, the focus of the competition is based in NavQ and not Hovergames drone kit.

Future Modifications

The capability of HoverLive is vast. With the help of my contributors, we wish to extend the capability of the HoverLive beyond the current APIs and make it a uniform universal framework for drones.

Since the current implementation is not hardware-dependent, it can be applied to a wide range of hardware that supports the APIs and their subsets. Since the hardware-dependent parts are incorporated as add-on modules, it makes the core package scalable.

Contributing Projects:

• HoverDock
• HoverDrone