Hardware components | ||||||
_copy_P5AWOdT4dX.png?auto=compress%2Cformat&w=48&h=48&fit=fill&bg=ffffff) |
| × | 1 | |||
 |
| × | 1 | |||
 |
| × | 1 | |||
 |
| × | 1 | |||
Winning an FPV drone race takes a lot of practice! I've spent lots of time burning batteries on practice tracks, in parks, and simulators like LiftOff (my favorite). To make my practice sessions more effective I wanted to create a tool to count laps, track my fastest times and give feedback.
Not familiar with FPV drone racing? Below is a video from one of my races.
Here is Drone Race Practice Companion in action!
Drone Racing Practice CompanionWalabot, Amazon Alexa, and Raspberry Pi can combine into an intuitive assistant to help a pilot focus on what is important: improving speed and reducing lap times.
Taking advantage of the Walabot’s radar sensing capabilities, the “Drone Race Practice Companion” is positioned at the start finish / gate. Aligned for sensing your drone as it passes through the gate, it is capable of measuring lap times and counting laps.
Wearing first person view (FPV) goggles keeps the pilot’s focus on the drone’s video feed, but inhibits interactions with any other equipment. Without Alexa, checking lap statistics would require the pilot to land and remove their FPV goggles. By integrating an Alexa voice user interface a pilot can ask “How’s my flying?”, “How many laps have I completed?”, or “What was my fastest lap”. The pilot gets real time feedback as they fly while staying focused on flying.
Drone Race Practice Companion: Mode & CapabilitiesDrone Race Practice Companion has 2 operational modes that are configurable with Alexa:
Time Attack: This mode gives the pilot 2 minutes and 30 second to complete as many laps as possible. Total laps completed are counted and best lap time is stored. At the beginning of time attack mode the lap counter and best lap time are both reset.
Free Practice: This mode gives the pilot an unlimited amount of time. Total laps are counted and best lap time is stored. At the beginning of free practice mode the lap counter and best lap time are both reset.
The user can also ask Alexa:
What is my best lap time? Alexa will respond with the best lap time completed in the current or most recent session.
How many laps have I flown? Alexa will respond with the number of laps completed in the current or most recent session.
What can I say? Alexa will respond with all of the potential things you can ask Drone Race Practice Companion to do.
Reset Best Lap Time. Alexa will reset the best lap time.
Reset Lap Counter. Alexa will reset the lap counter.
How is my flying? Alexa will respond with the laps completed and best lap time from the current or most recent session.
I'm done flying. Alexa will end the session and stop the current mode
Drone Race Practice Companion: LimitationsThe Drone Race Practice Companion is designed for a single pilot to use at any time. There is no way for the Walabot sensor to differentiate between different aircraft.
Alexa Skills are session based, and your session ends when you stop talking to Alexa and the blue light turns off. I had initially planned on having Alexa call out laps and lap times automatically when a lap is completed, but that is not possible. Instead I connected a speaker to the Rapberry Pi that plays the start tones, stop tones, and lap completed tones. To get your lap count and best time you have to invoke the Drone Race Practice Companion skill again.
Background: Drone Racing with MultiGPFor a typical race rectangular gates are placed to create a looping race track. Often cones or paint are used on the ground to mark flight path between gates and may indicate which direction a gate should be approached from. Up to 6 pilots might all race simultaneously. The race starts with a series of beeps. Pilots will have a set amount of time to complete as many laps as possible. When time expires you can complete your current lap to have it count towards points. At the end of the day your ranking is determined by the cumulative laps completed during your multiple heat races.
Using an Alexa Skill to interface with hardware on you local network unfortunately still requires you to go out over the Internet and use a cloud service. For this project I've used AWS IoT as the middle middle man between the local Drone Race Practice Companion hardware and the AWS Lambda function that the Alexa Skill talks to.
When a user tells Alexa "Alexa, ask Drone Race Practice Companion to start a Time Attack", a JSON event is sent to the AWS Lambda function that looks like this:
The Lambda function processes the JSON and updates the AWS IoT thing shadow to show that power should be on and in time attack mode.
Meanwhile, the Raspberry Pi with Walabot connected has subscribed to get the "Delta" updates from the thing shadow. When the desired state set by the Lambda function doesn't match the reported state set by the Raspberry Pi, a delta message is sent to the Raspberry Pi telling it to change state.
Hardware: Raspberry Pi + Walabot DetailsHardware-wise this project is pretty simple. Connect a Raspberry Pi 3 up to an Official Raspberry Pi touchscreen, plug in a Walabot creator device and set it up in front of your start / finish gate. That's it!
To protect the Walabot sensor I laser cut a simple top and bottom plate. The files to cut your own are in the project files below. I've also attached my design for a 3D printed Raspberry Pi Touchscreen case.
The Raspberry Pi Code zip file attached to this project has all the code used on the raspberry pi to run the Drone Race Practice Companion. There are detailed comments that should make it easy to read through and understand.
Start by installing the Walabot SDK for Raspberry Pi. Next unzip the Raspberry Pi Code zip file. In a terminal navigate to the folder containing the Raspberry Pi code. Use this command to start Drone Race Practice Companion: "python DRCP_w_Walabot.py"
Software: Raspberry Pi + Walabot: Drone Sensing Algorithm
The Drone Race Practice Companion uses the GetImageEnergy() function from the Walabot API to sense when a drone is flying through the start / finish gate. GetImageEnergy() returns the total reflected energy, which in our case is the amount bouncing back off the drone. In order to turn that into a lap counter a little signal processing is necessary.
The data that comes out of the GetImageEnergy() function is stored in the "RawEnergy" variable. It looks like the red trace above. There are a few issues with "RawEnergy", the first being that before a drone passes over the sensor, the returned data is much higher than it is for the remainder of the practice session. The blue trace shows "datamin", a variable that always contains the lowest value returned from RawEnergy.
"datamin" is subtracted from "RawEnergy" and result is stored in CalEnergy. Now that the first big bump is removed we can focus on the next issue. As time progresses the lowest level reported in CalEnergy slowly creeps up. The detector won't work with a moving target so CalEnergy is passed through a low pass filter. The output is stored in the "lpf_data" variable and is in blue above.
"Total Energy" is created by subtracting "lpf_data" from "CalEnergy". Now there is a well behaved signal to run the detector on! You can see lap count in green above. It is properly triggered at each spike in TotalEnergy from a drone passing over the sensor.
You can find all this processing in the "loop():" function of the Raspberry Pi code, DRCP_w_Walabot.py
Software: Lambda FunctionThe Lambda Function zip file attached to this project has all the code that runs the Drone Race Practice Companion Lambda function. Have a look at lambda.py, this is where all the action happens. When the Drone Race Practice Companion skill is invoked, the "lambda_handler()" function is called and passed a JSON file. From here the JSON is parsed.
If session.new = true then a connection is made to the AWS IoT thing shadow.
Next it branches to handle either a "LaunchRequest", "IntentRequest" or "SessionEndedRequest". A "LaunchRequest" happens when a user asks Alexa "Start Drone Race Practice Companion". The skill responds with a welcome message and asks what the user would like to do. An "IntentRequest" skips the welcome message and goes straight to what the user asked for such as "ask Drone Race Practice Companion to start a Time Attack". In this case the code in "TimeAttack_response()" runs which updates the thing shadow to turn on the power, set the LapCount to zero, set the BestTime to 120 seconds, and set the Mode to "TimeAttack".
A "SessionEndedRequest" does just that; it ends the race, stops the connection with the thing shadow, and tells the user that the session is ended.
Software: Alexa Skills KitThe skill that you make in the Alexa Skills Kit is really there just to connect utterances that a user will say to the "intents" that get sent to the Lambda function as JSON. Additionally it sets the location of the Lambda function that the JSON file gets sent to.
The sample utterances that Drone Race Practice Companion uses are in the Alexa Skill Utterances file attached to this project
Setup:Lambda Function, Alexa Skills Kit, and AWS IoT setup
I have to give credit to this hackster.io project: Coffee Machine: Amazon Alexa & Raspberry Pi. I learned a ton by reading through Bastiaan's work and based my Alexa Skill on the framework of his project. Instead of recreating Bastiaan's setup instruction head over to his project and start reading at Software: Amazon Setup
The Lambda Function is written in Python 2.7 which doesn't seem to be very common. However, it makes a lot of sense for a Raspberry Pi based Alexa skill as all the Raspberry Pi coding is also Python 2.7. This allows you to use the same libraries on both platforms to communicate with the AWS IoT thing shadow.
Alexa Skill: Drone Race Practice CompanionThe Drone Race Practice Companion Alexa Skill is now live in the Skill Marketplace! You can find it by searching for "Drone Race Practice Companion" or at this link: https://alexa.amazon.com/spa/index.html#skills/dp/B078P3Z59T
Drone Race Practice Companion Connection Diagram
Comments