Scooter Rental

As many of us have seen, the market for rentable scooters has exploded over the last few years in major cities around the world. While we've seen articles about what hardware runs inside of these scooters, I thought it'd be fun to try and replicate them with hardware that I had laying around.

My project hardware consists of:

• a Particle Electron (cellular connected) board, since it was mentioned in the original article and makes sense for use out in the world
• a standalone GPS module for more accurate location tracking
• a relay for turning the scooter on and off
• a switch for turning the control device on and off
• a small battery to run the hardware
• a Segway for use as the scooter.

As for the software side, I decided to use:

• Google Cloud: this allowed me to use pub/sub to send data anywhere within the Google system easily, which made sense for later expansion, and was easy to integrate with the Particle board
• Firebase for real time data and backend support
• Android for the mobile app controller

In the end, I have an app that could read a QR code specifically for the rental scooter, which then talks to Firebase and turns on the scooter. The scooter itself then reports its location periodically, though I'm not currently doing anything with that data, as that process is similar to another project I've put together.

What This Article Is and What It Isn't

This project is a proof-of-concept for activating a scooter remotely and tracking its location. The main goal is to showcase some cool ways of integrating various technologies together. As I move forward through this project, I'll try to keep it structured from the perspective that you, the reader, are following along in creating this same device.

That said, I did take a few shortcuts with data, and it falls short of being a complete product. Entire companies exist around this complete idea, and implementing every detail would be a company scale endeavor. I will try to point out areas where I stubbed or skipped functionality, as others can fill in those blanks if they really want to.

Voiding the Warranty

To get the scooter to turn on remotely, I decided that the easiest way would be to bypass the power button that would normally start it. The only way I could think of to get to that power button was to break a bunch of warranty voiding stickers by disassembling the scooter, but that's half the fun :)

Fair warning: there's also a bunch of stickers about components holding a lot of electrical charge, plus it's got a pretty serious battery. This may be dangerous and I'm sure I'm obligated to say that if you do open up a hardware device, you continue at your own risk. Also, opening the device automatically voids any warranties that may have applied, so that's something to be mindful of.

I started by removing the battery from the bottom of the device to expose the main circuit board.

Luckily, there's only one connector here that actually needs to be removed (though I definitely removed all of them while figuring this out). That connector ties the board to the center console with the power button.

Now you may be thinking "Wait, couldn't you just jump that connector and not spend a bunch of time disassembling this entire thing?", and the answer is probably "yes". Guess who didn't think about that while pulling everything apart? Oh well.

After unscrewing a bunch of things, I was able to remove the panel on the front that keeps the center console in place. Next I needed to remove the foot pads and top frame.

After removing the screws that are under the foot pad, the center console was able to come off of the scooter.

On the side of this center piece is a little rubber screw cover. I popped this out with a screwdriver and removed the remaining screws.

Finally, with the power button exposed. I soldered two long wires to the prongs with traces that looked like they'd create the starting connection.

After reassembling the scooter, a quick tap of these wires and we're off to the races.

Creating the Control Device

Now that the scooter turns on externally, it's time to actually do something with it. This part list can be found at the top of this article, though I'll reiterate here:

• Particle Electron
• GPS module
• Relay
• Small LiPo battery pack
• Switch
• Antennas for the Particle board and the GPS
• Breadboard and wires
• Female headers for inserting components into the breadboard

After some placing and soldering, we get a little something like this:

This device attaches to the scooter by putting the wires that were connected to the power button on the scooter into the relay.

While this is fine and working, it's not very practical. The next step was to design a case that could be attached to the scooter. Since I happened to have access to laser cutter, this seemed like the perfect excuse to use it with some quarter inch clear acrylic.

You'll notice a few holes added to the case, as well. These are used for mounting on a screw on the scooter, as well as antennas and the switch.

Side note about lasers: they're awesome.

Once that's done, we've got a nice set of cutouts that can be put together into a box.

After assembling, gluing everything (except the top - you still need to be able to charge the battery once in a while) and attaching the GPS and cellular antenna, we've almost got our completed hardware setup.

I also cut the wires between the controller and the scooter so I could add banana connectors, making it so I could remove the controller without a lot of hassle.

Finally, I printed off a QR code with the IMEI for the Particle board (it's printed on a sticker on the board), which we'll scan later to remotely start the device.

Firebase + Google Cloud Backend

While we could technically communicate directly from the mobile app to the scooter hardware, having an intermediary means we can edit our logic from a central point without updating our scooter's firmware or the mobile app, and it provides a lot of room for scalability/growth later on. For this project, I chose to use Google Cloud with Firebase, as Firebase gives me a lot of easy to use features, and I'm already super familiar with it (give that I work on that team :p).

Before we can start setting up our backend, we will need to create accounts and projects with Google Cloud, Firebase, and Particle.io.

Rather than reiterating what Particle has already put together here, you'll want to visit Particle's documentation on integrating with Google in order to create your Google Cloud infrastructure. At the end of that tutorial, you should have a Pub/Sub event created that you can then push data through (mine is called scooter-device-state). With that, it's time to dig into the Firebase setup.

The first thing I did after creating my Firebase project was upgrade to a Blaze (pay-as-you-go) plan. While I don't expect to go over the free tier cap, this was necessary for making network calls outside of the Google ecosystem. Secondly, I created a new Android app within Firebase with the package name com.scooterrental.demo.

Next, it's time to set up a database. I ended up using Firestore, as I wanted to organize my data into collections that could be queried easily if this project were to scale up. During the setup process for Firestore, you'll be asked if you want to continue in production mode or test mode. Production mode simply means that there's a security rule in place that requires users to be authenticated before they can read or write from the database, so I went with that. On the next step you'll be asked to pick a region for your database - simply pick the closest one to you.

While we'll get into detail on how our data is used soon, the general idea is that this database will have three top-level collections:

• devices
• users
• trips

In a production app, I would keep track of device state and location with the devices collection, a list of our active users and any relevant information for them in the users collection, and finally, which users used which devices and where they started and ended their trip via the trips collection. You can see an example of the dummy data I placed for testing here, though the app can be modified to use correct values relatively simply later. For testing purposes, setting the location to a point near your physical location will help you see the scooter show up as a pin in the mobile app later:

The next step was to set up authentication for our Firebase project. To keep things simple, I only used Google authentication, though you could add your own email and password authentication system, or rely on additional identification providers, such as Twitter, Facebook or Apple. You can enable this in the Authentication part of the Firebase console under the sign-inmethod tab.

Finally, it's time to add the Firebase Functions that will be used for handling our business logic. I'll set this up from the perspective of using an OSX device, though other operating systems will be similar.

To start, open a terminal and navigate to a directory where you would want to host your Firebase code. Once there, run the firebaselogin command to log into your email that's associated with your Firebase project. At this point your browser should open for you to select your email, and you will be presented with a consent screen.

After logging in, it's time to run the firebaseinit command. For now we'll only need to enable Firebase Functions.

Continue answering the prompts to set up the CLI with your Firebase project, select JavaScript as the language used by Functions, and if you want to use ESLint for catching bugs and enforcing styles, and finally install any required dependencies.

At this point you should have a folder named functions in your current directory, as well as a firebase.json file.

If you go into the functions directory, you'll find a file called index.js that you will want to open. This is where all of our Firebase Functions code will go.

We can start by adding all of the necessary global values at the top of the file:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
var Particle = require('particle-api-js');
var particle = new Particle();
const token = '*********';

admin.initializeApp();
const db = admin.firestore();

You'll notice that a token is being used here. This token is used to make API calls to Particle from Firebase Functions, and can be created from the Particle.io console by following these instructions.

Next there are two Functions that will be used. The first turns a scooter on or off when called by receiving the code for the device that we want to toggle, and then using the Particle library to call the toggleState method on that specific physical device.

exports.rentDevice = functions.https.onCall((data, context) => {
    const scooterId = data.code;
    return new Promise((resolve, reject) => {
        var fnPr = particle.callFunction({ deviceId: scooterId, name: 'toggleState', argument: 'on', auth: token });

        fnPr.then(
            function(data) {
                console.log('Function called successfully:', data);
            }, function(err) {
                console.log('An error occurred:', err);
            });
        });
});

The second method simply listens for any data coming through on our pre-defined Pub/Sub and responds to it. In this case I'm just logging the data to the console, though in reality we would want to update the Firestore data that we created earlier.

exports.updateDeviceData = functions.pubsub
    .topic('scooter-device-state')
    .onPublish((data, context) => {
        const tag = Buffer.from(data.data, 'base64').toString();
        console.log(tag);

    return Promise.all([
        true
    ]);
});

While I kept these examples fairly simple, they can definitely be expanded to do a lot more as need arises or the project needs to become more complex.

It's worth noting that you may need to run some additional commands during these steps to bring in node dependencies. If that situation comes up, you'll be prompted in the CLI, so moving forward shouldn't be too difficult.

Enabling Google Maps

Before we can use Google Maps in the mobile app, we will need to enable the API. You can do this by going into the Google Developers Console, selecting your project, clicking on ENABLE APIS AND SERVICES at the top of the screen

searching for Google Maps, selecting the option for the Android SDK,

and finally, clicking the enable button.

Once the API is enabled, we will also need to create an API key that will be used in the mobile app. You can do this by opening the side menu in the developer console, selecting credentials

selecting CREATE CREDENTIALS, and then API key

This will bring up a window containing your new API key that can be used with maps. It's important that you keep this secret so others don't use your key (I already deleted the one in this picture :))

Particle On-Device Code

The easiest way to publish code to a Particle device is through their Web IDE. While you can push directly to the device, it's actually recommended that you download your compiled code and flash it over USB using Particle's CLI tool when using the Particle Electron, otherwise you'll use up data that has to be paid for, though covering that is beyond the scope of this project.

As with most code cases, we will start with our imports and globals:

#include <Adafruit_GPS.h>
#define GPSSerial Serial1
#define GPSECHO false

Adafruit_GPS GPS(&GPSSerial);

static const int device = D7;
uint32_t timer = millis();

Pin D7 (named device) is used for controlling the relay that turns the scooter on and off, the GPS object handles communication between the Particle board and the GPS unit, and timer is used to make sure we don't spam location information back to our Cloud backend (which would eat up the data allowance on the Particle board pretty quickly).

Next up is the setup() method, which is used for setting initial states for the components on our device. One interesting thing to watch here is the Particle.function() method, as this is what allows a REST call to happen directly to the device at the toggleState endpoint. You'll notice that we make a call to that endpoint in our Firebase Function in the last section.

void setup() {
    Serial.begin(115200);
    pinMode(device, OUTPUT);
    digitalWrite(device, LOW);
    Particle.function("toggleState", toggleState);
    GPS.begin(9600);
    GPS.sendCommand(PMTK_SET_NMEA_OUTPUT_RMCONLY);
    GPS.sendCommand(PMTK_SET_NMEA_UPDATE_1HZ);
    Serial.print("setup done\n");
    delay(1000);
}

When the toggleState endpoint is called, the toggleState method will be triggered. This method will set the relay to connect the two wires coming out of the scooter, and then turn it off again. This will simulate pressing the power button on the scooter.

int toggleState(String state) {
    digitalWrite(device, HIGH);
    delay(300);
    digitalWrite(device, LOW);
    return 1;
}

The last bit of code running on the Particle device starts in the main loop() method, and pulls the location from the GPS unit. If enough time has passed, the information from the GPS unit will be sent to Firebase through the Google Cloud integration and Pub/Sub.

void loop() // run over and over again
{
    char c = GPS.read();
    if (GPSECHO)
        if (c) Serial.print(c);
        if (GPS.newNMEAreceived()) {
            Serial.println(GPS.lastNMEA());
            if (!GPS.parse(GPS.lastNMEA())) {
                Serial.println("cant parse last NMEA");
                return;
            }
        }

        if (timer > millis()) timer = millis();

        if (millis() - timer > 300000) {
            timer = millis();
            if (GPS.fix) {
                publishLocation();
            } else {
                Serial.println("No GPS fix");
            }
        }
    }

    double convertDegMinToDecDeg (float degMin) {
    double min = 0.0;
    double decDeg = 0.0;
    min = fmod((double)degMin, 100.0);
    degMin = (int) ( degMin / 100 );
    decDeg = degMin + ( min / 60 );

    return decDeg;
}

void publishLocation() {
    Serial.println("publishLocation");
    String lat = "";
    String lng = "";
    if( GPS.lat == 'S' ) {
        lat = "-";
    }

    lat = lat + convertDegMinToDecDeg(GPS.latitude);

    if( GPS.lon == 'W' ) {
        lng = "-";
    }

    lng = lng + convertDegMinToDecDeg(GPS.longitude);
    String data = "{ 'deviceId': '" + System.deviceID() + "', 'lat': '" + lat + "', 'lng': '" + lng + "' }";
    Particle.publish("scooter-device-state", data, PRIVATE);
}

Android Mobile App

To wrap up this project, we need to make a mobile app that can activate the scooter. We'll start with a stock Android app, and then build from there. You may remember that I used the package name com.scooterrental.demo earlier in this project when setting up Firebase. You will want to make sure that your Android package name matches up to the one you declared in the Firebase console. Once the app base Android app is created, we set up Firebase within the app by following these instructions.

We will also need to add our dependencies to the Android app by going into the app level build.gradle file and placing the following lines into the dependencies node:

implementation 'com.google.android.gms:play-services-maps:17.0.0'
implementation 'com.google.android.gms:play-services-location:17.0.0'

implementation 'com.google.firebase:firebase-core:17.2.1'
implementation 'com.google.firebase:firebase-ml-vision:24.0.1'
implementation 'com.google.firebase:firebase-firestore:21.3.0'
implementation 'com.google.firebase:firebase-ml-vision-image-label-model:19.0.0'
implementation 'com.google.firebase:firebase-functions:19.0.1'
implementation 'com.firebaseui:firebase-ui-auth:5.1.0'

This will bring in Google Maps, which will be used to display location information for our rental scooters, ML Kit for reading QR codes, Firestore for accessing our cloud data, auth (using the Firebase-UI library) for signing in users, and functions for calling the code that turns the scooter on and off.

We will also need to get into the AndroidManifest.xml file and add the necessary permissions and metadata for the app to use MLKit and Google Maps:

<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.INTERNET" />

...

<meta-data
    android:name="com.google.firebase.ml.vision.DEPENDENCIES"
    android:value="barcode,label" />

<meta-data
    android:name="com.google.android.geo.API_KEY"
    android:value="@string/google_maps_key" />

You may notice the google_maps_key strings value. This is the API key that we created earlier for Google Maps, and we'll set up that string value soon.

Next, let's create our layout file. Open activity_main.xml and replace the contents with the following code:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">
    <fragment
        xmlns:tools="http://schemas.android.com/tools"
        android:id="@+id/map"
        android:name="com.scooterrental.demo.ScooterMapsFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        tools:context=".MapsActivity" />

    <com.google.android.material.button.MaterialButton
        android:id="@+id/scan_scooter"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:minHeight="@dimen/locate_device_min_height"
        android:layout_alignParentBottom="true"
        android:layout_marginLeft="@dimen/locate_device_margin_left"
        android:layout_marginRight="@dimen/locate_device_margin_right"
        android:layout_marginBottom="@dimen/locate_device_margin_bottom"
        android:textSize="@dimen/locate_device_text_size"
        android:text="@string/start_scooter_text"/>
</RelativeLayout>

You will also want to set up your strings.xml file like so, replacing the google_maps_key with your API key:

<resources>
    <string name="app_name">Scoooter Rental</string>
    <string name="title_activity_maps">Scooter Rental</string>
    <string name="start_scooter_text">Scan Scooter</string>
    <string name="google_maps_key">****</string>
</resources>

To wrap up our Android app, we have two key classes: MainActivity.kt and ScooterMapsFragment.kt. MainActivity will be responsible for starting the camera, reading the QR code, and contacting the Firebase Function for activating the scooter. ScooterMapFragment will be responsible for anything related to controlling the Google Map, retrieving data from Firestore, and authorizing our user through Firebase.

Map Fragment

We'll start by creating the class ScooterMapsFragment. Once it's created, the class definition needs to be updated with the proper parent class and interfaces.

class ScooterMapsFragment : SupportMapFragment(), OnMapReadyCallback,
    GoogleMap.OnMyLocationButtonClickListener,
    GoogleMap.OnMyLocationClickListener {

We will also add our global values at the top of the class

private lateinit var fusedLocationClient: FusedLocationProviderClient
private lateinit var locationCallback: LocationCallback

private lateinit var mainMap: GoogleMap
private lateinit var firestoreDB : FirebaseFirestore
private lateinit var var auth : FirebaseAuth

private val RC_SIGN_IN = 123
private val LOCATION_PERMISSION = 42


private var scooters: ArrayList<Scooter> = ArrayList<Scooter>()
private var markers: ArrayList<Marker> = ArrayList<Marker>()

Outside of the class, you will need a Scooter data class to represent the data object coming down from Firestore.

data class Scooter(val location: GeoPoint, var id : String, var available: Boolean) {
    constructor() : this(GeoPoint(0.0, 0.0), "", false)

}

The onAttach() method is the entry point for our fragment. This is where we will initialize our fusedLocationClient,auth, and firestoreDB objects, retrieve our map if the user is logged in, or request that the user log in.

override fun onAttach(context: Context) {
    super.onAttach(context)


    fusedLocationClient = LocationServices.getFusedLocationProviderClient(requireActivity())
    auth = FirebaseAuth.getInstance()
    firestoreDB = FirebaseFirestore.getInstance()

    if( auth.getCurrentUser() != null ) {
        getMapAsync(this)
    } else {
        startActivityForResult(
            AuthUI.getInstance().createSignInIntentBuilder()
                .setAvailableProviders(
                    Arrays.asList(
                        AuthUI.IdpConfig.GoogleBuilder().build() ) )
                .build(), RC_SIGN_IN)
    }
}

If the user isn't logged in, a new Activity will be started prompting the user to log in using a Google account.

Once the user signs in, onActivityResult() will be called. This method will check to see if it was called because the user signed in, or because location permissions were granted (we'll get into this triggering case soon). If the user signed in, then their authorization information will be saved and the GoogleMap object will be retrieved.

override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)
    if (requestCode == RC_SIGN_IN) {
        if( resultCode == Activity.RESULT_OK) {
            auth = FirebaseAuth.getInstance()
            Toast.makeText(requireActivity(), "Welcome", Toast.LENGTH_SHORT).show()
            val bundle = Bundle()
            bundle.putString(FirebaseAnalytics.Param.METHOD, "FirebaseUI - Google")
            getMapAsync(this)
        } else {
            Toast.makeText(requireActivity(), "Login failed", Toast.LENGTH_SHORT).show()
            requireActivity().finish()
        }
    } else if( requestCode == LOCATION_PERMISSION && resultCode == Activity.RESULT_OK ) {
        initMap()
    }
}

Once the map is retrieved, we will pull down any pins we have available for our scooters from Firestore, and display them on the map. You may remember that I used a dummy lat/long in Firestore earlier - this is when that is used.

override fun onMapReady(googleMap: GoogleMap) {
    mainMap = googleMap

    firestoreDB.collection("scooters")
        .get()
        .addOnSuccessListener { documents ->
            scooters.clear()
            for (document in documents) {
                Log.e("test", "${document.id} => ${document.data}")
                scooters.add((Scooter(document.data.get("location") as GeoPoint, document.id, document.get("available") as Boolean)))
            }

            requireActivity().runOnUiThread({ populateMarkers() })
        }
        .addOnFailureListener { exception ->
            Log.e("test", "Error getting documents: ", exception)
        }

    if (ContextCompat.checkSelfPermission(requireActivity(), Manifest.permission.ACCESS_FINE_LOCATION)
        == PackageManager.PERMISSION_GRANTED) {
        initMap()
    } else {
        ActivityCompat.requestPermissions(requireActivity(),
            arrayOf(Manifest.permission.ACCESS_FINE_LOCATION),
            LOCATION_PERMISSION)
    }
}

private fun populateMarkers() {
    Log.e("Test", "populate markers")
    mainMap.clear()
    markers.clear()
    for( scooter in scooters ) {
        val marker = MarkerOptions().position(LatLng(scooter.location.latitude, scooter.location.longitude)).title("Available Scooter")
        markers.add(mainMap.addMarker(marker))
    }
}

We will also see if the user has previously granted the ACCESS_FINE_LOCATION permission. If they have, we'll call initMap() to display the user's location, otherwise we'll request that permission and call initMap() from the onActivityResult() method once it's granted.

private fun initMap() {
    mainMap.setMyLocationEnabled(true)
    mainMap.setOnMyLocationButtonClickListener(this)
    mainMap.setOnMyLocationClickListener(this)

    mainMap.setOnInfoWindowClickListener {
        val gmmIntentUri = Uri.parse("google.navigation:q=" + it!!.position.latitude + "," + it.position.longitude + "&mode=w")
        val mapIntent = Intent(Intent.ACTION_VIEW, gmmIntentUri)
        mapIntent.setPackage("com.google.android.apps.maps")
        startActivity(mapIntent)
    }

    fusedLocationClient.lastLocation
        .addOnSuccessListener { location : Location? ->
            updateMapLocation(location)
        }

    initLocationTracking()

}

private fun initLocationTracking() {
    locationCallback = object : LocationCallback() {
        override fun onLocationResult(locationResult: LocationResult?) {
            locationResult ?: return
            for (location in locationResult.locations){
                updateMapLocation(location)
            }
        }
    }

    fusedLocationClient.requestLocationUpdates(
        LocationRequest(),
        locationCallback,
        null)
}

private fun updateMapLocation(location: Location?) {
    mainMap.moveCamera(
        CameraUpdateFactory.newLatLng(
            LatLng(
                location?.latitude ?: 0.0,
                location?.longitude ?: 0.0)
        ))

    mainMap.moveCamera(CameraUpdateFactory.zoomTo(15.0f))
}

Finally, we'll include a few small methods for properly destroying our fusedLocationClient, as well as responding to some maps UI buttons that can be fleshed out later if you choose to.

override fun onDestroy() {
    super.onDestroy()
    fusedLocationClient.removeLocationUpdates(locationCallback)
}

override fun onMyLocationButtonClick(): Boolean {
    Toast.makeText(requireActivity(), "Location Button Clicked", Toast.LENGTH_SHORT).show()
    return false
}

override fun onMyLocationClick(p0: Location) {
    Toast.makeText(requireActivity(), "Location Button Clicked", Toast.LENGTH_SHORT).show()
}

MainActivity

With that done, we can jump over to the MainActivity class. The first thing we will do is declare our globally scoped values at the top of the class:

private val REQUEST_IMAGE_CAPTURE = 1
private lateinit var functions: FirebaseFunctions
private lateinit var mapFragment: ScooterMapsFragment

You'll notice that we have the REQUEST_IMAGE_CAPTURE value. This will be used by the camera in a similar flow as the location permission and authorization features in the map fragment.

Next we'll flesh out the onCreate() method. This will set our layout file that we created earlier, initialize the map fragment and add a click listener to the button we defined in our layout.

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_maps)

    mapFragment = supportFragmentManager
        .findFragmentById(R.id.map) as ScooterMapsFragment

    scan_scooter.setOnClickListener { _ ->
        startScanningActivity()
    }
}

The startScanningActivity() method will launch an Intent for the on device camera, allowing the user to take a picture and send it back to our mobile app.

private fun startScanningActivity() {
    Intent(MediaStore.ACTION_IMAGE_CAPTURE).also { takePictureIntent ->
        takePictureIntent.resolveActivity(packageManager)?.also {
            startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE)
        }
    }
}

This picture will be sent to the onActivityResult() method, where the photo will be rotated to align with the QR code model in MLKit, then the value for that particular scooter will be extracted.

override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)

    if( requestCode == REQUEST_IMAGE_CAPTURE && resultCode == Activity.RESULT_OK ) {
        Log.e("Test", "its an image")
        if( data == null || data!!.extras == null || data!!.extras!!.get("data") == null ) {

        } else {
            val photo = data?.extras?.get("data") as Bitmap

            var rotation = 0

            try {
                rotation = getRotationCompensation(this)
            } catch (e: CameraAccessException) {
                Log.e("Machine Learning", "camera access exception")
            }

            val matrix = Matrix()
            matrix.postRotate(rotation.toFloat())
            val rotatedPhoto =
                    Bitmap.createBitmap(photo, 0, 0, photo.width, photo.height, matrix, true)

            val firebaseVisionImage = FirebaseVisionImage.fromBitmap(rotatedPhoto)

            val options = FirebaseVisionBarcodeDetectorOptions.Builder()
                    .setBarcodeFormats(FirebaseVisionBarcode.FORMAT_QR_CODE)
                    .build()

            val detector = FirebaseVision.getInstance().getVisionBarcodeDetector(options)

            detector.detectInImage(firebaseVisionImage)
                    .addOnSuccessListener { firebaseVisionBarcodes ->
                        Log.e("Test", "ran through detect in image")
                        if (firebaseVisionBarcodes.size > 0) {
                            Log.e("Test", "we got a barcode")
                            processBarcode(firebaseVisionBarcodes.get(0).displayValue)
                        }
                    }
        }
    }
}

To separate our code a bit, the rotation calculation has been moved into its own method:

@Throws(CameraAccessException::class)
protected fun getRotationCompensation(activity: Activity): Int {

    val manager = activity.getSystemService(Context.CAMERA_SERVICE) as CameraManager

    var camIds: Array<String>? = null

    try {
        camIds = manager.cameraIdList
    } catch (e: CameraAccessException) {
        Log.w("Machine Learning", "Cannot get the list of available cameras", e)
    }

    if (camIds == null || camIds.size < 1) {
        Log.d("Machine Learning", "No cameras found")

        return 0
    }

    val deviceRotation = activity.windowManager.defaultDisplay.rotation
    var rotationCompensation = ORIENTATIONS.get(deviceRotation)

    val cameraManager = activity.getSystemService(Context.CAMERA_SERVICE) as CameraManager
    val sensorOrientation = cameraManager
        .getCameraCharacteristics(camIds[0])
        .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360

    val result: Int
    when (rotationCompensation) {
        0 -> result = FirebaseVisionImageMetadata.ROTATION_0
        90 -> result = FirebaseVisionImageMetadata.ROTATION_90
        180 -> result = FirebaseVisionImageMetadata.ROTATION_180
        270 -> result = FirebaseVisionImageMetadata.ROTATION_270
        else -> {
            result = FirebaseVisionImageMetadata.ROTATION_0
            Log.e("Machine Learning", "Bad rotation value: $rotationCompensation")
        }
    }
    return result
}

companion object {

    private val ORIENTATIONS = SparseIntArray()

    init {
        ORIENTATIONS.append(Surface.ROTATION_0, 90)
        ORIENTATIONS.append(Surface.ROTATION_90, 0)
        ORIENTATIONS.append(Surface.ROTATION_180, 270)
        ORIENTATIONS.append(Surface.ROTATION_270, 180)
    }
}

as well as the code to handle the information from the QR code:

private fun processBarcode(barcode: String?) {
    if( barcode == null ) {
        return
    }

    val builder = AlertDialog.Builder(this, android.R.style.Theme_Material_Dialog_NoActionBar)
    builder.setTitle("Rent Scooter")
    builder.setMessage("You are about to rent this scooter. It will cost $2.00 + $0.20 per minute of use. Are you sure?")
    builder.setPositiveButton("Rent", object: DialogInterface.OnClickListener {
        override fun onClick(p0: DialogInterface?, p1: Int) {
            Toast.makeText(this@MainActivity, "Barcode: " + barcode, Toast.LENGTH_SHORT).show()

            rentScooter(barcode)
            p0?.dismiss()
        }
    })
    builder.setNegativeButton("Cancel", object: DialogInterface.OnClickListener {
        override fun onClick(p0: DialogInterface?, p1: Int) {
            p0?.dismiss()
        }
    })

    val dialog = builder.create()
    dialog.show()

}

Once we have the value from the code, a dialog is shown to the user with some generic text about renting the scooter. While we say this will cost money, we haven't implemented anything to collect from the user, nor do we keep track of that sort of trip data. If you would like that functionality, you can implement it using a Firebase Function and any relevant pay tech (like Google Pay :))

One important thing to note here is that when the user clicks on the positive dialog button, another method will be called named rentScooter. This method will take the scooter's IMEI number and send it to the Firebase Function that we created earlier, which will then use the Particle API to tell the scooter to toggle its state.

fun rentScooter(code: String) {
    // Create the arguments to the callable function.
    val data = hashMapOf(
        "code" to code
    )

    functions
        .getHttpsCallable("rentScooter")
        .call(data)
        .continueWith { task ->
            val result = task.result?.data as String
            result
        }
}

While I haven't implemented it here, the user can be provided with a way to call this same method again in order to deactivate the device.

Conclusion

This project has gone over using Google Maps, Firebase Auth, Firestore, Firebase Functions and the Particle API in order to locate and activate a scooter remotely. Since the project isn't fully implemented, there's a lot of room for others to expand on the software with their own projects, such as storing and using location data for the scooters, managing trips, or even finding a way to extend the scooter to drive itself. In regards to the hardware, one of the biggest updates that I'd love to make would be finding a way to charge the Particle control device from the scooter, so they both charge when plugged in.

Hopefully others are inspired by some of the technology mentioned here, and go off to create more awesome things.