Lower Limb Exosuit Walking Algorithm

Think of a device which incorporates sensors and motion analysis to recognize walking patterns. The device is intended to provide paraplegics a comfortable walk in indoor spaces to help them access toilets and other utilities in a home/office environment. This reduces the terrain variations and the complexities associated with it, bringing down the cost of the product. This is a problem faced by every community all over the world.

How can we achieve this?

Walking patterns are different for each individuals. Making a device which generates comfortable walking patterns from the user's height and weight distribution can be used . A machine learning algorithms would be employed to create motor control codes.

The device should also learn normal walking pattern from healthy people, cross reference with data from disabled people and then implement to disabled user. The system would adapt as users become familiar with the device. The generated data could be accessed by doctors to assess conditions of patients.

Why Avnet Ultra 96 Development Board?

Compute-intensive tasks can be easily done by this board . This is critical in producing real-time response and decision making in systems and is perfect for our project.

The Basic Setup

We are using Pynq environment to make this AI project. Download the Ultra96 Pynq Image, and use Win32DiskImager to get it onto your microSD card.

You can refer to this video the instructions to setup Pynq is shown clearly here

The PYNQ environment can be accessed using a web browser over a network connection. You can connect to the board in three ways:

• USB cable
• Use the board’s WiFi access point
• Connect a monitor, keyboard and mouse directly to the board.

All three connection methods will be covered below. USB cable is the recommended way to start using PYNQ on the Ultra96.

USB cable board setup

Insert the Micro-SD card
Connect the Mirco USB cable from your PC to the board
Connect the power supply to the board
Press the SW3 power button to power on the board

Turning on the Ultra96

The green power LED (beside the power switch), and the redLED (near the SD Card socket) will turn on when the power button is pressed. After a few seconds, the red LED will switch off, and the green Done LED will switch on. The board will be ready after the blue and orange LEDs switch on.

Connecting to the Ultra96

USB cable

When connected to a computer using the USB cable, the board can act as an Ethernet device (“Ethernet Gadget”). This is the recommended way to connect to the board from your computer, and allows you use a browser on your host computer.

WiFi

The Ultra96 includes a WiFi chip that can be used to connect to other WiFi networks as a client, or act as a WiFi access point (hotspot).

WiFi access Point

On boot, the board will enable itself as a WiFi access point.

The Access point name will be pynq_<MAC Address> where <MAC address> is the MAC address for your board.

To connect to the board’s WiFi network (access point), we can open our wireless network configuration, search for and connect to an SSID starting with pynq_ followed by the MAC address of your board.

Other WiFi networks

You can connect the Ultra96 to other WiFi networks. If you can connect to the board using the USB cable, then you can use the WiFi notebook in the Jupyter home area to configure the board to connect to a WiFi network:

<Jupyter Home>/common/wifi.ipynb

If you are familiar with configuring network connections in Linux, you can also configure your connection using a terminal.

DisplayPort monitor

You can also use the board like a PC by connecting monitor mouse and keyboard. The Chromium browser is installed on the board and can be used with PYNQ.

Note: that you can try PYNQ examples in this mode, but for optimal performance it is recommended to use a browser from a computer connected to the board.

Connecting to Jupyter Notebooks

To connect to Jupyter Notebook, on the host computer, open a web browser (Chrome is preferred) and browse to one of the following addresses depending on how you setup your connection to the board in the instructions above:

• USB: http://192.168.3.1
• WiFi access point: http://192.168.2.1
• Other WiFi: Browse to the IP address of the board http://<IP address of board> You can find the IP from a terminal, from your router, or from a network scanning app.
• DisplayPort monitor: from the Chromium browser, browse to http://192.168.2.1

It may take a few seconds for your computer to resolve the IP address and connect to the board the first time it is powered on.

Note that the board can have multiple connections to different networks at the same time.

If your board is configured correctly you will be presented with a login screen. Enter the password as xilinx

After logging in, you should see the following screen:

We need to get online. We can use the Jupyter wifi notebook to get the wifi network running and get online. Connect to the Pynq AP, and browse to "http://192.168.3.1:9090/notebooks/common/wifi.ipynb" from the computer you connected to the Ultra96 board, and follow the instructions.

The key cells are:

from pynq.lib import Wifiport = Wifi()

This gets the Wifi module ready, then we connect to our network with the next cell:

ssid = input("Type in the SSID:")pwd = input("Type in the password:")port.connect(ssid, pwd)

You can then go to the Home screen ....click on the new tab and click terminal.

In the terminal window type

 pip3 install --upgrade pip 

Installing Scikit-Learn

Scikit learn is a simple and efficient tools for data mining and data analysis which is accessible to everybody, and reusable in various contexts. It is relatively simple and very easy to use . In this project we can use scikit learn.

In the terminal window type

$ pip3 install -U --user scikit-learn

then from the home screen click on upload button and upload the python notebook file which is given in tutorial.

The entire code is well documented it will be easier for you to understand. Click on run button to run the code.

In this project we are also installing Scikit-multilearn. why?

Let's take a look at our data

Sex,Body mass (kg),Lower limb length (mm) are our Targets.Velocity (m s−1),Hip flexion angle at H-flex (°),Hip flexion angle at H-ext (°),Knee flexion angle at K-flex (°),Knee flexion angle at K-ext (°),Ankle flexion angle at A-plant (°),Ankle flexion angle at A-dors (°) are our labels.

ML algorithms works like a function

F(X) = Y

where X= targets and Y= Labels.

In this case there are more labels so we need to use multi-label classification.

1. What is Multi-Label Classification?

Let us take a look at the image below.

What if I ask you that does this image contains a house? The option will be YES or NO.

Consider another case, like what all things (or labels) are relevant to this picture?

These types of problems, where we have a set of target variables, are known as multi-label classification problems. So, is there any difference between these two cases? Clearly, yes because in the second case any image may contain a different set of these multiple labels for different images.

Scikit-learn has provided a separate library scikit-multilearn for multi label classification.

Adapted algorithm

Adapted algorithm, as the name suggests, adapting the algorithm to directly perform multi-label classification, rather than transforming the problem into different subsets of problems.

For example, multi-label version of kNN is represented by MLkNN. So, let us quickly implement this on our randomly generated data set.

from skmultilearn.adapt import MLkNN
classifier = MLkNN(k=20)
# train
classifier.fit(X_train, y_train)
# predict
predictions = classifier.predict(X_test)

Sci-kit learn provides inbuilt support of multi-label classification in some of the algorithm like Random Forest and Ridge regression. So, you can directly call them and predict the output.

You can check the multi-learn library if you wish to learn more about other types of adapted algorithm.

Congratulations!!!!! you have successfully completed motion analysis from gait data.

Future Development

Reading datas from the sensors and predicting the motor angle has to be done in order to make effective use of this algorithm .

We have only scratched the surface of true potential of Ultra96 Development Board , There are many things we can do . In coming tutorials I will be sharing the other cool projects using Ultra 96 development board.

Comments and Questions welcome! Thanks for reading.