Hardware components | ||||||
| × | 1 | ||||
 |
| × | 1 | |||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
 |
| |||||
| ||||||
The Magic Leap 1 COVID-19 Detection System 2020 uses Tensorflow 2, Raspberry Pi 4 & Magic Leap 1 to provide a spatial computing detection system.
The project uses the COVID-19 Tensorflow DenseNet Classifier a Tensorflow implementation of DenseNet and the SARS-COV-2 Ct-Scan Dataset, a large dataset of CT scans for SARS-CoV-2 (COVID-19) identification created by our partners at Lancaster University, Plamenlancaster: Professor Plamen Angelov Centre Director @ Lira, & his researcher, Eduardo Soares PhD.
We use the trained model from COVID-19 Tensorflow DenseNet Classifier with the COVID-19 Tensorflow DenseNet Classifier For Raspberry Pi 4 and serve an API endpoint that exposes the Artificial Intelligence classifier allowing Magic Leap 1 to communicate with it.
DISCLAIMERThis project should be used for research purposes only. The purpose of the project is to show the potential of Spatial Computing, Artificial Intelligence, and the Internet of Things for medical support systems such as diagnosis systems.
Although the classifier used in this project is very accurate and shows good results both on paper and in real world testing, it is not meant to be an alternative to professional medical diagnosis.
Developers that have contributed to this repository have experience in using Artificial Intelligence for detecting certain types of cancer & COVID-19. They are not a doctors, medical or cancer/COVID-19 experts. Please use these systems responsibly.
PrerequisitesBefore you can install the Magic Leap 1 Unity COVID-19 Detection System, there are some prerequisites.
HIAS ServerIf you are going to be using the full system you will need to install the HIAS server. Follow the HIAS Installation Guide to complete your HIAS server setup.
If you chose not to use the full system, steps are provided in this tutorial that will allow you to use the system without a HIAS installation.
COVID-19 Tensorflow DenseNet ClassifierIf you want to train your own Artificial Intelligence required to detect COVID-19, you will need to complete the COVID-19 Tensorflow DenseNet Classifier tutorial. If you would like to use the pre-trained model we have provided, you can skip to the next step.
YOU MUST USE THE SAME TRAIN AND TEST DATA AS THE TUTORIAL SPECIFIES
The test data provided in the Magic Leap 1 Unity COVID-19 Detection System requires the same model to be used.
COVID-19 Tensorflow DenseNet Classifier For Raspberry Pi 4The COVID-19 Tensorflow DenseNet Classifier For Raspberry Pi 4 hosts a local server on a RPI4 allowing images to be classified remotely. In this project we use the AI model trained in the COVID-19 Tensorflow DenseNet Classifier, but we also provide the pre-trained model so that you can use the classifier "out of the box".
Regardless of whether you choose to train your own model or use the pre-trained model, you will need to complete the COVID-19 Tensorflow DenseNet Classifier For Raspberry Pi 4 tutorial as this is the project that creates the classifier that the Magic Leap 1 will interact with.
Clone the repositoryClone the COVID-19 AI Classification repository from the Peter Moss COVID-19 AI Research Github Organization.
To clone the repository and install the COVID-19 Tensorflow DenseNet Classifier For Raspberry Pi 4, make sure you have Git installed. Now navigate to the home directory on your device using terminal/commandline, and then use the following command.
$ git clone https://github.com/COVID-19-AI-Research-Project/Magic-Leap-1-Detection-System.gitOnce you have used the command above you will see a directory called COVID-19-AI-Research-Project in your home directory.
lsUsing the ls command in your home directory should show you the following.
Magic-Leap-1-Detection-SystemNavigate to Magic-Leap-1-Detection-System/Projects/Unity directory, this is your project root directory for this tutorial.
Developer ForksDevelopers from the Github community that would like to contribute to the development of this project should first create a fork, and clone that repository. For detailed information please view the CONTRIBUTING guide. You should pull the latest code from the development branch.
$ git clone -b "0.2.0" https://github.com/COVID-19-AI-Research-Project/Magic-Leap-1-Detection-System.gitThe -b "0.2.0" parameter ensures you get the code from the latest master branch. Before using the below command please check our latest master branch in the button at the top of the project README.
InstallationBelow you will find the steps required to setup for Magic Leap development.
Magic Leap BasicsFollow the Magic Leap Developer Setup guide to install the required Magic Leap development tools.
Unity 2019.3.4f1For Magic Leap apps you need Unity 2019.3.4f1. Head to the Unity 2019.3.4 page and download and install the Lumin Target Support exe.
Open Magic Leap's Lab and click on the Launch button for Unity Hub.
Unity Hub will now open.
Go to the Installs tab in Unity Hub and select Unity 2019.3.4f1 (LTS), click next and continue until Unity 2019.3.4f1 (LTS) is installed.
Click on the ADD button. This will open Windows Explorer allowing you to navigate to the project and import it.
To import Magic Leap 1 Unity COVID-19 Detection System into Unity Hub, navigate to the Magic-Leap-1-Detection-System/Src directory and click Select Folder.
Now for some configuration. In Unity, open File->Build Settings and change the target to Lumin if not already done.
Now go to Edit->Preferences and head to the Lumin section. You now need to use the browse button to navigate to your version of the Lumin SDK. It should be in the path C:/Users/YourUser/MagicLeap/mlsdk/v0.24.1.
Next you need to add your Magic Leap certificate. Follow the Get a Developer Certificate guide on Magic Leap's website.
Once you have your certificate you need to add it to your Unity project. Go to Edit->Project Settings->Publish Settings and click Sign Package then click the button ... to open Explorer, navigate to the location you saved your certificate to and select the folder.
Magic Leap 1You now need to setup your device for development. Power up your Magic Leap 1 and head to Settings->Device->Developer Mode. You need to Enable Developer Mode, Allow Untrusted Sources, and Enable MLDB Access.
UsageAfter following the installation guide you should now have all of the required tools installed. Use the following guides to start using the Magic Leap 1 Unity COVID-19 Detection System.
Unity HubOpen Magic Leap's Lab and click on the Launch button for Unity Hub.
Once you have done this you can open the project in Unity by clicking on it.
Now you need to setup your Magic Leap 1 device in Device Bridge. To do this first turn on your Magic Leap device and open your world. Make sure you are in the center of your room and facing the way you want your app to open.
Once turned on and inside your world, you need to attach your Magic Leap 1 to your computer using the USB type C connector.
Now head to Device Bridge in Lab. You should see the above in Device Bridge.
Click on the WiFi Bridge button to activate WiFi, Device Bridge will reload and you should see the IP assigned to your device. This means that your PC is now bridged to the Magic Leap 1 device via WiFi.
Zero IterationZero Iteration allows you to run your apps directly on the Magic Leap 1 device. Providing you have followed the above steps, open Zero Iterator from the Lab homepage, select your device, and restart the program.
Classification ServerNow you need to make sure that your COVID-19 Tensorflow DenseNet Classifier For Raspberry Pi 4 is running in Server mode and waiting for requests.
Run The AppNow you are ready to fire up the Magic Leap 1 Unity COVID-19 Detection System!
Head over to Unity and click on the play button. You can remove the cable connection and move to the center of your room. You should now see the Magic Leap 1 Unity COVID-19 Detection System in your view.
Now use your controller to hit the blocks by pointing the laser at the blocks. This will send the relevant image to the server for classification.
The server will return the response to the Magic Leap 1 and the color of the block will change.
- RED Specifies a true positive (COVID-19 positive)
- GREEN Specifies a true negative (COVID-19 negative)
- Magento Specifies a false positive (COVID-19 negative but classifier determined COVID-19 positive)
- Cyan Specifies a false negative (COVID-19 positive but classifier determined COVID-19 negative)
You can reset the application by hitting the RESET button with the laser.
ContributingThe Peter Moss COVID-19 AI Research Project encourages and welcomes code contributions, bug fixes and enhancements from the Github.
Please read the CONTRIBUTING document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page.
Contributors- Adam Milton-Barker - Asociacion De Investigation En Inteligencia Artificial Para La Leucemia Peter Moss President & Lead Developer, Sabadell, Spain
We use SemVer for versioning. For the versions available, see Releases.
LicenseThis project is licensed under the MIT License - see the LICENSE file for details.
Bugs/IssuesWe use the repo issues to track bugs and general requests related to using this project. See CONTRIBUTING for more info on how to submit bugs, feature requests and proposals.
Comments