DAWA: Pill Classifier

Pill Classifier

In the UK alone, more than 20,000 people pass away each year due to eating the wrong pill. In USA, Medication Error is the third leading cause of death. With more and more medicines being released, this is a growing problem. Furthermore, for elderly citizens who need to take the correct medicine at the correct time and many get confused as to which medicine they should take as many medicines look similar.

Identifying a pill that a patient has taken is a difficult task. For a patient who has accidentally taken an unknown pill or for a patient who needs to take the correct pill at the correct time, identifying the correct pill can be a life or death task. Often times, many pills look similar and patients and even doctors have a hard time identifying them. The first step however to identifying a pill is to know what shape it has.

Pills can come in various shapes: Oval, Round, Capsule etc. In this project, we use a Convolutional Neural Network (CNN) to identify what shape a pill has and hence help identify the pill.

Data

The data for this project was collected from the Pillbox dataset. The link for this can be found here.

From this dataset, we seperated the pills into 5 classes: Capsule, Oval, Oblong, Round and Other. The separated dataset which is the one that was used for training.

This is a new dataset that was built for this project. We believe that this dataset can be used also for learning about Deep Learning as it is simple and yet complex enough due to class imbalances to be challenging. You can find out more information about this dataset here.

Schematic

Schematic and connections for this is very simple. Just connect the Pi Cam to the Raspberry Pi and point the camera at the pills.

Neural Network

Since the neural network is meant to be run on an embedded or Edge device, the neural network model chosen was one built specifically for Edge devices: Xception.

You can train the neural network using this script here. You can specify how many epochs you want to train for, where to save the model and how many times to fine tune to model using command line arguments.

python3 xception_transfer.py --nb_epoch 1 --batch_size 16 --model models/model.h5

You can then perform classification on the model by using the classification script which is as follows:

import keras
from picamera import PiCamera
from picamera.array import PiRGBArray
from keras.models import Model, load_model
from keras.applications import Xception
from keras.applications.xception import preprocess_input
from keras.preprocessing import image

import numpy as np
#import matplotlib.pyplot as plt
import time
import cv2
import os

def pp_image():
   img = image.load_img('pic.png', target_size=(224, 224))
   x = image.img_to_array(img)
   x = np.expand_dims(x, axis=0)
   x = preprocess_input(x)
   return np.asarray(x)

prediction_list=['Capsule', 'Oval', 'Oblong', 'Round', 'Other']
model=load_model('models/model.h5')
camera = PiCamera()
rawCapture=PiRGBArray(camera)
for i in range(10):
   time.sleep(0.5)
   try:
       # import ipdb; ipdb.set_trace()
       # Access the image data.
       camera.capture(rawCapture, format='rgb')
       img=rawCapture.array
       cv2.imwrite('pic.png', img)
       #import ipdb; ipdb.set_trace()
       pred_img=pp_image()
       yo=model.predict(pred_img)
       pred=prediction_list[np.argmax(yo)]
       cv2.putText(img, pred, (10,1000), cv2.FONT_HERSHEY_SIMPLEX, 5, (0,0,0), 5, False)
       name='img'+str(i)+'.png'
       cv2.imwrite(os.path.join('prediction_images', name), img)
       rawCapture.truncate(0)
       #print("Gray value of first pixel: ", img[0, 0])
   except:
       print('Could not perform prediction')
camera.stop_preview()

Run it using the following command:

python3 classify_pills.py

Prediction Images

Requirements

This project requires `python3.6`. While you can train on a GPU or any other machine, for performing inference, you will need the Raspberry Pi. Other package requirements are in the requirements file.