Elham Harirpoush
Published

Accelerated ML inference on Raspberry Pi

Learn How to Accelerated ML inference on Raspberry Pi with TensorFlow Lite and Arm NN Delegate 🚀

IntermediateProtip30 minutes1,680
Accelerated ML inference on Raspberry Pi

Things used in this project

Hardware components

Raspberry Pi 4 Model B
Raspberry Pi 4 Model B
×1
Raspberry Pi 15W USB-C Power Supply
×1
SD Card , minimum 8G
×1

Software apps and online services

Raspberry Pi OS

Hand tools and fabrication machines

TensorFlow Lite
TensorFlow Lite
Arm NN

Story

Read more

Code

Accelerate Inference with Arm NN Delegate

Python
import numpy as np
from PIL import Image
import tflite_runtime.interpreter as tflite

# load delegate
armnn_delegate = tflite.load_delegate('./libarmnnDelegate.so.26', 
                                      options={"backends": "CpuAcc, CpuRef ", "logging-severity": "info"})

# load the TFLite model in TFLite interpreter and allocate tesnors
interpreter = tflite.Interpreter(model_path='./models/mobilenet_v1_1.0_224.tflite', experimental_delegates=[armnn_delegate])
interpreter.allocate_tensors()

# get input and output tensor
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# check the type of the input tensor
floating_model = input_details[0]['dtype'] == np.float32

# NxHxWxC, H:1, W:2
height = input_details[0]['shape'][1]
width = input_details[0]['shape'][2]

interpreter.invoke()  # warm up

# load an image to be classified
img = Image.open('grace_hopper.bmp').resize((width, height))

# add N dim
input_data = np.expand_dims(img, axis=0)

# Classify the image
if floating_model:
    input_data = (np.float32(input_data) - 127.5) / 127.5  # normalizing input

interpreter.set_tensor(input_details[0]['index'], input_data)  # get list of model input details
interpreter.invoke()  # invoke inference
output_data = interpreter.get_tensor(output_details[0]['index'])  # read output tensor values
results = np.squeeze(output_data)  # interpret output
top_k = results.argsort()[-5:][::-1]

# read the lables from text file
def load_labels(filename):
    with open(filename, 'r') as f:
        return [line.strip() for line in f.readlines()]

# read class labels
labels = load_labels('labels.txt')

# return the top 5 classification labels of the image
for i in top_k:
    if floating_model:
        print('{:08.6f}: {}'.format(float(results[i]), labels[i]))
    else:
        print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))

Credits

Elham Harirpoush

Elham Harirpoush

1 project • 3 followers

Comments