YOLO26 on the Dragonwing IQ-9075: 31x faster on the NPU

Run YOLO26 on the Dragonwing IQ-9075: from 4.9 FPS on the CPU to 153.6 FPS on the NPU, a 31x speedup, with every step verified on real hardware

By the end, you'll have a working object detector on your board, running on the CPU and on the NPU (a chip built just for AI), both on a video file and on a live webcam. Every command below is copy-paste ready and was tested on real hardware.

See it in action

The same model, the same board, two different scenes, straight off the board's NPU. A concert clip under stage lighting that trips up plenty of detectors:

A basketball game, mid-dunk, with the whole floor picked up:

Both are stock video clips run through the exact pipeline this guide builds. Nothing is hand-tuned per scene. The full annotated videos these frames came from are in results/.

How this works

Two ideas explain everything in this guide.

1. You don't build or convert the model by hand. Qualcomm AI Hub is a cloud service: you send it a model, it compiles a version optimized for your exact chip, and hands back a file you download.

2. One model file can run on different chips. The board has a general-purpose CPU and an NPU built just for AI. Which one actually runs your model is picked when you load it in Python, not when it's compiled:

Terms used in this guide

Quick definitions so nothing below is a surprise. Each links to further reading if you want more depth.

What you'll build

• A model compiled for this exact board, via Qualcomm AI Hub
• Object detection on a single image, running on the CPU
• Object detection on a video file and a live webcam, running on the CPU
• The same detection running on the NPU instead, with one line of code changed
• (Optional) The model quantized to run even faster on the NPU

Before you start

You'll need:

• A Dragonwing IQ-9075 EVK, already flashed with Ubuntu 24.04 and reachable over SSH. If you haven't done this yet, follow Qualcomm's own guides first: flash Ubuntu using QDL, then set up the device.
• A USB webcam (any UVC webcam works; we used a Logitech C922 Pro Stream)
• A computer to work from (Mac or Linux; the steps below are the same on Windows via WSL)
• A free account at aihub.qualcomm.com

Check the board is reachable before doing anything else. <user> is ubuntu unless you changed it during setup:

ssh <user>@<board-ip> 'echo board is up'

Don't know your board's IP? Try ssh <user>@ubuntu.local first (the board announces itself on the local network under its hostname, ubuntu by default). Otherwise check your Wi-Fi router's list of connected devices.

Install Python on your computer. AI Hub's tools need 3.10, 3.11, 3.12, or 3.13. Check what you have:

python3.12 --version

If that command isn't found, install it:

• Mac:brew install [email protected]
• Linux (Ubuntu 24.04):sudo apt install python3.12
• Windows: use WSL, then follow the Linux instructions inside it

Then create a virtual environment and install the AI Hub tools:

python3.12 -m venv ~/qai-hub-env
source ~/qai-hub-env/bin/activate
pip install "qai-hub-models[yolo26-det]"

Get your API token from aihub.qualcomm.com → Settings, then connect your computer to your account:

qai-hub configure --api_token YOUR_TOKEN
qai-hub list-devices | grep -i 9075

If Dragonwing IQ-9075 EVK shows up in the output, you're ready.

Step 1: get the model from AI Hub

Run this in the same terminal as "Before you start, " so the qai-hub-env virtual environment is still active. Opened a new terminal since then? Reactivate it first: source ~/qai-hub-env/bin/activate.

This one command uploads YOLO26n and compiles it specifically for your board's chip:

qai-hub-models export yolo26_det --device "Dragonwing IQ-9075 EVK" \
    --runtime tflite --precision float --ckpt-name yolo26n.pt \
    --skip-profiling --skip-inferencing --output-dir models/

Rename the downloaded file so the rest of the guide's commands match:

mv models/yolo26_det-tflite-float/yolo26_det.tflite models/yolo26n_det_qcs9075.tflite

Stuck, or want to skip ahead? Both compiled models (this one and the Step 8 one) are already included in this repo's models/ folder. You can copy those straight to the board and skip to Step 2.

Step 2: set up the board

Create a project folder and a Python virtual environment on the board itself:

ssh <user>@<board-ip>
mkdir -p ~/projects/dragonwing-yolo26-cpu-to-npu/{scripts,models,results,test-data}
cd ~/projects/dragonwing-yolo26-cpu-to-npu
python3 -m venv venv
./venv/bin/pip install ai-edge-litert opencv-python numpy pillow
exit

Then, from your computer, copy the model, the scripts, and a test image to the board:

scp models/yolo26n_det_qcs9075.tflite <user>@<board-ip>:projects/dragonwing-yolo26-cpu-to-npu/models/
scp scripts/*.py <user>@<board-ip>:projects/dragonwing-yolo26-cpu-to-npu/scripts/
scp test-data/*.jpg test-data/*.mp4 <user>@<board-ip>:projects/dragonwing-yolo26-cpu-to-npu/test-data/

One thing to know: we install plain opencv-python, not opencv-python-headless. Step 5 opens a live window on the board's monitor, and the headless version can't do that at all. Everything before Step 5 works the same either way.

Which script do I run?

Every command from here on is run on the board (ssh in, then cd ~/projects/dragonwing-yolo26-cpu-to-npu/scripts). Here's what each script does:

Step 3: detect objects in one image

../venv/bin/python run_yolo26_litert.py --model ../models/yolo26n_det_qcs9075.tflite \
    --image ../test-data/bus.jpg --out ../results/bus_annotated.jpg
inference: 208.5 ms/run (avg of 20, CPU via LiteRT)
detections (conf>=0.4):
  person         0.90  [529,233,640,521]
  bus            0.90  [4,135,636,441]
  person         0.87  [177,238,273,510]
  person         0.87  [39,235,186,536]
  person         0.68  [-0,330,58,518]

One bus, four people, about 209 ms per image. If your output matches, everything from the cloud compile to the board is working. Remember that number, by Step 8 the same model answers in 6.5 ms.

Step 4: detect objects in a video file

video_demo.py runs the same detector on every frame of a video, in order. This repo includes a 450-frame highway traffic clip in test-data/ so you have something to run right away:

../venv/bin/python video_demo.py --model ../models/yolo26n_det_qcs9075.tflite \
    --source ../test-data/car-highway.mp4 --out ../results/car-highway-annotated.mp4
saved 450 frames to car-highway-annotated.mp4
average 4.7 FPS inference-only

If your numbers come out close to this, everything is working.

That 4.7 FPS is LiteRT's default setting of one CPU thread. Add --threads 8 and the same clip finishes in 32 seconds instead of 106, at 22.7 FPS, with no NPU involved. Every CPU number in this guide leaves the flag off so the comparisons stay on one basis, and because the NPU is about to beat both figures by a wide margin.

Once you're comfortable, point --source at your own clip instead, any short .mp4 works, record one on your phone and scp it to the board, or download a free sample from pexels.com/videos.

The output video plays at the clip's normal speed; 4.7 FPS is how fast the board classifies each frame, not a playback speed. Watch the full annotated video and you'll catch the model flickering on the Penske van, sometimes truck, sometimes bus; high-roof vans do that to detectors trained on COCO's 80 categories. More on that in the results section, and remember this exact frame when you reach Step 8.

Step 5: detect objects from a live webcam

Plug a USB webcam into the board and check it's detected:

ls /dev/video*

You should see /dev/video0 and /dev/video1. If not, unplug and firmly reconnect the webcam (try a different USB port), then check again.

Option A: save to a video file (works over plain SSH).

../venv/bin/python video_demo.py --model ../models/yolo26n_det_qcs9075.tflite \
    --seconds 30 --out webcam_demo.mp4
saved 85 frames to webcam_demo.mp4
average 4.7 FPS inference-only
average 4.2 FPS end-to-end (capture + inference + draw)

The second number also counts capturing and drawing. On the CPU the two sit close together because the model is the slow part; watch this gap in Step 7.

Copy the result to your computer to watch it:

scp <user>@<board-ip>:projects/dragonwing-yolo26-cpu-to-npu/scripts/webcam_demo.mp4 .

Option B: a live window on the board's own monitor. Only works if you're on a terminal open directly on the board's desktop (not a plain SSH session):

../venv/bin/python video_demo.py --model ../models/yolo26n_det_qcs9075.tflite --display

The overlay in the corner shows the live FPS and which model file is running. Press f to toggle fullscreen, q to quit.

You now have working object detection on the CPU. Next: the same thing, much faster.

Step 6: switch to the NPU

This is the part of the guide that's specific to Qualcomm's chip. Changing one setting moves the exact same model, unchanged file, from the CPU to the NPU.

First, let's prove it works with a small check. This code needs to run on the board, so create the file on your own computer first (any text editor), save it as check_npu.py, then copy it over the same way you copied everything else in Step 2:

scp check_npu.py <user>@<board-ip>:projects/dragonwing-yolo26-cpu-to-npu/scripts/

The file itself:

from yolo_common import YoloDetector
import cv2, time

det = YoloDetector("../models/yolo26n_det_qcs9075.tflite", backend="htp")
img = cv2.cvtColor(cv2.imread("../test-data/bus.jpg"), cv2.COLOR_BGR2RGB)
for label, score, box in det.detect(img):
    print(f"{label:10s} {score:.2f}  {[int(v) for v in box]}")

t0 = time.perf_counter()
for _ in range(50):
    det.detect(img)
print(f"{(time.perf_counter() - t0) / 50 * 1000:.1f} ms per frame")

Then, back in your SSH session on the board:

cd ~/projects/dragonwing-yolo26-cpu-to-npu/scripts
../venv/bin/python check_npu.py
person     0.89  [670, 394, 810, 879]
bus        0.89  [2, 228, 804, 744]
person     0.87  [223, 402, 345, 860]
person     0.87  [50, 398, 233, 905]
person     0.66  [0, 556, 70, 873]
16.6 ms per frame

Same five detections as Step 3, at about 13x the speed. Two things to expect on the first run, both normal, not errors:

• A pause and a wall of log text. Loading the NPU delegate does one-time setup (about 1.4 seconds for this model). It only happens once per program run, never per frame.
• A harmless line that says rpcmem_android.c: dummy call to rpcmem_init. Ignore it.

Step 7: run video and webcam on the NPU

video_demo_npu.py is video_demo.py with backend="htp" already added. Same commands as Steps 4 and 5, just point at this script instead:

Same clip as Step 4:

../venv/bin/python video_demo_npu.py --model ../models/yolo26n_det_qcs9075.tflite \
    --source ../test-data/car-highway.mp4 --out ../results/car-highway-npu.mp4
QNN delegate loaded: inference runs on the Hexagon NPU (htp)
saved 450 frames to ../results/car-highway-npu.mp4
average 69.8 FPS inference-only

Step 4 took 106 seconds on the CPU. The NPU finishes in 19 seconds (about 1.4 of those are the one-time setup from Step 6).

Live webcam, same script:

../venv/bin/python video_demo_npu.py --model ../models/yolo26n_det_qcs9075.tflite \
    --seconds 20 --out webcam-npu.mp4
saved 578 frames to webcam-npu.mp4
average 68.4 FPS inference-only
average 28.9 FPS end-to-end (capture + inference + draw)

Step 5's two numbers have now split apart: the NPU can process almost 70 frames a second, but the webcam only delivers about 29. Once the model is fast, the camera is the limit. That's the NPU doing its job: every frame now has spare compute budget for the rest of your application.

And here's what it looks like running live with --display, fullscreen on a monitor:

Step 8 (optional, more advanced): make it even faster

This step is not required to finish the guide. It shrinks the model further so the NPU runs it in about 6.5 ms instead of 12.4 ms. If Step 7's speed is already enough for your project, skip to Results.

What quantization is: the model's numbers are normally 32-bit floats. Shrinking them to 8-bit integers makes the model smaller and lets the NPU's fastest circuits handle it, at an accuracy cost we'll show you honestly below. To do this, AI Hub needs a handful of real sample images ("calibration data") to learn the right number ranges.

This runs on your computer, in the same qai-hub-env from Step 1. The code below is the full picture, but note that my_list_of_26_tensors is a placeholder: you'll need to load and preprocess your own set of about 20-30 images into that format first (each one a (1, 3, 640, 640) tensor). We used bus.jpg plus frames sampled from a car-highway clip.

import qai_hub as hub
import torch
from qai_hub_models.models.yolo26_det.model import Yolo26Detector
from qai_hub_models.models.common import Precision
from qai_hub_models.utils.input_spec import make_torch_inputs

model = Yolo26Detector.from_pretrained()
input_spec = model.get_input_spec()
device = hub.Device("Dragonwing IQ-9075 EVK")

# trace, then compile to ONNX first (quantize_job needs an ONNX target, not a raw trace)
traced = torch.jit.trace(model.to("cpu"), make_torch_inputs(input_spec), check_trace=False)
onnx_job = hub.submit_compile_job(
    model=traced, input_specs=input_spec, device=device,
    options=f"--target_runtime onnx --output_names {','.join(model.get_output_names())}",
)
onnx_job.wait()

# calibration_data wants a LIST of individual (1,C,H,W) tensors, one per sample
calibration_data = {"image": my_list_of_26_tensors}

quantize_job = hub.submit_quantize_job(
    model=onnx_job.get_target_model(),
    calibration_data=calibration_data,
    activations_dtype=Precision.w8a8.activations_type,
    weights_dtype=Precision.w8a8.weights_type,
    options=model.get_hub_quantize_options(Precision.w8a8),
)
quantize_job.wait()

compile_job = hub.submit_compile_job(
    model=quantize_job.get_target_model(), device=device,
    options="--target_runtime tflite",
)
compile_job.wait()
compile_job.get_target_model().download("yolo26n_det_w8a8_qcs9075.tflite")

Gotcha: the obvious shortcut doesn't work here. If you try the convenience path instead (qai-hub-models export yolo26_det --precision w8a8), it fails: the model zoo's own quantize path for YOLO26 detection is w8a16-only and doesn't produce a .tflite at all, only a QNN context binary. The raw API above sidesteps that by quantizing the ONNX export directly rather than going through the zoo's packaged job, and it does produce a working .tflite.

Don't want to build the calibration set yourself? The finished file is already in this repo at models/yolo26n_det_w8a8_qcs9075.tflite. Copy it to the board and skip straight to running it.

Copy the result to the board's models/ folder and rerun Step 6's check_npu.py against it: edit the YoloDetector(...) line so it points at "../models/yolo26n_det_w8a8_qcs9075.tflite" instead, then run it again:

person     0.50  [683, 396, 817, 882]
bus        0.50  [0, 229, 808, 748]
person     0.50  [33, 407, 217, 916]
person     0.50  [16, 553, 79, 860]
person     0.50  [217, 402, 335, 877]
6.5 ms per frame

Same five objects, same boxes to within a couple of pixels. But look at the scores: every one landed at exactly 0.50. That's not a fluke of this one image, it's the honest cost of quantization on this model. Under float, this model's confidence output uses its full range; under w8a8, ours came back with only 16 distinct values across the whole scale, capped at 0.50. bus.jpg happens to be an image where every real object is confident enough to land in that top bin, which is why every score reads identically here.

Here is the exact frame from Step 4, now through the quantized model on the NPU:

The corner overlay has gone from 4.7 FPS to 93.0, and the boxes sit almost exactly where they did before. Two things changed. Every score collapsed to 0.50, as expected. And the Penske van switched class: bus 0.66 under the float model, truck 0.50 here.

That second change is the practical consequence of the first. The van was always a borderline call between two categories, which is why it flickers between them even in Step 4's float video. Under the float model that call is settled by a real margin between two different numbers. Once quantization flattens the confidence scale, the same call comes down to a tie between two values that both read 0.50. If you think truck is the better answer for a Penske rental van, you're right, but the model didn't get more correct here; it got less certain, and the tie broke your way.

So the boxes hold up and the model still finds every vehicle. What you lose is resolution in the confidence number, and with it the tie-breaking between classes that were already close. If your application reads the score as a meaningful quantity (filtering on a tight threshold, ranking detections, logging it for later analysis) rather than as a yes/no gate, budget for that before you rely on it. This is quantization at work, not the NPU; the same quantized file gives the same answers on the CPU too.

One technical note if you write your own loading code: the quantized model's input shape came back as (1, 3, 640, 640) instead of the original (1, 640, 640, 3). yolo_common.py already detects and handles both automatically.

Optional: how we measured everything (CPU vs NPU)

Everything below is background reading, not required to reproduce this guide. scripts/bench.pyis the tool behind every timing number above: it loads one chip configuration, runs 50 timed passes, and logs mean/p50/p90 timing plus a correctness check to results/bench_log.csv.

python3 bench.py --model ../models/yolo26n_det_qcs9075.tflite --backend htp --label htp_default

CPU: the default is misleading. LiteRT uses a single CPU thread unless told otherwise. The same float model on all 8 cores runs at 35.45 ms (28.2 FPS) instead of 205.55 ms (4.9 FPS), nearly 6x faster, with no NPU involved at all. The two CPU scripts, video_demo.py and run_yolo26_litert.py, take --threads 8 if you want to see it yourself. The NPU script has no such flag, since thread count is a CPU setting.

That 28.2 FPS is the model in isolation, though. Run the whole video pipeline with --threads 8 and it settles at 22.7 FPS, because eight inference threads now compete with capture, box drawing, and video encoding for the same eight cores. The NPU sidesteps that entirely: it runs the model off-core and hands the CPU back to your application.

NPU: the actual headline. 12.43 ms float, 6.51 ms quantized. 2.9x faster than 8-thread CPU at the same precision, with the CPU cores left free for your application.

Farm profiles differ from board numbers. AI Hub can also profile your compiled model on its own device farm (hub.submit_profile_job), without you owning a board at all. We ran this for the float model: the farm reports 8.74 ms on the NPU, against 12.43 ms measured here. Farm numbers are a useful sanity check before your hardware even arrives, and they are the right tool for confirming every layer landed on the NPU, but they run optimistic. Trust your own board's numbers for anything you are shipping.

Results

Every configuration was also checked for correctness on bus.jpg, not just speed; all five produce matching detections (the w8a8 rows at compressed confidence values, see Step 8).

Full pipeline speed, same 450-frame clip, start to finish:

One more thing worth knowing: this model only recognizes 80 object categories (COCO's classes). Anything outside those gets mapped to the closest match it knows, hence the Penske van landing on bus. Normal behavior, not a bug. Raise --conf 0.7 for quieter output, or fine-tune on your own classes (a later guide in this series).

Project structure

Troubleshooting

Everything here actually happened during this build.

qai-hub-models won't install, or installs an ancient version. Check your Python version: recent releases require >=3.10,<3.14. If your default python3 resolves to 3.14, pip may silently install an old, incompatible release instead of failing loudly. Use python3.12 explicitly, as shown in Before you start.

qai-hub-models export yolo26_det --precision w8a8 fails or produces a QNN context binary instead of a .tflite. Expected: the zoo's convenience path for YOLO26 detection is w8a16-only with no TFLite target. Step 8 uses the raw API instead, which does produce a working .tflite.

--display fails immediately with a Qt/xcb error (could not connect to display, Could not load the Qt platform plugin "xcb"). Your session has no display to open a window on. This is expected over a plain SSH connection; run the command from a terminal on the board's own desktop instead, or reconnect with ssh -X for X11 forwarding.

First NPU inference takes a couple of seconds and prints a wall of Starting stage: logs. Normal: one-time setup when the delegate loads the model (about 1.4 s for the float model here, the quantized one is comparable). Happens once per program run, never per frame.

Live webcam mode runs near 10 FPS no matter which chip you pick. The camera is sending uncompressed video, which saturates USB bandwidth at 720p around 10 FPS. Both demo scripts already request the compressed MJPG format to avoid this; if you write your own capture code, set that option before setting the resolution. Indoor lighting can also hold a camera below its rated 30 FPS through auto-exposure.

ValueError: Cannot set tensor: Dimension mismatch on the quantized model. The w8a8 model uses a different input layout than the float one. yolo_common.py already handles both; if you're writing your own code, check interpreter.get_input_details() rather than assuming.

Going further

• Point video_demo_npu.py at an RTSP URL instead of a webcam index, and it becomes a network-camera analytics node with 8 idle CPU cores to spare.
• Fine-tune YOLO on your own classes and bring it through AI Hub as a custom model instead of a zoo entry (a later guide in this series).
• Our Step 8 calibration set was 26 images picked for convenience, not curated for coverage; a larger, more representative set may narrow the confidence compression we measured.
• When you move any model onto an accelerator, verify the outputs, not just the speed; a backend can load a model, report fast timings, and still be wrong.