AMD Ryzen AI SW: Model Execution Flow

Introduction

As AI workloads move from the cloud to the edge, the ability to run high-performance inference locally on a laptop has become a game-changer. At the heart of this shift is the AMD Ryzen™ AI NPU, a dedicated accelerator built on the XDNA™ architecture. For developers using the Ryzen AI "Phoenix" series (such as the Ryzen 7040 or 8040), unlocking this hardware requires a specific software dance: the Ryzen AI Software (RYZENAI-SW) stack.

In this project, I walk through the end-to-end process of transforming a standard Windows machine into a powerful AI development environment. Unlike raw hardware programming, we will focus on the official Ryzen AI SW workflow. This involves installing the NPU drivers, configuring a Conda-based toolchain, and utilizing the Vitis AI Execution Provider (EP) to offload computations from the CPU to the NPU.

In this session we are looking into following,

• Tool Installation
• Hardware Verification
• Test Model Run

Prerequisites

Before installing the Ryzen AI Software, ensure the system meets the following hardware and software requirements:

• Hardware: A laptop or PC featuring an AMD Ryzen™ 7040 or 8040 series processor with an integrated NPU (Phoenix/Hawk Point).
• Operating System: Windows 11 64-bit (Version 22H2 or later recommended).
• Software Tools:Visual Studio 2022 with C++ Desktop Development workload.
• Miniconda: latest version for python environment management.
• CMake (version 3.26 or higher).

Tool Installation

1. Driver Setup

The Phoenix NPU requires the AMD NPU driver to communicate with the OS.

• Download: AMD Ryzen™ AI Software Driver from download link.
• Verification: Open Device Manager > Neural processors. Confirm AMD NPU Device is present.

2. Software Stack Installation

Install the core components (ONNX Runtime, Vitis AI Execution Provider (EP), and Quantizer) using the Ryzen AI SW package:

• Download: Obtain the Ryzen AI Software installer from download link.
• Launch Installer: Run the EXE and follow the setup wizard:
• License Agreement: Review and accept the terms.
• Destination Folder: Select the installation path (Default: C:\Program Files\RyzenAI\1.7.0).
• Environment Name: Specify the name for the automated Conda environment (Default: ryzen-ai-1.7.0).
• Completion: The installer automatically configures the Ryzen AI Software packages within the new Conda environment.

Hardware Verification

Check if the system recognizes the NPU hardware:

• Open Task Manager.
• Navigate to the Performance tab.
• Verify an NPU entry exists.

Test Model Run

1. Model Selection & Download

For this test, we use the MobileNet V2 model optimized by AMD. This model is pre-quantized for the XDNA architecture, ensuring it runs efficiently on the Phoenix NPU.

• Source: Download the mobilenet_v2_1.0_224.onnx file from the AMD Hugging Face repository.
• Format: The model is in ONNX format and has been quantized to INT8 to meet the NPU's hardware requirements.
• The MobileNet V2 model requires a specific input shape of [1, 3, 224, 224] (Batch Size, Channels, Height, Width). To execute the test, use matching dataset from Hugging Face that provides image data compatible with these dimensions.

2. Model Run and Verification

conda activate <env_name>

Run below commands to execute on CPU (68.68s)

python eval_onnx.py –onnx_model mobilenetv2_int8.onnx –data_dir data\imagenet-mini\

Run below commands to execute on NPU (35.50s)

python eval_onnx.py –onnx_model mobilenetv2_int8.onnx –data_dir data\imagenet-mini\  –ipu

Verify NPU offloading by checking the Activity of NPU in Task Manager >> Performance Tab.

Conclusion

This project successfully demonstrates the setup and verification of the Ryzen AI Software stack on Phoenix hardware. By installing the XDNA drivers and utilizing the Vitis AI EP, we moved the MobileNet V2 workload from the CPU to the dedicated NPU. The successful inference run confirms that the pipeline—from environment configuration to hardware execution—is fully operational, providing a high-efficiency foundation for local AI development on AMD Ryzen™ AI.