DeepSeek On the ZUBoard

Introduction

Unless you have been living under a rock for the last few years, you cannot have missed the rise of Large Language Models (LLMs) in the field of AI.

Large Language Models are a form of AI that is able to understand human text queries and generate a text response. The "large" comes from the fact that the LLM is trained on a large amount of data from web pages, books, emails, conversations, etc. It is one of the reasons why data is so valuable to many organizations, especially social media companies.

What this means is we can use LLMs for a range of applications, from answering questions to helping write essays, check grammar, and even write code. There is no real intelligence behind this — just deep learning models based on complex mathematical operations such as linear algebra and optimization. For a given prompt, the LLM responds with the predicted best text response determined by its weights and biases.

As with most AI training, this results in weights and biases that define how the model functions. However, as there is such a large training set, it is typical for LLMs to have billions or tens of billions of these parameters.

Typically, this means that if we want to achieve good performance, we need to leverage high-performance GPUs such as AMD Instinct™ GPUs.

However, there are some use cases where we might want to deploy an LLM on an edge device locally. I was also curious if this could be achieved, so in this project, we are going to get one of the simpler DeepSeek models up and running on the Tria Technologies ZUBoard, which features Arm® Cortex®-A53 cores within an AMD Xilinx Zynq™ UltraScale+™ MPSoC.

Before we do that, however, I should outline the use cases for a standalone edge deployment.

• Privacy — The model is hosted locally on the edge device
• Offline access — No need for connection to the wider internet
• Cost — Saves on hosted cloud inference costs

Like all engineering, deploying at the edge is a compromise between privacy, cost, and performance.

What is DeepSeek

If you are not familiar with DeepSeek, it is a suite of high-performance open-weight LLMs and large multimodal models (LMMs) developed to push the frontiers of open AI research. DeepSeek LLMs are trained from scratch on trillions of tokens across multilingual web crawls, academic literature, codebases, and curated datasets. Architecturally, they are based on transformer designs with enhancements like Grouped Query Attention (GQA) for scaling efficiency, SwiGLU activation functions for faster convergence, and multi-query attention layers for optimized inference.

The models are available in various sizes (1.5B, 7B, 13B parameters and beyond), enabling different trade-offs between computational resource usage and accuracy.

The DeepSeek-VL (Vision-Language) models extend the foundation by introducing multimodal pretraining, combining large image datasets (like LAION-5B) with text supervision to create models capable of cross-modal reasoning, visual question answering, and image-text generation.

Quantized versions (e.g., 4-bit, 8-bit) are also provided for edge and mobile deployments.

DeepSeek models demonstrate state-of-the-art or near SOTA performance across popular benchmarks like MMLU, GSM8K, HumanEval (code generation), and MMMU (multimodal benchmarks). They are particularly optimized for efficient inference on both CPU and GPU environments using techniques like flash attention and mixed-precision training.

Getting started with the ZUBoard

To get started with the project, we are going to use the AMD PYNQ™ platform as the base for the DeepSeek installation.

This means we will first need to download the appropriate image for the Avnet ZUBoard from the PYNQ website and burn the image to an SD card.

Once the image has been burned onto the SD card, the next step is to set the boot switches to SD card, insert the SD card, and connect the USB UART cable and Ethernet cable into the ZUBoard.

Once the PYNQ image has booted, you will be able to use a browser to go to the address

http://pynq:9090.

The next step is to open a new terminal window and start the installation of DeepSeek.

Preparation

As the ZUBoard only has 1GB of LPDDR4, we need to correctly configure a swap file. If you are not sure what a swap file is, a swap file is a special file or reserved space on your storage drive (like an SSD or hard drive) that your computer uses as "extra memory" when your physical RAM is full. In this case, it will be located on the SD card.

Using a swap file prevents the application from crashing when memory space runs out; however, it will impact system performance.

We are going to provide the ZUBoard with at least 4GB of swap space on the SD Card by using the following commands:

swapon --show
sudo swapoff /var/swap
sudo rm /var/sawp
sudo chmod 600 /var/swap
sudo mkswap /var/swap
sudo swapon /var/swap

Running swapon --show at the end of the sequence should show the swap space is now 4GB.

Ollama

To install the DeepSeek model, we are going to use the Ollama framework. Ollama is an open-source framework that makes it easy to run LLMs locally on your own machine — whether it's a laptop, desktop, server, or edge device.

To get started, the first thing we need to do is install Ollama using the following command:

curl -fsSL https://ollama.com/install.sh | sh

We can ignore the warning about GPU as we know we do not have one within our system.

Once we have Ollama installed, we are able to install DeepSeek. For simplicity and to keep the number of parameters manageable, we will download the DeepSeek 1.5 billion parameters model.

Before we can download the model, we need to define the HOME directory:

mkdir deepseek
export HOME=/home/xilinx/deepseek

We are now ready to download and install the DeepSeek 1.5B model onto the ZUBoard:

ollama pull deepseek-r1:1.5b

Testing

With DeepSeek installed, the next step is to try working with the model running on the ZUBoard. Reminder: as it has only 1.5 billion parameters, it might not be as "intelligent" as larger models. As we are also using an edge device with a swap file, we will experience some performance impact.

We can start the DeepSeek model by using the command:

ollama run deepseek-r1:1.5b

You will see the prompt waiting for you to ask it something.

The first thing I did was to ask it about the model

The model information can be seen in the image below.

I then interacted with the model as one would when working with a normal LLM.

Running top in another terminal will show the demands on the system.

Wrap Up

This project has shown that we can implement an LLM at the edge, without the need for a GPU. Its performance is reduced; however, it does provide a standalone solution which can be beneficial in some edge cases.

Ollama also enables us to deploy other models on the Tria Technologies ZUBoard outside of DeepSeek.