OpenTPU: an open FPGA accelerator running language models

I built openTPU to make an AI accelerator readable from a Python kernel down to the hardware. It now runs language models on an Inspur YPCB-00338 card with a Xilinx Kintex-7 xc7k480t and 4 GiB of DDR3.

The repository contains the SystemVerilog RTL, instruction set, bit-exact simulator, kernel language/compiler, profiler, and Linux host tools. The project is also an experiment in using AI agents for hardware development.

See it running

The README includes an animated demo of LFM2.5-230M chatting through otpu-chat while otpu-smi displays utilization and DRAM bandwidth:https://github.com/FeSens/openTPU

Hardware and data path

The board connects over PCIe Gen2 x8 and provides two 2 GiB DDR3 channels. The current design combines a four-column systolic matrix unit, eight vector lanes, LiteDRAM memory controllers, and a sequencer that explicitly schedules data movement and computation. The matrix unit works with int8 operands; the vector unit handles fp32 operations, and quantization connects the two.

Physical-card results

With 4-bit weights and an int8 LM head, the README reports 82.1 tokens/s for LFM2.5-230M and 30.7 tokens/s for Qwen3-0.6B, including host overhead. The protocol generates 64 greedy tokens after a 512-token prompt. The repository documents quantization accuracy tradeoffs and the test procedures.

Try it without buying a board

The simulator and software run on a laptop. Clone the repository, install the Python package and dependencies, and download LiquidAI/LFM2.5-230M:

git clone https://github.com/FeSens/openTPU.git
cd openTPU
pip install -e .
pip install pytest torch transformers
hf download LiquidAI/LFM2.5-230M --local-dir models/LFM2.5-230M
otpu-chat --model lfm2 --backend isa

Reproduce the board build

The board guide covers Vivado, bitstream generation, JTAG programming, PCIe setup, DDR3 calibration, and diagnostics. Building for this Kintex-7 requires a Vivado license covering the xc7k480t. RTL tests require Verilator 5.

Board guide: https://github.com/FeSens/openTPU/blob/main/docs/board.mdISA: https://github.com/FeSens/openTPU/blob/main/docs/isa.mdProfiler: https://github.com/FeSens/openTPU/blob/main/docs/lens.md

The project code is Apache-2.0. I'd welcome board-port contributions, build feedback, and ideas for making the accelerator easier to study and reproduce.