Paul DeCarlo
Published © CC BY

LLaMa 2 LLMs w/ NVIDIA Jetson and textgeneration-web-ui

NVIDIA Jetson Orin hardware enables local LLM execution in a small form factor to suitably run 13B and 70B parameter LLama 2 models.

IntermediateFull instructions provided1 hour9,999

Things used in this project

Hardware components

NVIDIA Jetson Orin 64 GB Developer Kit

Software apps and online services



Read more


Standalone Dockerfile for text-generation-webui on NVIDIA Jetson Embedded devices


Paul DeCarlo

Paul DeCarlo

28 projects • 240 followers
Paul DeCarlo is a prof @ #Bauer college of Business @UniversityOfHouston and Software Engineer @Microsoft focused on IoT, Cloud, and Mobile.