Paul DeCarlo
Published © CC BY

LLaMa 2 LLMs w/ NVIDIA Jetson and textgeneration-web-ui

NVIDIA Jetson Orin hardware enables local LLM execution in a small form factor to suitably run 13B and 70B parameter LLama 2 models.

IntermediateFull instructions provided1 hour5,230

Things used in this project

Hardware components

NVIDIA Jetson Orin 64 GB Developer Kit
×1

Software apps and online services

texgeneration-web-ui

Story

Read more

Code

Standalone Dockerfile for text-generation-webui on NVIDIA Jetson Embedded devices

Credits

Paul DeCarlo

Paul DeCarlo

27 projects • 235 followers
Paul DeCarlo is a prof @ #Bauer college of Business @UniversityOfHouston and Software Engineer @Microsoft focused on IoT, Cloud, and Mobile.

Comments