Serge - LLaMA made easy 🦙

Serge is a chat interface crafted with llama.cpp for running Alpaca models. No API keys, entirely self-hosted!

🌐 SvelteKit frontend
💾 Redis for storing chat history & parameters
⚙️ FastAPI + LangChain for the API, wrapping calls to llama.cpp using the python bindings

🧠 Supported Models

Category	Models
Alpaca 🦙	Alpaca-LoRA-65B, GPT4-Alpaca-LoRA-30B
Chronos 🌑	Chronos-13B, Chronos-33B, Chronos-Hermes-13B
GPT4All 🌍	GPT4All-13B
Koala 🐨	Koala-7B, Koala-13B
LLaMA 🦙	FinLLaMA-33B, LLaMA-Supercot-30B, LLaMA2 7B, LLaMA2 13B, LLaMA2 70B
Lazarus 💀	Lazarus-30B
Nous 🧠	Nous-Hermes-13B
OpenAssistant 🎙️	OpenAssistant-30B
Orca 🐬	Orca-Mini-v2-7B, Orca-Mini-v2-13B, OpenOrca-Preview1-13B
Samantha 👩	Samantha-7B, Samantha-13B, Samantha-33B
Vicuna 🦙	Stable-Vicuna-13B, Vicuna-CoT-7B, Vicuna-CoT-13B, Vicuna-v1.1-7B, Vicuna-v1.1-13B, VicUnlocked-30B, VicUnlocked-65B
Wizard 🧙	Wizard-Mega-13B, WizardLM-Uncensored-7B, WizardLM-Uncensored-13B, WizardLM-Uncensored-30B, WizardCoder-Python-13B-V1.0

Additional weights can be added to the serge_weights volume using docker cp:

docker cp ./my_weight.bin serge:/usr/src/app/weights/

LLaMA will crash if you don't have enough available memory for the model:

Nathan Sarrazin and Contributors. Serge is free and open-source software licensed under the MIT License.

Need help? Join our Discord