Add Serge Chat (#1283)

README.md

@@ -127,6 +127,7 @@ This is the official repository for the Tipi App Store. It contains all the apps
 - [Sabnzbd](https://github.com/sabnzbd/sabnzbd) - Sabnzbd makes Usenet as simple and streamlined as possible by automating everything we can"
 - [SearXNG](https://github.com/searxng/searxng) - SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled
 - [Send](https://gitlab.com/timvisee/send) - Simple, private file sharing
+- [Serge](https://github.com/serge-chat/serge) -  A web interface for chatting with Alpaca through llama.cpp.
 - [SimpleX SMP](https://github.com/simplex-chat/simplexmq) - A reference implementation of the SimpleX Messaging Protocol for simplex queues over public networks.
 - [Sonarr](https://github.com/Sonarr/Sonarr) - TV show manager for Usenet and BitTorrent
 - [Sshwifty](https://github.com/nirui/sshwifty/) - Web SSH & Telnet (WebSSH & WebTelnet client)

apps/serge/config.json

@@ -0,0 +1,16 @@
+{
+  "$schema": "../schema.json",
+  "name": "Serge",
+  "available": true,
+  "exposable": true,
+  "port": 8008,
+  "id": "serge",
+  "tipi_version": 1,
+  "version": "0.4.1",
+  "categories": ["ai"],
+  "description": "",
+  "short_desc": "LLaMA made easy",
+  "author": "nsarrazin",
+  "source": "https://github.com/serge-chat/serge",
+  "form_fields": []
+}

apps/serge/docker-compose.yml

@@ -0,0 +1,38 @@
+version: "3.7"
+services:
+  serge:
+    image: ghcr.io/serge-chat/serge:0.4.1
+    container_name: serge
+    restart: unless-stopped
+    ports:
+      - ${APP_PORT}:8008
+    networks:
+      - tipi_main_network
+    volumes:
+      - ${APP_DATA_DIR}/data/weights:/usr/src/app/weights
+      - ${APP_DATA_DIR}/data/db:/data/db/
+    labels:
+      # Main
+      traefik.enable: true
+      traefik.http.middlewares.serge-web-redirect.redirectscheme.scheme: https
+      traefik.http.services.serge.loadbalancer.server.port: 8008
+      # Web
+      traefik.http.routers.serge-insecure.rule: Host(`${APP_DOMAIN}`)
+      traefik.http.routers.serge-insecure.entrypoints: web
+      traefik.http.routers.serge-insecure.service: serge
+      traefik.http.routers.serge-insecure.middlewares: serge-web-redirect
+      # Websecure
+      traefik.http.routers.serge.rule: Host(`${APP_DOMAIN}`)
+      traefik.http.routers.serge.entrypoints: websecure
+      traefik.http.routers.serge.service: serge
+      traefik.http.routers.serge.tls.certresolver: myresolver
+      # Local domain
+      traefik.http.routers.serge-local-insecure.rule: Host(`serge.${LOCAL_DOMAIN}`)
+      traefik.http.routers.serge-local-insecure.entrypoints: web
+      traefik.http.routers.serge-local-insecure.service: serge
+      traefik.http.routers.serge-local-insecure.middlewares: serge-web-redirect
+      # Local domain secure
+      traefik.http.routers.serge-local.rule: Host(`serge.${LOCAL_DOMAIN}`)
+      traefik.http.routers.serge-local.entrypoints: websecure
+      traefik.http.routers.serge-local.service: serge
+      traefik.http.routers.serge-local.tls: true

apps/serge/metadata/description.md

@@ -0,0 +1,82 @@
+# Serge - LLaMA made easy 🦙
+
+Serge is a chat interface crafted with [llama.cpp](https://github.com/ggerganov/llama.cpp) for running Alpaca models. No API keys, entirely self-hosted!
+
+- 🌐 **SvelteKit** frontend
+- 💾 **[Redis](https://github.com/redis/redis)** for storing chat history & parameters
+- ⚙️ **FastAPI + LangChain** for the API, wrapping calls to [llama.cpp](https://github.com/ggerganov/llama.cpp) using the [python bindings](https://github.com/abetlen/llama-cpp-python)
+
+
+## 🧠 Supported Models
+
+| Category      | Models |
+|:-------------:|:-------|
+| **Alpaca 🦙** | Alpaca-LoRA-65B, GPT4-Alpaca-LoRA-30B |
+| **Chronos 🌑**| Chronos-13B, Chronos-33B, Chronos-Hermes-13B |
+| **GPT4All 🌍**| GPT4All-13B |
+| **Koala 🐨**  | Koala-7B, Koala-13B |
+| **LLaMA 🦙**  | FinLLaMA-33B, LLaMA-Supercot-30B, LLaMA2 7B, LLaMA2 13B, LLaMA2 70B |
+| **Lazarus 💀**| Lazarus-30B |
+| **Nous 🧠**   | Nous-Hermes-13B |
+| **OpenAssistant 🎙️** | OpenAssistant-30B |
+| **Orca 🐬**   | Orca-Mini-v2-7B, Orca-Mini-v2-13B, OpenOrca-Preview1-13B |
+| **Samantha 👩**| Samantha-7B, Samantha-13B, Samantha-33B |
+| **Vicuna 🦙** | Stable-Vicuna-13B, Vicuna-CoT-7B, Vicuna-CoT-13B, Vicuna-v1.1-7B, Vicuna-v1.1-13B, VicUnlocked-30B, VicUnlocked-65B |
+| **Wizard 🧙** | Wizard-Mega-13B, WizardLM-Uncensored-7B, WizardLM-Uncensored-13B, WizardLM-Uncensored-30B, WizardCoder-Python-13B-V1.0 |
+
+Additional weights can be added to the `serge_weights` volume using `docker cp`:
+
+```bash
+docker cp ./my_weight.bin serge:/usr/src/app/weights/
+```
+
+## ⚠️ Memory Usage
+
+LLaMA will crash if you don't have enough available memory for the model:
+
+| Model       | Max RAM Required |
+|-------------|------------------|
+| 7B          | 4.5GB            |
+| 7B-q2_K     | 5.37GB           |
+| 7B-q3_K_L   | 6.10GB           |
+| 7B-q4_1     | 6.71GB           |
+| 7B-q4_K_M   | 6.58GB           |
+| 7B-q5_1     | 7.56GB           |
+| 7B-q5_K_M   | 7.28GB           |
+| 7B-q6_K     | 8.03GB           |
+| 7B-q8_0     | 9.66GB           |
+| 13B         | 12GB             |
+| 13B-q2_K    | 8.01GB           |
+| 13B-q3_K_L  | 9.43GB           |
+| 13B-q4_1    | 10.64GB          |
+| 13B-q4_K_M  | 10.37GB          |
+| 13B-q5_1    | 12.26GB          |
+| 13B-q5_K_M  | 11.73GB          |
+| 13B-q6_K    | 13.18GB          |
+| 13B-q8_0    | 16.33GB          |
+| 33B         | 20GB             |
+| 33B-q2_K    | 16.21GB          |
+| 33B-q3_K_L  | 19.78GB          |
+| 33B-q4_1    | 22.83GB          |
+| 33B-q4_K_M  | 22.12GB          |
+| 33B-q5_1    | 26.90GB          |
+| 33B-q5_K_M  | 25.55GB          |
+| 33B-q6_K    | 29.19GB          |
+| 33B-q8_0    | 37.06GB          |
+| 65B         | 50GB             |
+| 65B-q2_K    | 29.95GB          |
+| 65B-q3_K_L  | 37.15GB          |
+| 65B-q4_1    | 43.31GB          |
+| 65B-q4_K_M  | 41.85GB          |
+| 65B-q5_1    | 51.47GB          |
+| 65B-q5_K_M  | 48.74GB          |
+| 65B-q6_K    | 56.06GB          |
+| 65B-q8_0    | 71.87GB          |
+
+## 🧾 License
+
+[Nathan Sarrazin](https://github.com/nsarrazin) and [Contributors](https://github.com/serge-chat/serge/graphs/contributors). `Serge` is free and open-source software licensed under the [MIT License](https://github.com/serge-chat/serge/blob/master/LICENSE).
+
+## 💬 Support
+
+Need help? Join our [Discord](https://discord.gg/62Hc6FEYQH)