chore(ollama-cpu): small config adjustments

This commit is contained in:
Nicolas Meienberger 2024-05-11 11:10:25 +02:00
parent fe2e985564
commit 11961342e5
2 changed files with 10 additions and 16 deletions

View File

@ -5,19 +5,17 @@ services:
image: ollama/ollama:0.1.33 image: ollama/ollama:0.1.33
restart: unless-stopped restart: unless-stopped
container_name: ollama-cpu container_name: ollama-cpu
environment:
- PORT=11436
ports: ports:
- '${APP_PORT}:11436' - '${APP_PORT}:11434'
networks: networks:
- tipi_main_network - tipi_main_network
volumes: volumes:
- ${APP_DATA_DIR}/.ollama:/root/.ollama - ${APP_DATA_DIR}/data/.ollama:/root/.ollama
labels: labels:
# Main # Main
traefik.enable: true traefik.enable: true
traefik.http.middlewares.ollama-cpu-web-redirect.redirectscheme.scheme: https traefik.http.middlewares.ollama-cpu-web-redirect.redirectscheme.scheme: https
traefik.http.services.ollama-cpu.loadbalancer.server.port: 11436 traefik.http.services.ollama-cpu.loadbalancer.server.port: 11434
# Web # Web
traefik.http.routers.ollama-cpu-insecure.rule: Host(`${APP_DOMAIN}`) traefik.http.routers.ollama-cpu-insecure.rule: Host(`${APP_DOMAIN}`)
traefik.http.routers.ollama-cpu-insecure.entrypoints: web traefik.http.routers.ollama-cpu-insecure.entrypoints: web

View File

@ -1,11 +1,9 @@
# Ollama - CPU
[Ollama](https://github.com/ollama/ollama) allows you to run open-source large language models, such as Llama 3 & Mistral, locally. Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile.
---
## Usage ## Usage
⚠️ This app runs on port **11436**. Take this into account when configuring tools connecting to the app.
### Use with a frontend ### Use with a frontend
- [LobeChat](https://github.com/lobehub/lobe-chat) - [LobeChat](https://github.com/lobehub/lobe-chat)
- [LibreChat](https://github.com/danny-avila/LibreChat) - [LibreChat](https://github.com/danny-avila/LibreChat)
- [OpenWebUI](https://github.com/open-webui/open-webui) - [OpenWebUI](https://github.com/open-webui/open-webui)
@ -14,9 +12,11 @@
--- ---
### Try the REST API ### Try the REST API
Ollama has a REST API for running and managing models. Ollama has a REST API for running and managing models.
**Generate a response** **Generate a response**
```sh ```sh
curl http://localhost:11434/api/generate -d '{ curl http://localhost:11434/api/generate -d '{
"model": "llama3", "model": "llama3",
@ -25,6 +25,7 @@ curl http://localhost:11434/api/generate -d '{
``` ```
**Chat with a model** **Chat with a model**
```sh ```sh
curl http://localhost:11434/api/chat -d '{ curl http://localhost:11434/api/chat -d '{
"model": "llama3", "model": "llama3",
@ -33,16 +34,11 @@ curl http://localhost:11434/api/chat -d '{
] ]
}' }'
``` ```
---
### Try in terminal
```sh
docker exec -it ollama-cpu ollama run llama3 --verbose
```
--- ---
## Model library ## Model library
Ollama supports a list of models available on [ollama.com/library](https://ollama.com/library 'ollama model library') Ollama supports a list of models available on [ollama.com/library](https://ollama.com/library 'ollama model library')
Here are some example models that can be downloaded: Here are some example models that can be downloaded: