datastax/astra-assistants repository overview

⁠Astra Assistant API Service

A drop-in compatible service for the latest OpenAI Assistants API v2 (with streaming) with support for persistent threads, files, vector_stores, assistants, retreival, function calling and more using AstraDB⁠ (DataStax's db as a service offering powered by Apache Cassandra⁠ and jvector⁠).

Supports dozens of third party LLM providers (or even local models) for both completion and embeddings (powered by LiteLLM⁠).

You can use our hosted Astra Assistants service, or host the open source API server yourself.

⁠Run with Docker

with docker, first pull the image from docker hub

docker pull datastax/astra-assistants

or a specific version if you don't want latest:

docker pull datastax/astra-assistants:v0.2.18

then run (-p to map your docker port 8080 to your host port 8080):

docker run -p 8080:8080 datastax/astra-assistants

⁠Docker-compose with ollama

For with docker-compose for integration with ollama clone the repo:

git clone [email protected]:datastax/astra-assistants-api.git

and run the docker compose under examples:

cd examples/ollama/gpu # or examples/ollama/cpu for cpu only for gpu you need docker-toolkit

docker-compose up -d

you need to pull the model you want to ollama before using it

curl http://localhost:11434/api/pull -d '{ "name": "deepseek-coder-v2" }'

your assistants client should route to the ollama container setting OLLAMA_API_BASE_URL. OLLAMA_API_BASE_URL should be set to http://ollama:11434⁠ if you are using docker-compose. If you are using ollama on your localhost you can set it to http://localhost:11434⁠