# AbstractGateway Server Image

This image packages the AbstractGateway HTTP/SSE server for durable
AbstractRuntime runs:

```bash
ghcr.io/lpalbou/abstractgateway-server:0.2.8
```

Release images are published for `linux/amd64` and `linux/arm64`.
A separate experimental full NVIDIA image is published best-effort for
`linux/amd64`:

```bash
ghcr.io/lpalbou/abstractgateway-server-nvidia:0.2.8
```

The image installs:

```bash
abstractgateway==<version>
```

The default image uses the base remote-light server install. It includes
`AbstractRuntime[multimodal]`, AbstractCore
remote/commercial provider support, OpenAI-compatible text provider routing,
workflow-backed and direct image generation through Runtime/Core/AbstractVision,
direct Gateway voice/audio endpoints through AbstractVoice, provider/session
prompt-cache helpers, KG memory via AbstractMemory/LanceDB, media parsing,
tool helpers, token counting, compression helpers, FastAPI/Uvicorn,
AbstractAgent, and AbstractFlow compatibility. It
intentionally does not bundle local model runtimes such as MLX, vLLM,
HuggingFace Transformers, local Diffusers/sdcpp vision backends, or local
voice/music generation engines.

The NVIDIA image installs `abstractgateway[gpu]` on a CUDA/PyTorch base and
adds vLLM/HuggingFace, local Diffusers image generation, and local
voice engines. It is experimental until a real CUDA host build/smoke gate is
part of release validation. Apple MLX inference is not packaged as a Docker image because
Linux containers do not have access to Apple's Metal/MLX runtime. For Apple
Silicon, run the MLX-capable service natively on macOS and point this container
at its OpenAI-compatible endpoint. Docker Model Runner is reachable from
containers at `http://model-runner.docker.internal/engines/v1`; LM Studio can
serve local MLX models on `http://localhost:1234/v1`; Ollama can serve its
OpenAI-compatible API on `http://localhost:11434/v1`, including builds/models
that route through its native MLX runner.

## Run

Keep secrets in an uncommitted env file:

```bash
ABSTRACTGATEWAY_AUTH_TOKEN=replace-with-a-gateway-token
ABSTRACTGATEWAY_PROVIDER=openai-compatible
ABSTRACTGATEWAY_MODEL=your-model
OPENAI_API_KEY=sk-...
OPENROUTER_API_KEY=sk-or-...
ANTHROPIC_API_KEY=sk-ant-...
PORTKEY_API_KEY=pk_...
PORTKEY_CONFIG=pcfg_...
OPENAI_COMPATIBLE_BASE_URL=http://host.docker.internal:1234/v1
OPENAI_COMPATIBLE_API_KEY=optional
ABSTRACTVISION_BASE_URL=https://api.openai.com/v1
ABSTRACTVISION_API_KEY=optional
ABSTRACTVISION_MODEL_ID=gpt-image-1
ABSTRACTVOICE_TTS_ENGINE=openai
ABSTRACTVOICE_STT_ENGINE=openai
ABSTRACTVOICE_REMOTE_BASE_URL=
ABSTRACTVOICE_REMOTE_API_KEY=
```

Then run:

```bash
docker run --rm --name abstractgateway-server \
  -p 127.0.0.1:8080:8080 \
  --env-file .env \
  -v "$PWD/runtime/gateway:/data/gateway" \
  -v "$PWD/flows/bundles:/data/flows:ro" \
  -v "$PWD/workspace:/workspace" \
  ghcr.io/lpalbou/abstractgateway-server:0.2.8
```

`ABSTRACTGATEWAY_AUTH_TOKEN` is the gateway bearer token. Clients send it as
`Authorization: Bearer <token>`, including from Swagger UI's `Authorize`
button. Provider keys stay inside the container.

For local OpenAI-compatible text endpoints such as Docker Model Runner, LM
Studio, `mlx_lm.server`, or Ollama's `/v1` server, point the container at a URL
reachable from Docker:

```bash
docker run --rm --name abstractgateway-server \
  -p 127.0.0.1:8080:8080 \
  -e ABSTRACTGATEWAY_AUTH_TOKEN="$ABSTRACTGATEWAY_AUTH_TOKEN" \
  -e ABSTRACTGATEWAY_PROVIDER="openai-compatible" \
  -e ABSTRACTGATEWAY_MODEL="your-model" \
  -e OPENAI_COMPATIBLE_BASE_URL="http://host.docker.internal:1234/v1" \
  -e OPENAI_COMPATIBLE_API_KEY="$OPENAI_COMPATIBLE_API_KEY" \
  ghcr.io/lpalbou/abstractgateway-server:0.2.8
```

Use `http://model-runner.docker.internal/engines/v1` for Docker Model Runner,
or `http://host.docker.internal:11434/v1` when routing through Ollama's
OpenAI-compatible server. These text endpoints are not assumed to provide
`/v1/images/generations`.

## Docker Compose

```bash
cp docker/abstractgateway-server/.env.example docker/abstractgateway-server/.env
docker compose --env-file docker/abstractgateway-server/.env \
  -f docker/abstractgateway-server/compose.yml up -d
```

On an NVIDIA host with the NVIDIA Container Toolkit, use the experimental
overlay:

```bash
docker compose --env-file docker/abstractgateway-server/.env \
  -f docker/abstractgateway-server/compose.yml \
  -f docker/abstractgateway-server/compose.nvidia.yml up -d
```

Useful compose variables:

- `ABSTRACTGATEWAY_PORT`: host port, default `8080`
- `ABSTRACTGATEWAY_HOST_FLOWS_DIR`: host directory containing `*.flow` bundles
- `ABSTRACTGATEWAY_STORE_BACKEND`: `file` or `sqlite`
- `ABSTRACTGATEWAY_MEMORY_STORE_BACKEND`: `lancedb` or `memory`; `sqlite` works when the installed AbstractMemory build exposes `SQLiteTripleStore`
- `ABSTRACTGATEWAY_PROVIDER` / `ABSTRACTGATEWAY_MODEL`: defaults for bundles
- `ABSTRACTGATEWAY_TOOL_MODE`: `approval`, `passthrough`, `delegated`, or local dev modes
- `ABSTRACTGATEWAY_EMBEDDING_PROVIDER` / `ABSTRACTGATEWAY_EMBEDDING_MODEL`: optional embedding space
- `ABSTRACTVISION_*`: AbstractVision image backend or OpenAI-compatible image endpoint
- `ABSTRACTVOICE_*`: AbstractVoice TTS/STT backend, local/remote engine, and model controls
- `ABSTRACTGATEWAY_EXTRAS`: build-time install extra for local image builds (empty for the default image, `gpu` for NVIDIA)
- `ABSTRACTGATEWAY_ABSTRACTCORE_SERVER_BASE_URL`: optional standalone Core server for dynamic voice/vision catalogs

Release scope: TTS, STT, and generated images are direct Gateway endpoints.
Generated images are also available through Runtime/Core workflows with
AbstractVision installed and configured.

For unreleased local checkouts, build the image from this repository:

```bash
ABSTRACTGATEWAY_INSTALL_MODE=local \
ABSTRACTGATEWAY_IMAGE_TAG=0.2.8-local \
docker compose -f docker/abstractgateway-server/compose.yml up -d --build
```

## Smoke Checks

```bash
curl http://localhost:8080/api/health

curl -H "Authorization: Bearer $ABSTRACTGATEWAY_AUTH_TOKEN" \
  http://localhost:8080/api/gateway/bundles
```
