Connecting DRAGON to an LLM

DRAGON's copilot is model-agnostic. It talks to any OpenAI-compatible /v1/chat/completions endpoint — a model running locally on your workstation, one on a server in your enclave, or a cloud API — and speaks Anthropic's native Messages API when pointed at Claude. You choose the model; DRAGON handles the prompting, the JSON contract, secret redaction, and the audit trail.

Everything is configured in one place: Settings → AI → Model & inference. Pick a Provider preset — LM Studio, Ollama, ChatGPT (OpenAI), Claude (Anthropic), or Custom endpoint — and the endpoint URL pre-fills; the Model and Embed model pickers then populate from live model discovery at that endpoint, and the key field is hidden for local backends that don't need one.

Air-gapped / classified networks: run a local model (LM Studio or Ollama). Nothing leaves the workstation. Cloud providers (OpenAI, Claude, Perplexity) require outbound internet and send your redacted session context off-device — only use them where egress is authorized. See Security & redaction.

How the connection works

• DRAGON sends chat requests to <Endpoint URL>/chat/completions. The core automatically appends /v1 if you leave it off, so http://localhost:1234 and http://localhost:1234/v1 are equivalent.
• An endpoint on anthropic.com is auto-detected and served by DRAGON's native Claude client — Anthropic's /v1/messages API with the x-api-key header — no OpenAI-compat shim involved.
• The API key (when required) is sent as a Bearer token (x-api-key for Anthropic), stored in your OS keychain, and never written to disk or shown in the UI.
• Before any request leaves the process, DRAGON redacts secrets from the prompt (passwords, keys, community strings). Remote endpoints get a stricter redaction policy than local ones.
• The same endpoint serves both chat and embeddings. Embeddings power local RAG (indexing your docs/configs). If your provider has no embeddings API (Claude, Perplexity), the copilot still works — vector indexing is unavailable and RAG retrieval degrades to keyword-only search. See Embeddings & RAG.

Quick reference

Pick the matching Provider preset for each of these; use Custom endpoint for anything not listed.

Local models

Local inference is the default recommendation: zero egress, no API bills, works offline, and satisfies air-gapped requirements.

LM Studio

A desktop app with a built-in model catalog and a one-click local server. Best "just works" option on Windows/macOS.

In LM Studio (one-time):

1. Install LM Studio: lmstudio.ai.
2. On the Discover tab, download an instruct model. Good starting points: Qwen2.5 14B Instruct (quality) or Qwen2.5 7B Instruct (lighter). A model that reliably emits JSON matters — instruct-tuned models in the 7B–14B range work well.
3. Go to the Developer tab (the >_ icon) → Start Server. It listens on http://localhost:1234 by default. Confirm your model is loaded (the server panel shows the loaded model's identifier — you'll paste that into DRAGON).
4. (Optional, for RAG) Also download an embedding model such as nomic-embed-text-v1.5; LM Studio serves it from the same port.

• Reference: LM Studio local-server docs.

In DRAGON (Settings → AI):

Click Save AI settings.

Ollama

A lightweight CLI/daemon. Great on Linux and for servers.

On the host (one-time):

1. Install Ollama: ollama.com/download.
2. Pull a chat model:
   bashCopy

   ollama pull qwen2.5:14b        # or llama3.1:8b, mistral-nemo, etc.
   

3. (Optional, for RAG) pull an embedding model:
   bashCopy

   ollama pull nomic-embed-text
   

4. Ollama runs a server on http://localhost:11434 automatically (or run ollama serve). It exposes an OpenAI-compatible API at /v1.

• Reference: Ollama's OpenAI compatibility.

In DRAGON (Settings → AI):

Remote Ollama: if the model runs on another box in your enclave, use its address instead of localhost, e.g. http://10.0.0.20:11434, and make sure Ollama binds a reachable interface (OLLAMA_HOST=0.0.0.0 ollama serve).

Cloud models

Cloud providers give you the strongest models with no local hardware, at the cost of egress and per-token billing. They receive your redacted session context — only use them where that is permitted.

OpenAI (ChatGPT)

Get a key (one-time):

1. Create an account and add billing at platform.openai.com.
2. Generate an API key at platform.openai.com/api-keys. Copy it now — it is shown only once.
3. Pick a model from the OpenAI model list. gpt-4o is a strong default; gpt-4o-mini is cheaper and fast.

In DRAGON (Settings → AI):

Anthropic (Claude)

DRAGON ships a native Claude client: pick the Claude (Anthropic) provider preset and DRAGON speaks Anthropic's Messages API directly — the correct paths, the x-api-key header, token limits, and Anthropic's JSON conventions are handled for you, and the model list is fetched live from your account.

Get a key (one-time):

1. Create an account at console.anthropic.com and add credit.
2. Generate an API key at console.anthropic.com/settings/keys.
3. Pick a model from the Model list DRAGON fetches (or see the Anthropic model list; claude-sonnet-5 is a strong default).

In DRAGON (Settings → AI):

RAG with Claude: Anthropic doesn't serve embeddings, so vector indexing is unavailable at this endpoint — DRAGON disables the embedder and RAG retrieval degrades to keyword-only search. For full vector RAG, use a local model for both chat and embeddings. See Embeddings & RAG.

Perplexity

Perplexity's API is OpenAI-shaped and search-augmented.

1. Get a key at perplexity.ai/settings/api (requires a Pro/API plan).
2. Models are listed at docs.perplexity.ai (e.g. sonar, sonar-pro).

Caveat — not plug-and-play today. Perplexity serves chat at https://api.perplexity.ai/chat/completions without a /v1 segment, but DRAGON currently always appends /v1. Until per-provider endpoint handling lands (roadmap), Perplexity won't connect through the standard endpoint field. Prefer OpenAI or Claude for cloud, or a local model.

Embeddings & RAG

Embeddings are only needed if you use RAG (Settings → AI → RAG corpus sources) to index your own docs, device configs, or session history so the copilot can cite them.

• Embed model must be reachable at the same Endpoint URL as your chat model, at /v1/embeddings.
• Local backends (LM Studio, Ollama) serve both from one port — set the Embed model field and add a corpus.
• Cloud chat providers without an embeddings API (Claude, Perplexity) can't compute vector embeddings at their endpoint — RAG retrieval degrades to keyword-only search there. Options: (a) accept keyword-only retrieval — the copilot works fine; or (b) use a local chat model too — running a cloud model for chat with a separate local embedder for RAG is not currently splittable (one endpoint field), so for full vector RAG use a local chat model as well.
• Recommended embed models: nomic-embed-text (Ollama), text-embedding-nomic-embed-text-v1.5 (LM Studio), text-embedding-3-small (OpenAI).

Leave the Embed model blank if you're not using RAG.

Security & redaction

• DRAGON scrubs secrets from every prompt before it leaves the process, and applies a stricter policy to non-local endpoints. Redactions are recorded in the tamper-evident audit log (the exact post-redaction prompt sent to the model is stored as evidence).
• The API key lives in the OS keychain, is sent only as a Bearer header, and is never echoed to the UI or written to browser storage.
• For classified or air-gapped work, use a local model. Cloud endpoints transmit redacted-but-real session context (commands, device output, your questions) to a third party that may log or retain it. Choose a provider whose data-handling terms match your authorization, and confirm egress is permitted.

Troubleshooting

Recommended starting points

• Air-gapped / DoD workstation: LM Studio + Qwen2.5 14B Instruct (+ nomic-embed-text-v1.5 if you want RAG). No egress, no keys.
• Local server in the enclave: Ollama + qwen2.5:14b, pointed at the server's address.
• Cloud, egress permitted: OpenAI gpt-4o or Anthropic claude-sonnet-5.