Inference API

Updated 12 October 2026

Reviewed against Console v0.2.2. Available actions depend on project access and runtime configuration; check your installed CLI version for command differences.

Choose and test a route in Console#

1. Open Models and select an enabled model for the required modality. Click its row to inspect its exact identifier, provider, and compute status.
2. Load sample fills the model-test form. Run model test makes an inference request and may consume tokens or provider credit.
3. Open Traces in the intended project and inspect the request status, captured payload, route, and usage.

A Needs compute model requires setup first. See model readiness and first request. A metering-only trace is useful for accounting but is not captured evaluation input/output.

Discover a model first#

stimulir models

Model availability depends on the workspace, routing configuration, modality and provider access. Use an exact returned identifier instead of copying a model name from an old example.

Chat completions#

curl https://api.stimulir.com/api/v1/inference/chat/completions \
  -H "Authorization: Bearer $STIMULIR_API_KEY" \
  -H "X-Project-Id: $STIMULIR_PROJECT_ID" \
  -H "Content-Type: application/json" \
  --data-binary @request.json

{
  "model": "<enabled-model-id>",
  "messages": [{"role": "user", "content": "Say hello"}],
  "tags": ["first-call"]
}

Set stream: true for supported streaming responses. Keep conversation history in messages. Read the HTTP status and error body; receiving a response does not necessarily mean the model completed successfully.

Use the OpenAI client#

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.stimulir.com/api/v1/inference",
    api_key=os.environ["STIMULIR_API_KEY"],
    default_headers={"X-Project-Id": os.environ["STIMULIR_PROJECT_ID"]},
)
response = client.chat.completions.create(
    model=os.environ["STIMULIR_MODEL"],
    messages=[{"role": "user", "content": "Say hello"}],
    extra_body={"tags": ["first-call"]},
)

Keep attributable evidence#

Use stable workflow tags and the actual prompt key, version and label when you manage prompts in Stimulir. Do not invent lineage for an unregistered prompt. The project header must identify a project accessible to the key. Inspect the corresponding trace before beginning an evaluation.

Other request types#

Select a model that supports the requested input and output. Chat compatibility does not make every provider support every parameter, image format or audio operation. Realtime voice uses a separate websocket lifecycle.

Send a CLI request#

stimulir infer chat "Say hello" --model <enabled-model-id> --stream

Choose a model first and inspect usage after testing.

Hosted gateway versus a direct runtime#

The hosted origin routes and meters application traffic. A separately operated runtime has its own endpoint, model inventory and security configuration. Do not replace the hosted base URL with a localhost example unless you are operating that runtime. Deployment modes explains the distinction.

Command catalogue#

Expand a command for its arguments. These commands follow the released CLI source; check stimulir --version and the command’s --help for your installation. Required flags are marked; some operations also validate combinations of inputs.