Engineering Workspace

Inference API

OpenAI-compatible chat completions through the console, routed to your BYOK providers or Managed Inference, plus the HybrIE runtime endpoints for BYOC deployments.

Models

The model string decides the route. Managed inference runs on Stimulir's own capacity with no key, send the bare name, or prefix with stimulir/ to pin managed even when you hold a BYOK credential. A provider prefix routes to your own BYOK credential instead. Prices are reference rates in USD per 1M tokens (input · output); see Usage & Billing for current pricing.

Managed inference, no key

Model	Served by	$/1M in · out
`stimulir/claude-opus-4-6`	Claude Opus 4.6 frontier, platform Bedrock	15 · 75
`stimulir/claude-sonnet-4-6`	Claude Sonnet 4.6 frontier, platform Bedrock	3 · 15
`moonshotai/Kimi-K3`	Kimi K3, MI engine	MI rate
`moonshotai/Kimi-K2.6`	Kimi K2.6 (vision), MI engine	MI rate
`moonshotai/Kimi-K2.7-Code`	Kimi K2.7 Code, MI engine	MI rate
`MiniMaxAI/MiniMax-M3`	MiniMax M3, MI engine	MI rate
`MiniMaxAI/MiniMax-M2.5`	MiniMax M2.5, MI engine	MI rate
`Qwen/Qwen2.5-VL-72B-Instruct`	Qwen2.5-VL 72B (vision), MI engine	MI rate
`stimulir/fusion`	Panel + judge over the open models	sum of parts
`stimulir/fusion-max`	Panel + judge over the frontier models	sum of parts

Managed frontier is Claude today (claude-opus-4-6, claude-sonnet-4-6). Open models are managed on their bare vendor name; the stimulir/ prefix is optional there but forces managed on any name. See Fusion for the recipe presets.

BYOK, bring your key

Prefix a model with your provider to route through your own credential. BYOK is not an allowlist, any model your provider serves works; the tables below are the metered reference rates.

openai/

Model	$/1M in · out
`gpt-4o`	2.5 · 10
`gpt-4o-mini`	0.15 · 0.6
`gpt-4o-realtime`	5 · 20
`o1`	15 · 60
`o1-mini`	3 · 12
`o3-mini`	3 · 12

anthropic/, Claude, direct

Model	$/1M in · out
`claude-opus-4`	15 · 75
`claude-sonnet-4.6`	3 · 15
`claude-sonnet-4.5`	3 · 15
`claude-haiku-4.5`	0.8 · 4

bedrock/ or vertex/, Claude, via cloud

Model	$/1M in · out
`claude-opus-4-8`	15 · 75
`claude-opus-4-7`	15 · 75
`claude-opus-4-6`	15 · 75
`claude-sonnet-4-6`	3 · 15
`claude-3-7-sonnet`	3 · 15
`claude-3-5-sonnet`	3 · 15
`claude-3-5-haiku`	0.8 · 4

google_gemini/ or vertex/, Gemini

Model	$/1M in · out
`gemini-3.5-flash`	1.5 · 9
`gemini-3.1-pro-preview`	2 · 12
`gemini-3.1-flash-lite`	0.25 · 1.5
`gemini-3-pro-preview`	2 · 12
`gemini-3-flash-preview`	0.5 · 3
`gemini-2.5-pro`	1.25 · 5
`gemini-2.5-flash`	0.075 · 0.3
`gemini-2.5-flash-lite`	0.05 · 0.2
`gemini-2.5-flash-native-audio-preview-12-2025`	0.5 · 2

mistral/

Model	$/1M in · out
`mistral-large-2.5`	2 · 6
`mistral-medium-3.1`	0.4 · 2
`mistral-small-3.2`	0.2 · 0.6
`codestral-2.5`	0.3 · 0.9
`pixtral-large-2.5`	2 · 6

Vision is per-request: send image parts to any chat model on any lane. The Gemini, GPT-4o, and Claude families accept images, plus pixtral-large-2.5, moonshotai/Kimi-K2.6, and Qwen/Qwen2.5-VL-72B-Instruct (the managed hybrie-vision tier). Realtime voice runs on Gemini native-audio,gpt-realtime, and Qwen2.5-Omni, see Realtime Voice. stimulir/ is the only managed prefix; see Deployment Modes for BYOK vs Managed vs BYOC.

Console inference endpoint

POST/api/v1/inference/chat/completions

Drop-in OpenAI-compatible chat completions at https://api.stimulir.com. Authenticate with a hyb_* API key:

curl

curl https://api.stimulir.com/api/v1/inference/chat/completions \
  -H "Authorization: Bearer hyb_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-4-6",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

A system prompt and full conversation history go in the same messages array, standard OpenAI shape, one call:

curl, system prompt + history

curl https://api.stimulir.com/api/v1/inference/chat/completions \
  -H "Authorization: Bearer hyb_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "stimulir/claude-sonnet-4-6",
    "messages": [
      {"role": "system",    "content": "You are a concise assistant."},
      {"role": "user",      "content": "earlier question"},
      {"role": "assistant", "content": "earlier answer"},
      {"role": "user",      "content": "follow-up"}
    ],
    "max_tokens": 800
  }'

Requests are routed by model prefix: models matching one of your BYOK credentials are sent upstream with your own provider key; everything else is served by Managed Inference. Usage is metered per request, see Usage & Billing.

Python

The Python path for inference is handled directly by the Stimulir SDK. Use client.agent() to execute one-shot prompts against the agent. Before execution, the system checks your Bring Your Own Key (BYOK) settings. If a matching key is found for the requested model, it uses it; otherwise, it falls back to Managed Inference. For a system prompt plus conversation history in one call, use client.request() with a full messages array, see Multi-turn conversations.

chat completions

from stimulir import StimulirClient

client = StimulirClient(
    api_key="hyb_...",  # or os.environ["STIMULIR_API_KEY"]
    project_id="<project-id>",  # optional: project attribution
)

response = client.agent(
    prompt="Draft a runbook for failed payouts",
    role="user",
    model="moonshotai/Kimi-K3",  # any Stimulir-routed model, incl. stimulir/fusion
)

if response.error:
    print(f"Error: {response.error}")
else:
    print(response.content)

vision

response = client.agent(
    prompt=[
        {"type": "text", "text": "What does this diagram show?"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}
    ],
    role="user",
    model="moonshotai/Kimi-K2.6",  # or stimulir/fusion (vision perceiver)
)

Oversized images are downscaled and context-budgeted by the gateway before they reach the model, so full-resolution screenshots cannot overflow a model's context window. Inputs that exceed a model's confirmed context window return a clean 400 with code context_length_exceeded.

Trace tags

Add tags to inference requests when you want runs to be searchable by tenant, product surface, experiment, customer, or workflow. Tags are stored on the usage trace, shown in the Engineering workspace trace table, and available as a dropdown filter as new tags appear.

curl with trace tags

curl https://api.stimulir.com/api/v1/inference/chat/completions \
  -H "Authorization: Bearer hyb_..." \
  -H "X-Project-Id: <project-id>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-2.5-flash",
    "messages": [{"role": "user", "content": "Summarize this assessment response"}],
    "tags": ["intervia", "assessment", "question-plan"]
  }'

Stimulir SDK

from stimulir import StimulirClient

client = StimulirClient(
    api_key="hyb_...",
    project_id="<project-id>",
)

response = client.agent(
    prompt="Summarize this assessment response",
    role="user",
    model="gemini-2.5-flash",
    tags=["intervia", "assessment", "question-plan"]
)

If your existing client already has a metadata object, Stimulir also reads metadata.tags, metadata.stimulir.tags, and stimulir.tags. In Console, open Engineering → Traces and filter by tag.

From the CLI

bash

stimulir infer chat "Draft a runbook for failed payouts" --model claude-opus-4-6 --stream
stimulir models

HybrIE runtime endpoints

In BYOC deployments you run the HybrIE runtime yourself. It serves an OpenAI-compatible HTTP API on port 8080 (gRPC on 9090).

Chat completions

POST/v1/chat/completions

Streaming chat completions. Local models (Qwen3 / Qwen3-Coder) run via Candle on Metal or CUDA, with cloud fallback to OpenAI, Anthropic, Gemini, or Mistral when configured. Supported sampling parameters:

Parameter	Type	Description
`temperature`	float	Sampling temperature.
`top_p`	float	Nucleus sampling probability mass.
`top_k`	integer	Top-k sampling cutoff.
`thinking_mode`	boolean	Enable model thinking/reasoning traces where the model supports it.

Routing headers

Runtime requests can be steered per request with x-hybrie-* headers (v0.1.65):

Parameter	Type	Description
`x-hybrie-adapter-id`	string	Serve the request through a loaded LoRA adapter, no restart. Also settable as gRPC metadata `hybrie.adapter_id`; the header wins. See Hot-swap Inference for loading and adapter details.
`x-hybrie-execution-mode`	string	`local`, `cloud`, or `hybrid`, where the request executes.
`x-hybrie-cloud-provider`	string	Cloud provider to use when execution goes to the cloud.
`x-hybrie-preferred-peer-id`	string	Route the request to a specific registered compute peer.

Multimodal (vision)

Multimodal models accept image content parts in the OpenAI image_url shape. The runtime currently advertises three multimodal models, claude-sonnet-4-6, gemini-3-flash-preview, and gpt-4.1-mini; check GET /v1/models for the live list.

vision request body

{
  "model": "gpt-4.1-mini",
  "messages": [
    { "role": "user", "content": [
      { "type": "text", "text": "What is in this image?" },
      { "type": "image_url",
        "image_url": { "url": "https://example.com/receipt.png" } }
    ] }
  ]
}

Embeddings

POST/v1/embeddings

OpenAI-compatible embeddings.

Audio

POST/v1/audio/transcriptions

Speech-to-text using local Whisper.

POST/v1/audio/speech

Text-to-speech. The local Qwen2.5-Omni voice model isExperimental

Qwen2.5-Omni also serves the bidirectional voice loop, see Realtime Voice.

Models

GET/v1/models

List models available on the runtime.

Managed inference, no key

BYOK, bring your key

Console inference endpoint#

Python#

Trace tags#

From the CLI#

HybrIE runtime endpoints#

Chat completions

Routing headers

Multimodal (vision)

Embeddings

Audio

Models

Console inference endpoint

Python

Trace tags

From the CLI

HybrIE runtime endpoints