Diagnose real failures
Start with recorded behavior instead of guessing which instruction or model to change.
AI evaluation for production agents
Stop repeating AI failures.
Turn real traces into evaluation datasets, compare prompts and models, catch regressions, and require approval before promotion.
7 days of Pro. No card required. Then keep 20 free traces per month.
Your test cases can’t predict every real request.
Pre-launch tests cover the cases you thought to write. Real traffic brings new wording, missing context and unexpected combinations. Capture those failures, make them repeatable, and use the same evidence to test a proposed fix.
The workflow
From a failed trace to a reviewed change.
Capture real traces
Open Traces to inspect recorded inputs, outputs, model, latency and cost where available. Find a recurring answer-quality failure in your own workflow.
Inspect Console tracesCurate failures into datasets
Choose eligible gateway text traces or a supported dataset, check sensitive data and define what a good answer must do. Keep a fixed dataset for the comparison.
Prepare evaluation dataEvaluate against a baseline
Use Lab to evaluate the current behavior against that dataset and explicit criteria. Resolve missing or ineligible evidence before comparing.
Define the baselineCompare prompt and model candidates
Change one thing, then inspect candidate and incumbent results on comparable cases. Look for regressions as well as quality, latency and cost.
Compare the evidenceApprove promotion
Review the proposal and its limits. Promotion is a separate decision: approve a supported change only when the evidence justifies it.
Review the promotion boundary
Catch regressions before promotion
Reuse comparable cases and task criteria to see where a candidate improves, stays the same or gets worse.
Keep a human decision
Evaluation produces evidence. A reviewer decides whether a change should reach production.
Inside the Console
See the tools behind the workflow.
Actual Console components, rendered with clearly labeled sample data. These show the controls you use, not a measured result.
Music: Happy Beats & Business Moves Vol. 12 by Sascha Ende, CC BY 4.0. Excerpt mixed with fades.
Read the video walkthrough
- Build: choose the model, skills and instructions; keep changes in an explicit version.
- Connect: on a supported Mac, download and open Auto, approve your project in Console and keep Auto open. The demo shows setup, not a connected device. Advanced CLI setup is separate.
- Observe: inspect the recorded request and answer. A completed call can still give the wrong answer.
- Curate: choose eligible cases, review sensitive data and keep a fixed evaluation dataset.
- Compare: compare a baseline with a candidate on the same cases. No results are claimed in this setup demo.
- Decide: review the evidence before approving a supported promotion. You decide what goes live.
Inspect the recorded request.
The Console trace list keeps model, status and timing together. Field availability depends on how the request was captured.
Set up a fair comparison.
Choose the task and candidates in Lab. A setup screen is not a measured improvement claim.
Review the next step in Lab.
Keep dataset preparation, evaluation and the promotion decision visible as separate steps.
Build with control.
Improve with evidence.
Pricing
Build your first agent.
Improve the one you have.
One workspace to build, trace, evaluate and approve. Start with 7 days of Pro, then choose what fits.
Founding offer · first 10 paying teams
Pro
$50/ workspace / month
Regularly $100/month. Keep $50/month for your first 12 paid months, then $100/month.
From a first visual agent to a better production workflow, with evidence behind every change.
Start 7-day free trial
No card required. After the trial, keep 20 free traces per month unless you upgrade.
$25
Monthly managed-model credit
Included with paid Pro. Additional usage billed separately.
Build & connect
- Visual drag-and-drop agent builder
- 1 workspace, up to 5 members and 10 live agents or workflows
- SDK and API access
Run on your terms
- Managed, proprietary and supported open-weight models
- Bring your own provider keys
- 1 supported local or remote compute peer
Measure & improve
- 10,000 traces per month
- 1,000 evaluation executions per month
- Datasets, model comparisons and regression checks
Stay in control
- Version history and lineage
- Human approval and controlled promotion
- Community and email support
What happens during and after the trial?
Try Pro for 7 days with no card required: build an agent or connect an existing workflow, inspect traces, evaluate a change and approve the better version. A small starter allowance covers managed-model usage.
Customer-controlled compute peers during the trial are available by arrangement. The $25 monthly managed-model credit starts with paid Pro. If you do not upgrade, continue on Free with 20 traces per month.
What is included, and what is billed separately?
Your subscription covers the workspace, builder and improvement tools within your plan’s limits. Paid Pro includes $25 in managed-model credit each month; additional managed-model usage is billed separately.
With your own provider keys, you pay your provider directly with no Stimulir model markup. Your own compute carries no Stimulir GPU charge; normal platform limits still apply. If Stimulir provisions rented compute for you, provider costs and any orchestration margin are disclosed separately.
How does the founding price work?
The first 10 paying teams keep Pro at $50 per workspace per month for their first 12 paid months. After that, Pro is $100 per workspace per month. The free trial does not start your 12 paid months.
All prices in USD, excluding applicable taxes.
Bring one live AI failure.
30 minutes with the founder. Bring one live AI failure; leave with an improvement plan.