Evals

#Where eval cases come from

An eval case is a saved { input, expected output } pair for one agent. The main way to create one is via feedback: submitting corrected output on a finished run (see Runs lifecycle & statuses) saves that run's input alongside your correction, unless you opt out:

bash

curl -X POST "https://api.agentflowbind.com/v1/runs/5e6a7b8c-...-9d0e/feedback" \
  -H "Authorization: Bearer afb_live_xxxxxxxxxxxx" -H "Content-Type: application/json" \
  -d '{ "rating": 3, "correctedOutput": { "total": 1249.50 }, "useAsExample": true }'

Pass "useAsExample": false if a correction is a one-off and shouldn't become a permanent regression case.

#Running the eval suite

bash

curl -X POST "https://api.agentflowbind.com/v1/agents/{id}/evals" \
  -H "Authorization: Bearer afb_live_xxxxxxxxxxxx" -H "Content-Type: application/json" \
  -d '{}'

json

{ "batchId": "9d0e5e6a-...", "status": "queued", "estimateCredits": 210 }

This runs every saved eval case for the agent through the kernel using its current profile version (pass "versionId" in the body to evaluate a specific past version instead — useful for confirming a prompt/schema change didn't regress accuracy before making it current). Each case is priced and billed the same as a real run against the agent, so the endpoint pre-checks the workspace balance can cover cases × per-case estimate and returns 402 billing.insufficient_credits up front if not — no partial batch is started that then runs out of credit partway through.

#Reading results

bash

# Every batch this agent has run, newest first
curl "https://api.agentflowbind.com/v1/agents/{id}/evals" -H "Authorization: Bearer afb_live_xxxxxxxxxxxx"

# One batch's per-case detail
curl "https://api.agentflowbind.com/v1/agents/{id}/evals/9d0e5e6a-..." -H "Authorization: Bearer afb_live_xxxxxxxxxxxx"

json

{
  "batchId": "9d0e5e6a-...",
  "profileVersionId": "3c1a9e40-...",
  "accuracy": 0.92,
  "cases": [
    { "caseId": "...", "passed": true, "actualOutput": { "...": "..." }, "expectedOutput": { "...": "..." }, "costCredits": 12 }
  ],
  "createdAt": "2026-03-14T09:00:00.000Z"
}

GET /v1/agents/:id also surfaces the most recent batch's accuracy as lastEval, so you can see an agent's trend without fetching every batch's detail.