Eval

Know if the agent
got it right

Ingest traces, score gold sets, export data. Hosted at eval.phthos.ai or self-host with pip. Eval does not use Gateway or Task AI credits.

Open Eval Eval plans

Ingest

Push traces from Task AI or your own stack with a pk_ key.

Gold sets

Score against labeled examples. Export anytime.

OSS SDK

pip install phthos-eval. Self-host has no volume cap.

Hosted judge

Pro uses Phthos Gateway as a judge — billed on Eval, not console.phthos.ai.

Eval plans

Volume caps, not Phthos credits. Checkout at eval.phthos.ai/billing.

Self-host

$0

  • OSS package
  • No ingest cap
  • You run Postgres
View on PyPI

Hosted Free

$0

  • 2,000 ingest/day
  • 10,000 scores/month
  • 3 seats
  • 30-day retention
Start hosted Free
Pro

Pro

$49/mo

  • 100,000 ingest/day
  • 500,000 scores/month
  • 25 seats
  • 365-day retention
  • Hosted judge + SAML
Start Eval Pro

Usage and plan live at eval.phthos.ai/plan and /usage.