Gate responses that fail quality thresholds
LLM-as-judge scores every response server-side. Agentic evaluation judges whole runs: tool choice, trajectory, and multi-agent coordination.
Fluiq/ testsTotal Evals
847
across 312 traces
Avg Score
0.91
threshold ≥ 0.7
Pass Rate
88.4%
749 / 847 passed
By Metric
What you get
Metrics
Six judge metrics
Hallucination, faithfulness, relevance, toxicity, coherence, and completeness, scored per response.
Thresholds
Per-metric gates
Set a threshold for each metric. Warn mode logs the score; block mode stops the response.
Agentic
Whole-run evaluation
Layered judging of a full agent run: deterministic checks, tool-selection quality, trajectory against the goal, and multi-agent coordination across fan-outs and joins.
Jury
Multi-model judge panel
Borderline verdicts convene a jury of different judge models and aggregate their votes, with every member's score and reasoning kept for audit.
Your call
Pick the judge, or the whole jury
Choose which model scores a run, and which models sit on the panel, per evaluation or per dataset batch.
Your keys
Bring your own provider key
Save an OpenAI, Anthropic, or Google key and judge tokens bill to your account at the rate you already negotiated.
Auditable
Every score shows its prompt
Each result carries the exact judge prompt and version that produced it, so a shifting metric is traceable to a prompt change.
The judge runs server-side. Opt in with one call.
- instrument() only traces; scoring starts when you call fluiq.eval()
- Warn mode logs scores; block mode raises FluiqEvalError below threshold
- Run Agentic Eval on any root trace to judge the whole run: tools, trajectory, coordination
- CI gates run the same checks in GitHub Actions
import fluiq
fluiq.instrument(api_key="fl_...")
fluiq.eval(
metrics=["hallucination", "relevance", "toxicity"],
thresholds={"hallucination": 0.8, "relevance": 0.75},
mode="warn", # "block" raises FluiqEvalError
)Part of the Fluiq platform
Compare plansUnlimited traces, always free.
Tracing, security, evaluation, datasets, and prompts all run on the free tier. Caching and Slack alerts unlock on a paid plan.