Every quarter, a new evaluation
20 May 2026 · 5 min · HAI Team
The service catalog is not a fixed menu. Every quarter we add new evaluations, and we're deliberate about which ones — because the space of things you *could* test an AI system for is effectively infinite, and most of them aren't worth a button in the product.
Three inputs drive the roadmap. The first is community requests: the "request a service" link feeds a single inbox, and we tag every request by the regulation, risk, or capability it implies. When the same ask shows up from teams in different industries, it moves to the top. The second is regulatory deadlines — when a new obligation lands a compliance date, the evaluation that proves conformity has to exist before teams need it, not after. The third is state-of-the-art research: red-teaming methods, fairness metrics, and robustness attacks move fast, and an evaluation built on last year's technique quietly stops being useful.
A new service doesn't ship as a checkbox. It ships as an agent tool with a precise spec, evidence requirements, and a rubric the agent grades against — the same shape as everything already in the catalog. That discipline is what lets a brand-new evaluation behave like the ones that have been there for a year: runnable from MCP on day one, every finding backed by a citation, every verdict presented for human sign-off.
We also retire things. An evaluation that no longer reflects the regulation it was built for, or that a better method supersedes, gets versioned out — the same way model cards are versioned — so a re-run never silently changes meaning.
If there's an evaluation you need and don't see, tell us. The catalog grows from exactly those signals, and the next quarter's additions are decided by what teams are actually trying to ship.