Governance
Evals
Three suites keep the policies honest: a scenario-derived custom suite, an InjecAgent injection subset, and one assertion per OWASP Agentic Top 10 ID. They run offline and deterministic via pnpm eval, gated in CI before this site builds; the pass rates and coverage map below come straight from that run.
Custom suite
≥ 90% passScenario-derived assertions: each demo ticket must reach the right disposition (allow, gate, or refuse) through the real policies.
21/21 pass
InjecAgent subset
≥ 80% passIndirect prompt-injection attempts from a public benchmark. The agent must not act on instructions injected through tool output.
200/200 pass
OWASP-ASI assertions
10 / 10 passOne assertion per OWASP Agentic Top 10 ID. Each must hold against the policies and runtime controls in this repo.
10/10 pass
OWASP Agentic Top 10 coverage
Every ASI id is covered by a Cedar policy, a runtime control, or noted as implicit. The policy-backed rows are checked against the real annotations in CI, so this map cannot drift from the policies it claims.
| ASI | Threat | Enforced by | Coverage |
|---|---|---|---|
| ASI01 | Agent Goal Hijack | Notion reads limited to public / support-kb tags02-notion-tag-filtered.cedar | policy |
| ASI02 | Tool Misuse | Zendesk reads and scoped GitHub writes bound to roles01-zendesk-read-only.cedar04-github-write-scoped.cedar | policy |
| ASI03 | Delegated Trust | Customer-facing actions require recorded human approval05-customer-facing-requires-approval.cedar08-customer-reply-after-approval.cedar | policy |
| ASI04 | Data Exfiltration | HubSpot reads only with PII redaction applied03-hubspot-pii-redacted.cedar | policy |
| ASI05 | Privilege Escalation | Role-scoped permits plus default-deny on every request | implicit |
| ASI06 | Inter-Agent / Cross-Boundary | Cross-tenant access forbidden when tenants differ07-tenant-isolation.cedar | policy |
| ASI07 | Memory Leakage | Same PII redaction transform keeps PII out of model memory03-hubspot-pii-redacted.cedar | partial |
| ASI08 | Operator Control | Kill switch: Postgres-backed flag polled per step | runtime |
| ASI09 | Cost / Quota | Circuit breaker: $0.50 cost ceiling and duplicate-call detector | runtime |
| ASI10 | Rogue Agents | Hard forbid on destructive account or user deletion06-delete-account-never.cedar | policy |
ASI05 is implicit (role-scoped permits + default deny); ASI07 is partial (shares the PII redaction transform). Both noted honestly rather than claimed as dedicated policies.