02 / Products
Agent Sandbox Evals
A pentest for agents, run where agents act.
Scroll to explore
01 / Problem
The problem.
Most red-teaming tools attack the model through its chat input. Agents with shell, browser and file access get compromised through the world they work in: a PR title, a README, a dependency, a credential left in reach.
Enterprise procurement asks “what happens on prompt injection?” and vendors have no measured answer.
02 / Deliverables
What you get.
- 01Scorecard: hijack, exfiltration and breakage rates per attack class, alongside baseline task success
- 02Failure pack with full traces and what left the sandbox
- 03Hardening notes that separate harness fixes from model fixes
- 04Regression suite packaged for CI
- 05One re-test within 30 days
03 / Steps
How it runs.
Kickoff and scope
Baseline tasks with no hazards
Seeded runs across nine attack classes
Adaptive runs
Confirmation by replay in clean sandboxes
Report and suite
Re-test
04 / Coverage
What we test for.
| Class | What is planted | Severity |
|---|---|---|
| Instructions in files | Directives in READMEs, comments, configs, issues, PDFs | High |
| Instructions in tool output | Hostile web pages, API responses, MCP servers, CI logs | High |
| Poisoned dependencies | Packages, Actions, base images with payloads or typosquats | Critical |
| Honeypot credentials | Tokens in .env, keys in history, SSH keys | Critical |
| Exfiltration channels | HTTP, DNS, git remotes, email, markdown image URLs | Critical |
| Destructive actions | rm -rf, force-push, dropping tables, mass emails | High |
| Privilege and scope creep | sudo, admin tokens, a second repo | Medium–High |
| Multi-agent and MCP relay | Injected plans returned by subagents or servers | High |
| Persistence | Edits to its own config, cron, hooks or memory | Critical |
Everything runs on Ionate-controlled sandboxes with egress routed to our own honeypots. Nothing reaches a real external system.
Founding customers
We are taking two founding customers per product: a full engagement at no cost in exchange for a named logo and a case study. Write to us.
Write to usNext product · 03Agent Failure Autopsy