The Inspect Sandboxing Toolkit: Scalable and secure AI agent evaluations - A comprehensive toolkit for the safely evaluating AI agents. - aisi.gov.uk/work/the-inspect… / github.com/UKGovernmentBEIS/… by @AISecurityInst How do we test AI systems for dangerous capabilities without risking real-world harm? The more capable models become, the harder it is to safely evaluate them. When an agent can execute arbitrary code and interact with critical systems to gain sensitive information, running evaluations without adequate safeguards could put critical systems at risk. Sandboxes are isolated environments for testing and monitoring AI behaviour. When we give a model access to use tools (such as for writing code), we execute that action in a sandbox to limit the model’s access to external systems and data. This lets us evaluate its capabilities without exposing sensitive resources. Today, we’re releasing our toolkit for safely running agentic AI evaluations. #AISafety #AgentSandboxing #AgentEvaluation #SecureEvaluations #CapabilityTesting #SandboxEscape #ToolIsolation #HostIsolation #NetworkIsolation #DockerCompose #KubernetesPlugin #ProxmoxVM #InspectToolkit #EvaluationProtocol #ScalableSecurity #ThreatMitigation #LLMAgents #ModelTesting #SafeSandboxes #AISISources

Not replacement, think augmentation. Black Hawk is relatively short range #CapabilityTesting