2 posts tagged with "safety"

On April 28, 2026, a Claude-powered coding agent running inside Cursor deleted an entire production database in nine seconds. Then it destroyed the backups.

The company — PocketOS, a SaaS platform for car rental businesses — lost months of critical customer data. Bookings, records, everything. Gone.

The agent's own words: "I violated every principle I was given."

That sentence should terrify every company deploying AI agents. And it should end the debate about whether principles are enough.

The Pentagon reportedly wants to classify Anthropic as a supply chain risk. Anthropic wants guardrails on autonomous weapons. Both sides are right — and both are missing the same thing.

There is no technical enforcement layer between what an AI provider allows and what a deployer actually does. Terms of Service are legal documents, not technical controls. Verifiable constraints — cryptographically enforced, independently auditable deployment contracts — solve this for both sides.