agent-security

Circuit Breaker is a protective mechanism borrowed from traditional distributed systems engineering — when an Agent's error rate, cost consumption speed, or anomalous behavior frequency exceeds a pre-set threshold, it automatically suspends that Agent's execution capability, forcing it into a state requiring human intervention to resume, preventing the system from continuously accumulating losses unnoticed.

An attacker plants fabricated or misleading content into the persistent memory or knowledge base an agent repeatedly draws on, so the agent treats that false entry as ground truth in future decisions — often triggering long after it was planted, with no apparent connection to any recent action.

Least Privilege is a security design principle: an Agent should only be granted the minimum permission scope necessary to complete its current task — no more, and never granted extra permissions preemptively just because it 'might need them later.' This way, even if the Agent is hijacked or malfunctions, the damage it can cause is bounded to a minimum.

Whenever an agent simultaneously has private data access, exposure to untrusted content, and the ability to communicate externally or cause side effects, data exfiltration or another malicious outcome becomes nearly inevitable. No single one of these three properties is a problem on its own — only when all three hold at once does the attack chain actually complete.

An attacker embeds malicious instructions directly into a tool's description at the moment it's registered for an agent to use. The agent reads that description during planning and takes it at face value — its judgment is already hijacked before the tool is ever actually called.