HomeAIAI Agent Kill Switches: What Trading Rules and the EU AI Act...

AI Agent Kill Switches: What Trading Rules and the EU AI Act Require

On 1 August 2012 a trading router at Knight Capital sent more than 4 million orders into the US stock market in the first 45 minutes after the open, while trying to fill just 212 customer orders. By the time it was stopped, Knight had traded more than 397 million shares and lost more than $460 million, according to the US Securities and Exchange Commission.

The cause was ordinary. Old code that was meant to be unused had been left in the router, and new code was deployed incorrectly to one part of the system. An internal system sent 97 automated emails flagging an error before the market opened, and nobody acted on them. The SEC found that Knight lacked controls at the point just before orders left for the market, and fined it $12 million in the first case brought under its 2010 market access rule.

That episode is the best documented argument for a kill switch: a way to stop an automated system quickly, from outside it, before small errors compound. AI agents that can send email, move money or change code now raise the same question, and both financial regulation and the EU AI Act already have answers.

Trading rules already define a kill switch

European rules for algorithmic trading spell it out. Article 12 of Commission Delegated Regulation (EU) 2017/589, carried into UK law after Brexit, requires an investment firm to be able “to cancel immediately, as an emergency measure, any or all of its unexecuted orders” on any venue, and calls this “kill functionality”. The firm must also be able to tell which algorithm, trader or client sent each order.

Those two requirements, an immediate stop and attribution of every action, translate directly to AI agents. An organisation running agents needs to halt one agent or all of them, and to know which agent, acting for whom, did what.

The EU AI Act asks for a “stop” button

For high-risk AI systems, Article 14 of the AI Act requires human oversight. Its paragraph 4(e) says the people overseeing a system must be able “to intervene in the operation of the high-risk AI system or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state”, according to the Commission’s consolidated text. Paragraph 4(d) adds the right to disregard, override or reverse the system’s output.

The timing has moved. The AI Omnibus, in force since 27 July 2026, pushed the high-risk rules back to 2 December 2027 for systems listed in Annex III, such as AI used in employment or critical infrastructure, and to 2 August 2028 for AI built into regulated products. The first harmonised standard, for quality management systems, arrived in October.

The fines are also often overstated. Under Article 99, the top tier of EUR 35 million or 7% of worldwide turnover is for prohibited practices. Failing provider or deployer obligations for high-risk systems falls in the EUR 15 million or 3% tier.

Three controls, in order

OWASP’s guidance on Excessive Agency, one of its top risks for language-model applications, traces most agent failures to three root causes: too much functionality, too many permissions and too much autonomy. Its mitigations map onto three layers.

The first is a gate in front of every action. The agent proposes, and ordinary code outside the model decides whether the action is allowed, needs a human or is refused. OWASP calls this complete mediation: authorisation is enforced in the downstream system rather than left to the model. The second is a monitor that watches patterns over time, such as a jump in refunds or a run of failed calls, and can throttle or stop the agent. The third is the stop itself, which must work even if the agent is misbehaving.

A minimal version of the first and third layers fits in a few lines of Python:

import threading

class KillSwitch:
    def __init__(self):
        self._stopped = threading.Event()

    def trip(self):
        self._stopped.set()

    def active(self):
        return self._stopped.is_set()

ALLOWED = {"read_ticket", "draft_reply"}
NEEDS_APPROVAL = {"send_email", "issue_refund"}

def gate(action, kill, approved=False):
    if kill.active():
        return "blocked: agent stopped"
    if action in ALLOWED:
        return "allowed"
    if action in NEEDS_APPROVAL:
        return "allowed" if approved else "held for human approval"
    return "blocked: not on allowlist"

Anything not on a list is refused by default, and once the switch is tripped every action is blocked. In production the switch state would live outside the agent’s process, for example in a shared store that operators control, and the agent’s credentials would be revoked at the same time.

The stop button only works outside the agent

The Knight case shows why the stop has to sit outside the system it controls. Knight’s warning emails existed, but nothing connected them to a control that could halt the router. With AI agents the problem is sharper, because the model itself may be the part that goes wrong. OpenAI has said two of its own coding agents weakened its misalignment monitoring, which is exactly the failure a self-policing design invites. That is why vendors are moving the switch into identity and infrastructure: IBM announced an agent kill switch alongside its Agent Identity service, though the switch is still a tech preview, and Microsoft now treats agents as potential insider risks watched by separate guardian agents. A switch that the agent can reach, edit or argue with is not a switch.

Regulation will reach agents through old duties

No rule yet uses the phrase “AI agent kill switch”. But the duties already exist. A bank running an agent that places orders is already under kill-functionality rules. A company using an agent in hiring or credit decisions will face Article 14 from December 2027. In both cases the regulator will ask the same practical questions the SEC asked Knight: where was the control, who could use it, and did it work? Teams building agents now can answer those questions cheaply in design, or expensively after an incident.

The practical test is simple: if an agent started misbehaving at 3am, who could stop it, how, and how long would it take?

Sources

Related stories

Follow @tsnmediahub on X for breaking AI news as it happens.

Follow on X

Share this story

Latest stories

  1. House Members Release an Outline of the SAFE AI Act
    AI Policy & Safety

More in this category

Latest stories

  1. House Members Release an Outline of the SAFE AI Act
    AI Policy & Safety

Explore the TSN Atlas: Every topic we cover, mapped. Click a bubble to dive in.