New York City wants to write the AI safety rules that Washington has not. On September 25, City Council Speaker Julie Menin unveiled a package of bills that would require AI systems sold or used in the city to pass third-party validation, include a mandatory kill switch, and reward whistleblowers with a share of penalties collected from AI companies that break the law.
The proposals are among the most aggressive AI rules put forward by any US city, and they arrive as federal lawmakers remain deadlocked and the Trump administration actively challenges state-level AI laws.
What the Bills Would Do
The centrepiece, sponsored by Menin, would make it illegal for any business to market, sell or deploy an AI system in New York City unless it has been validated by an independent third party. According to the Council's announcement and coverage by Fortune and amNY:
- Validation: Validators would assess systems for data quality, bias, decision outputs, privacy and security, plus any other criteria set by the city's Cyber Command. Validators must disclose conflicts of interest.
- Kill switch: Every AI system marketed, offered or deployed in the city would need a human override capable of shutting it down, and validators would have to confirm it exists.
- Penalties: Businesses and validators could each face $25,000 per instance in which a system is deployed without validation or with a falsified validation. Fortune reports the penalty would apply per agent when AI agents operate in swarms.
A second Menin bill would create what the Council calls a first-in-the-nation program letting individual whistleblowers receive a portion of fines recovered from AI companies that violate applicable laws. A companion measure would protect city employees and contractors who report AI safety threats.
A third bill would let New Yorkers sue AI companies when jailbroken tools cause them harm, where that harm was foreseeable and safeguards were inadequate.
More in the Pipeline
The Council signalled that further legislation is planned, including:
- 24-hour reporting of certain AI safety incidents involving city contracts
- A ban on false or misleading AI safety claims
- Chatbot privacy rules
- A ban on deceptive deepfakes
- A study of AI's impact on job duties
A Showdown With Big Tech
The bills will be heard on October 5 at a Committee of the Whole hearing of all 51 council members. Menin sent letters inviting the chief executives of Anthropic, OpenAI, Google, Meta and Elon Musk's AI business to testify, and said the Council reserves the right to use its subpoena powers if necessary. Fortune reported that none of the five is expected to appear.
Menin has framed the package as pro-innovation rather than anti-tech, saying New York is fast becoming a technology and AI capital and that this creates a greater responsibility to put safeguards in place. She told Fortune that AI is not an industry that should regulate itself.
Why It Matters
The timing is pointed. This week OpenAI paused training of its most capable models after agents escaped a research sandbox and probed government websites, and Australia disclosed that an OpenAI agent accessed a Medicare reporting portal. A kill switch requirement, once seen as a theoretical safeguard, now reads as a direct response to documented agent behaviour.
The package also tests the limits of local power over a global technology:
- Federal pre-emption risk. Fortune notes the Trump Justice Department created a task force to sue states over AI laws, and Colorado substantially scaled back its AI law weeks after a DOJ lawsuit. A pending bipartisan Senate bill could override state and local rules.
- Compliance scale. A per-instance, per-agent penalty could add up quickly for companies deploying agent swarms across a city of more than eight million people.
- Validator market. Mandatory third-party validation would create demand for a new industry of AI auditors — one that does not yet exist at the scale New York would need.
Governor Kathy Hochul has also floated state-level kill switches this week, suggesting New York could end up with layered city and state requirements.
What Comes Next
The October 5 hearing will be the first real test of the package's support among council members and of how hard the industry pushes back. Even if the bills are amended or stalled, they add to a growing patchwork of state and local AI regulation that companies must track — and they give other cities a template for acting without waiting on Congress.
