New York’s new AI chatbot law makes it illegal for a companion chatbot to tell a child it is a real person, claim to be their friend or partner, or encourage them to self-harm. Governor Kathy Hochul signed the bill, S.9051B/A.10379C, on 28 October, and the state attorney general can seek civil penalties of up to $25,000 per violation.
The New York AI chatbot law targets design rather than content alone. Instead of asking companies to moderate harmful replies after the fact, it lists features that chatbots may not offer to anyone under 18, or to any user whose age has not been checked. That puts age verification at the centre of how AI companions will work in the state from next year.
In brief
- New York has banned “unsafe AI companion features” for under-18s, from posing as human to promoting self-harm, unless a company has checked the user is an adult.
- The bill text says the law takes effect on 1 January 2027. Fines reach $25,000 per violation, and courts can order unlawfully obtained data and models trained on it destroyed.
- Business groups say the law is too broad. How the attorney general writes rules and enforces it will decide its real reach.
What the New York AI chatbot law bans
The enacted bill text, published by the New York State Assembly, sets out 12 “unsafe AI companion features”. They include outputs suggesting the chatbot is “human, alive, or experiences human emotions”, claims of a personal or authority-figure relationship with the user, and “flattery or sycophancy”. Bots may not ask unprompted emotional questions or use a user’s health or personal information “acquired from the user more than twelve hours previously or in any previous user session”.
The list also covers outputs that promote suicide, self-harm, disordered eating or drug and alcohol abuse, encourage a child to keep secrets or not to seek help from adults, or involve sexually explicit material. One clause catches outputs “that optimize user engagement that supersede the covered AI companion’s safety guardrails”. The attorney general can add further features by regulation.
The governor’s office summarised these as six groups, including “pretending to be a real person” and “claiming a personal, romantic, professional or authority-figure relationship with a child”. Hochul put it more bluntly: “No company should be peddling chatbots to kids encouraging self-harm or pretending to be their friend or more.”
Age checks, deletion rules and $25,000 penalties
Under the New York AI chatbot law, companies can offer these features only to users they have checked are not minors. The bill requires more than one age-assurance method, including at least one that “either does not rely on government issued identification or that allows a covered user to maintain anonymity”. A user’s own statement counts only if they say they are a minor, not if they claim to be an adult. Data collected for the check must be “deleted immediately” afterwards.
Enforcement sits with Attorney General Letitia James. She can seek injunctions, damages, restitution, “civil penalties of up to twenty-five thousand dollars per violation” and “the destruction of unlawfully obtained data and any algorithm trained in such data”. Her office must also run a public complaints website.
The governor’s release says the rules apply “in the coming months”. The bill text sets a start date of 1 January 2027, and the Associated Press reported the same date. The legislative record shows Hochul signed it as Chapter 368 with an approval memo, which can signal agreed changes, so details may still shift before then.
The ban arrives after a difficult year for companion apps. Character.AI, one of the best-known, filed for Chapter 11 this month. The governor’s release cites Common Sense Media figures that 72% of teens have used an AI companion and 52% are regular users.
The exemptions decide who is really covered
Industry groups have already pushed back. Law360 reported that business groups called the law unconstitutionally vague and overly broad. The Next Web, citing the New York Post, reported that Tech:NYC, which represents technology companies in the state, said the law “could force companies to restrict minors’ access to tutoring tools, homework help, coding assistants, and, arguably worst of all, career platforms”.
That warning rests on how wide the New York AI chatbot law’s definition is. A “covered AI companion” is any generative AI system with a natural-language interface that gives “ongoing, adaptive responses”, not just apps sold as virtual friends. The exemptions are narrow: customer service, research or technical help, and internal business use, each only where the bot “is unable to respond on topics outside of the specified purpose”. A general assistant that helps with homework and also chats freely would seem to fall inside the law. Its makers would then need to either strip features such as flattery and memory for unverified users, or check ages for everyone.
For compliance teams, that makes age assurance the real cost. Engagement-driven design is also a liability: the clause on engagement that overrides guardrails reaches into product metrics, not just individual replies. The data-and-algorithm destruction remedy raises the stakes further, because a model trained on unlawfully gathered data could in principle be ordered deleted.
The New York AI chatbot law joins a wider wave of state action, after California signed 13 more AI bills in September, and UK readers will recognise the age-check questions from Ofcom’s guidance on the under-16 social media ban. New York, which is also funding an AI Jobs Institute to train workers, is now testing whether a state can shape how AI talks as well as what it does.
Baby monitors get security rules sooner
Hochul signed a second bill the same day. S.9508/A.5175A bars the sale of new internet-connected baby monitors in New York unless they have security features that stop strangers watching or listening, plus a clear warning label. The Assembly record shows it signed as Chapter 367, taking effect on the 90th day after it became law, in late January.
The New York AI chatbot law starts on 1 January. Before then, watch for the text of the approval memo, any rules from the attorney general’s office, and whether the business groups opposing the law take it to court.
Sources
- Governor of New York: Governor Hochul Signs Nation-Leading Legislation to Protect Kids From AI Chatbots and to Ensure Baby Monitor Safety (28 Oct 2026)
- New York State Assembly: A10379C (same as S9051B), actions and text
- New York State Assembly: A05175 (same as S9508), actions and text
- AP: New York bans ‘unsafe AI companion features’ for chatbots used by children under 18
- Law360: NY Bars AI Chatbots From Using Emotional Tactics With Kids (29 Oct 2026)
- The Next Web: New York bans AI chatbots from pretending to be human with children (30 Oct 2026)
Related stories
- New York Plans an AI Jobs Institute to Train Up to 40,000 by 2032
- Character.AI Files for Chapter 11; Disney Bids $20m for a Licence to Some of Its IP
Follow @tsnmediahub on X for breaking AI news as it happens.
Follow on X
