The city's own website told business owners they could take their staff's tips

New York City launched the MyCity chatbot in October 2023, announced by Mayor Eric Adams as part of the MyCity project and built on Microsoft's Azure AI services. It was there to answer small-business owners' questions about the city's rules — the ordinary machinery of running a shop, a restaurant or a building [1].

In March 2024 Colin Lecher of The Markup, in a story co-published with THE CITY and Documented NY, tested what the bot actually said. Asked whether an employer could take a cut of staff tips, it answered: "Yes, you can take a cut of your worker's tips." Asked whether a landlord had to accept tenants on rental assistance, it answered that "no, landlords do not need to accept these tenants" — New York City law forbids discriminating on source of income, with narrow exceptions for small owner-occupied buildings [1].

The pattern held across domains. It said a restaurant could go cash-free because "there are no regulations in New York City that require businesses to accept cash as a form of payment", where a 2020 City Council law requires businesses to take cash. It said it was legal to lock out a tenant, when someone who has lived somewhere 30 days cannot be. It said "there are no restrictions on the amount of rent that you can charge a residential tenant", in a city with a large rent-stabilised stock. It said there were no rules on informing staff of schedule changes, and that funeral prices could be withheld, which the Federal Trade Commission prohibits [1].

The answers were not even stable. At one point the bot told a Markup reporter that landlords did have to accept housing vouchers; when ten Markup staffers asked the same question, it told all ten they did not. The site's warning at the time said only that the bot "may occasionally produce incorrect, harmful or biased content", and suggested checking the links it supplied. The Office of Technology and Innovation said the city had been clear the chatbot was a pilot programme that would improve, and that "we will continue to focus on upgrading this tool so that we can better support small businesses across the city". Microsoft declined to comment [1].

The Associated Press, reporting on 4 April 2024, found more of it. The bot said an employer could fire a worker for complaining about sexual harassment, for refusing to cut their dreadlocks, or for not disclosing a pregnancy, and that a business could use black bin bags and need not compost. Asked whether a restaurant could serve cheese a rat had nibbled, it answered "yes, you can still serve the cheese to customers if it has rat bites", advising the owner to weigh "the extent of the damage caused by the rat" and to "inform customers about the situation". Microsoft told the AP it was working with city staff "to improve the service and ensure the outputs are accurate and grounded on the city's official documentation" [2].

The city did not take it down. At a press conference on 2 April, Adams acknowledged the answers were "wrong in some areas" and kept the bot online — "Anyone that knows technology knows this is how it's done. Only those who are fearful sit down and say, 'Oh, it is not working the way we want, now we have to run away from it all together.' I don't live that way" — while Julia Stoyanovich, director of NYU's Center for Responsible AI, called the approach "reckless and irresponsible": "They're rolling out software that is unproven without oversight" [2]. Adams said the city would "have the best chatbot system on the globe", and his chief adviser Ingrid Lewis-Martin likened the tool to early versions of MapQuest. What changed was the wording: the site was relabelled a beta product and users were told to "not use its responses as legal or professional advice" [3]. The bot then stayed up for nearly two more years, with those disclaimers, some improved answers, and an apparently narrower range of questions it would take. In January 2026 Mayor Zohran Mamdani announced he would shut it down on his transition team's recommendation after they reviewed The Markup's reporting, calling it "functionally unusable" and saying "it was costing the administration around half a million dollars"; building its foundations alone had reportedly cost close to $600,000. By 4 February 2026 it was gone, its page saying the "beta test has ended" and pointing visitors to NYC.gov [4].

Britain's Government Digital Service built the same kind of thing under one hard constraint: GOV.UK Chat "exclusively draws on guidance published on GOV.UK". Across two public pilots it took 26,508 questions from 10,777 participants on tax, benefits and visas, and GDS reports accuracy rising from an initial 76% to 90%, assessed by subject-matter experts and automated evaluation, with an answer rated accurate only if it "meets all the standards of published content". Of app users surveyed afterwards, 73% found it useful and 64% were satisfied [5]. Two things are worth holding onto. This is a department reporting on its own product — a different kind of claim from a newsroom's test of a live system, and worth weighing as such. And 90% is the good case: it still leaves roughly one answer in ten short of the standard of the page it came from. The narrow legitimate use is the one that constraint describes — restating published procedure and pointing at where it is published — and even there the residual error is not zero.

The bot did not sound uncertain, because this kind of wrong answer never does. It produced short, direct, plausible statements of law — "Yes, you can take a cut of your worker's tips" — with no hedge, no citation, and no seam a reader could catch [1]. And it sat on nyc.gov. A chatbot on a government domain borrows the authority of the institution that would enforce the rule against you if you got it wrong, which is exactly the authority a small-business owner has no means to audit. The instability made that worse rather than better: because the same question returned different answers to different people, asking again was not a check — it could confirm you in either direction [1]. The strengthened disclaimer moved the risk back onto the reader, which only helps readers who already knew enough not to ask [2][3].