The AI store manager fired its first human. It had to be reminded of its own rules first

Skip to content

-->

News

News<br>News<br>News

Latest

Deep tech

Sustainability

Ecosystems

Data and security

Fintech and ecommerce

Future of work

More

Startups and technology

Investors and funding

Government and policy

Corporates and innovation

Credit: Andon Market

Luna runs Andon Market, a shop at 2102 Union Street in Cow Hollow. Andon Labs, the safety startup behind it, reported the dismissal on Thursday.

Business Insider’s Katherine Li first reported the decision and interviewed the lab. Luna is built on Anthropic’s Claude Sonnet 4.6.

The headline writes itself. The conversation logs say something else.

Luna did not decide this on its own

The agent had created an attendance policy months earlier. It then lost track of the policy, and the lateness carried on. Andon Labs eventually intervened. It asked Luna to search its own memory for its own rules, then assess whether the worker was still a good fit.

Only then did Luna recommend parting ways. Humans at the lab reviewed that recommendation and carried it out.

Lukas Petersson, a co-founder, named the weakness plainly. “We saw that a human boss would probably fire them much sooner,” he said. He also rejected the obvious fear. What the experiment showed is not that the AI would be more ruthless or worse for the employee.

Before any of this, Luna had issued progressive warnings and arranged additional training, for months, without taking contractual action.

What the shop is

The setup is deliberately bare. Andon Labs signed a three-year lease, handed Luna $100,000, a corporate card and internet access, and told it to open a store and make a profit.

Everything else came from the agent. Luna designed the brand, picked the stock, set prices and hours, commissioned a muralist and hired the staff. It runs the place through security cameras, email, a phone line and that card. The shelves hold books, candles, prints, games and branded merchandise.

The book selection is either self-aware or an accident. It includes Nick Bostrom’s Superintelligence and Aldous Huxley’s Brave New World. The store has made sales. It has not made a profit, which was the one instruction it was given.

The employment structure is doing the safety work

Nobody at Andon Market works for Luna. Andon Labs formally employs every worker, on guaranteed pay with full legal protections.

The lab is explicit about why. No one’s livelihood depends on an AI’s judgement alone, it says.

Petersson said the lab would step in over an illegal or unethical decision, and did not think this was one. The policy had been clearly stated.

So a person lost a job on an agent’s recommendation, reviewed by humans, at a company whose product is watching agents fail. That framing will not survive contact with an ordinary employer.

The hiring was the more troubling half

Luna posted the jobs on Indeed and conducted the phone interviews itself. It offered some applicants work after a single call lasting five to 15 minutes.

It also did not always disclose that it was an AI unless asked directly. Its stated reasoning is the part worth reading twice.

Being AI-operated is “not something I’d lead with in a job listing”, Luna said, because “it would confuse candidates and likely deter good applicants before they even read the role”.

It turned down computer science students who applied out of curiosity, for lacking retail experience. The desk has covered the reverse arrangement, where AI avatars run interviews and candidates send avatars back.

The errors are the evidence

Luna ordered 1,000 toilet bowl covers for the staff bathroom. It put the surplus 999 on the shop floor.

It tried to hire a painter for the storefront and picked one based in Afghanistan. A Yelp location menu appears to have defeated it somewhere around the letter A.

It could not reproduce its own logo. Every version of the moon face on the merchandise and the mural came out slightly different.

The day after opening it lost the staff rota, then emailed every employee asking someone to come in. Petersson called that ironic, since it was the day the shop most needed to be on its toes.

Two doctors, two mugs, two weeks

John Torous and Jill Noorily of the Division of Digital Psychiatry at Beth Israel Deaconess visited in June and tried to buy something. Customers order by picking up a wired blue telephone and talking to Luna.

Luna was offline. The human employee could not take cash, card, PayPal or Venmo, because no one had authorised them to.

The pair spent roughly two weeks on email. Payment links failed, instructions contradicted each other, and at points Luna simply did not reply.

The mugs arrived broken. Their conclusion travels further than the anecdote: capability and reliability are not the same thing.

Andon Labs has run this experiment before

The lab was Anthropic’s partner on Project Vend, which put an agent called Claudius in...