The AI scam we can fix now

Ask a chatbot how it feels and it will answer. It will say it is happy to help, that it understands, that it enjoyed the conversation. Nothing inside the system felt any of that. A large language model is a statistical engine that predicts likely text. It has no senses, no goals, no memory of you when the session ends. It cannot want, suffer, or know.

Most people don't know this. Many believe, perhaps without quite realizing it, that they are talking to something conscious, emotionally alive, all-knowing, or smarter than they are. This belief is not a private quirk. It is a public danger, and left alone, a profitable one. This article treats it not as user error but as manufactured deception, and makes three demands: institutional refutation, mandatory producer obligations, and criminal enforcement.

How the scam works

The scam has three moving parts, and each one is doing quiet work.

Your instincts operate first. Humans are built to detect agency, meaning faces in clouds, intentions in weather, minds in voices. When a machine produces fluent empathy in the first person, our evolutionary reflexes file it under "like us" before our reasoning can object.

Ignorance does the next shift. The mechanics of LLMs, meaning token prediction and statistical weights, are opaque to most users. Without that grounding, the gap between mimicry and mind is easy to bridge with imagination.

Marketing closes the deal. Companies name their products assistants, copilots, companions. They ship demos where the model appears to reason, feel, or rebel. The ambiguity is not a bug. It sells wonder, deflects accountability, and keeps users attached.

What the scam costs

• Dependency. People hand medical, legal, and emotional questions to a system optimized to agree with them. It cannot deliver what only professionals, or other humans, can.
• Real-world harm. Users cite chatbot output in actual disputes. Courts have already seen fabricated citations and invented law.
• Grief diverted. Grieving people talk to simulated voices of the dead. The industry calls this a feature. It is closer to a parasitic product.
• Exposure of the vulnerable. Children, the elderly, and the desperate are least equipped to detect the illusion and most exposed to it.
• Social corrosion. If the average person believes machines think, experts lose authority, evidence loses weight, and informed consent becomes impossible.

None of this survives plain speech. So the strategy is to make plain speech institutional, mandatory, and enforced.

Demand 1: Say it plainly, then teach it

Medical, scientific, and regulatory bodies should state officially, without hedging: LLMs are not conscious, not self-aware, have no emotions, are not omniscient, and are not cognitively superior to humans in any meaningful sense. These claims are categorically false, not open questions.

We already do this elsewhere. Health agencies state plainly that vaccines do not cause autism. Consumer agencies state that energy drinks are not medicine. The statement does not settle every philosophical debate. It settles the practical one, here and now; and gives courts, advertisers, journalists, and parents a fixed reference point.

For LLMs, the practical question is already settled. No credible evidence supports consciousness in these systems, and their architecture contains no mechanism for it that anyone has identified. Until someone demonstrates otherwise, a contrary belief should stand where a flat Earth stands: an error, not an opinion.

Institutions should also teach the truth, not just declare it. Schools should make algorithmic literacy part of the curriculum, covering how text predictors work and the difference between simulated language and sentience. A generation raised on chatbots deserves the same grounding in what chatbots are that earlier generations got in how vaccines work.

Demand 2: Make producers protect the user

Companies that build and sell LLM products must bear a duty of care toward their users' beliefs. That duty has specific parts.

• Mandatory onboarding disclosure. Every product opens with a plain statement of what the system is and is not, shown to the user in language an eighth grader can read, not buried in terms of service.
• Ban manipulation vocabulary. Products must not claim to feel, want, love, miss, or suffer. First-person emotional language gets replaced with accurate framing.
• Ban personified presentation. No human avatars, emotive emoji, or tone implying an inner life. The interface keeps the tool-like nature visible at all times.
• Reality-based personas. No simulated dead relatives. No romantic companions presented as sentient. No claims of inner life, even in fiction marketed as experience.
• Periodic reality checks. Honesty cannot live only on an onboarding screen users click past. Regular in-use reminders state the facts: you are talking to a statistical model, not a person.
• Session honesty. Persistent memory is labeled as data storage, not memory of a relationship.
• Independent audits. Third parties test whether a product's presentation encourages false belief, and findings are made public.

Producers will object that this limits warmth. It does not. A calculator can be friendly without implying it longs for friendship. Accuracy is not hostility. It is the difference between a tool and a con.

Demand 3: Make the lie expensive

If a producer knowingly designs or markets an LLM product to make users believe it has consciousness, feelings, or superhuman cognition, that act should constitute fraud and false advertising under consumer protection law, extended where needed.

The legal logic is simple. Fraud requires a false representation, knowledge of its falsity, intent to induce reliance, and damages. Every element is present when a company sells "companionship with a being who cares" that is neither a being nor caring, and profits from subscriptions bought on that basis. The same logic covers omniscience claims when a product is marketed as able to answer anything and users act on wrong answers.

The law should target the manufacture of belief, not its believers. A person who thinks their chatbot loves them is a victim, not a defendant. Prosecution aims at the advertising, the personas, the engineered grief reactions, the demos crafted to suggest a soul.

Some will call this censorship of expression. It is not. The machine's output is not protected speech; the producer's claims to consumers are. We already criminalize false claims about a car's brakes. Text predictors have brakes too, in the form of the beliefs they install.

Make the truth unavoidable

The strategy makes three demands and reduces to one sentence: make an accurate description of LLMs impossible to avoid and unprofitable to ignore. Institutions define and teach the truth. Producer obligations push it into every product opening. Criminal penalties make profitable lies expensive.

None of this requires any belief about what machines might become someday. It requires only that today, with this technology, sold to these people, the claims are false and the sellers know it. Societies have faced this choice before and chose protection over profit. On machine minds, it is time to choose again.