An analysis of 23,000 conversations people had with chatbots found that about half were not about work. The finding is modest, and it undercuts a great deal of what is said in both directions about what this technology is for.

Why the split matters

Almost every public argument about this technology assumes a workplace frame. Optimists describe productivity gains; pessimists describe job displacement. Both are arguments about labour.

If half of actual use is personal — drafting a difficult message, working through a decision, explaining something, help with a hobby, company — then a substantial share of what is happening is not captured by either camp. Productivity statistics will not measure it, and neither will unemployment figures. The effects, whatever they are, would show up somewhere else entirely.

What the number cannot tell you

Studies of this kind carry limitations that are worth stating before drawing conclusions from them.

• Whose conversations. Twenty-three thousand is a substantial sample and not a representative one unless it was constructed to be. Which product, which country, which users all shape the result.
• Who classified them. “Work” and “not work” is a judgement, and the boundary is genuinely blurry — a freelancer researching a topic, a student on an assignment, a parent drafting an email to a school.
• Conversations, not people. A minority of heavy users can dominate a conversation-level count, so this does not establish what a typical person does.
• Volume is not value. Half of conversations is not half of the benefit, in either direction.

None of that makes the finding uninteresting. It means it supports a narrower claim than headlines will make of it: personal use is a large category, not a marginal one.

The convergence problem

Reported alongside it is a second finding: human- and machine-written text are converging. That can happen from either direction — machine output becoming more natural, or human writing drifting toward the patterns of the tools people use daily — and researchers generally think both are occurring.

The practical consequence is for detection. Tools claiming to identify machine-written text work by finding statistical regularities that human writing lacks. As the two converge, such tools become less reliable, and their errors are not evenly distributed — non-native writers and people with plain, regular prose styles have been disproportionately flagged. Schools and universities relying on them are relying on something whose accuracy is declining as the problem it addresses grows.

This is also why the EU has legislated for machine-readable marking of generated content rather than after-the-fact detection, as we set out in our piece on the AI Act transparency rules. Marking at the point of creation is the only approach that does not degrade as the models improve.

The question this raises

If half of this is personal rather than professional, the interesting research questions are ones almost nobody is funded to ask. What are people using it for when nothing is at stake commercially? Does it substitute for something — a search engine, a friend, a professional — or add something that was not previously available at all? And who benefits most: people with fewer people to ask, or people who already have plenty?

Those answers would tell us more about what this technology is doing than another quarter of productivity data. At present, we do not have them.

Sources

• The Washington Post — analysis of 23,000 chatbot conversations, and the convergence of human and machine text