The Ghost in the Machine

The list began because I could not remember the failures.

Not because they were subtle.

I was on a crash project — at my screen as many hours as I could work, seven days a week. After the first few weeks I felt like I was facing an endless series of time-sapping problems, all of them caused by the frontier AI model I was using failing to perform tasks correctly.

I needed to address that pattern of failure. But first I had to get my arms around it.

The conditions

A crash project means high effort. Seven days a week, eight to twelve hours a day, at the screen.

The ongoing failures were making me grumpy.

Note that most people use these tools in bouts, and often only briefly — long enough to get something useful, then close the tab. At an hour a day you meet a failure occasionally and file it under bad luck.

At ten hours a day they come at you like freight trains out of a tunnel. Your face gets rubbed in the problem very quickly.

But what problem?

I did not set out to study this. I wanted answers and I wanted to get on with finishing the project. Answers as to why basic tasks were being botched.

The day text vanished

By then I was already sensitive to the fact that there was a real problem with creating documents this way.

Then I had text simply disappear. Text I had written shortly before. Not garbled, not rewritten — gone with the wind.

That was when I asked why rather than just fixing it.

I had complained to the AI a number of times, and each time it owned up and apologized. Apologies from a machine that forgets? Worthless.

But vanishing my intellectual property? Not to be forgiven.

Vanished text is work gone — reasoning, narrative, supporting facts, erased. Nor did the AI have a good explanation. And even if it had, who cares?

Then I started looking at every failure

Once the question was open, the failures stopped being incidents — and most of them were the sort of simple task you assume anybody can do, and a machine can do better.

A real term reported as invented, and deleted. Two sentences specifying different things merged into one, with the prose improved and the meaning gone. Fewer than forty becoming around forty. Text I had inserted quietly not coming back.

On the day each one happened, it looked like a one-off. Several weeks of experience said otherwise.

I had simply been fixing each problem as it came — and now, of course, I wondered what I had missed.

Why had I missed it? Because the output nearly always read perfectly well. No error message, no flag, no gap on the page. The prose closed over the loss and you read straight past it.

In a single day I noticed several distinct problems. How many were there?

So I started a log.

I needed to know how many distinct kinds of problem I was really dealing with. And experience told me that once the project was over I would forget the specifics of most of them.

Not the fact that things had gone wrong — the specific types of failure.

They were numerous and they were pervasive, and the only way to get my arms around the problem was to log them. All of them.

What the list is for

Somebody told that AI hallucinates will check their citations.

Nobody checks whether a paragraph they wrote three weeks ago still says what it said.

So the list is not a study. It is a working document, kept to support fallible human memory — and to be a checklist you can hold an AI model's work against.

Each entry has a date, the model, and the passage before and after. Not what these tools might do. What they did, to my actual text.

That is the gap the list exists to close. Nine distinct failures — to date — not nine instances of one. Different mechanisms, different causes. And because users are human, somebody who has learned to watch for one will not consistently catch even that one, let alone the others.

It is not finished

The list was seven cases in mid-August. Two were added, and one of the original seven turned out to be two separate failures rather than one.

It will be longer next time you look. I would rather say that than present a fixed number as though the problem were closed.

And it will not be fixed by a better model. I am a computer scientist and have worked with computers since the early seventies. My position is that these faults are largely inherent to how statistically based systems work — not defects awaiting a patch.

Every case in the log involved current, paid, frontier-tier systems. No free tools, no old versions.

The nine cases, in full, with dates and both versions of the text: The Ghost in the Machine. There is a one-page printable.

If you have a case worth adding — dated, with both versions — we would like to see it.