The Knife Post Lasted About Two Minutes

I was trying to write a post about how I’ve always seen AI as a Swiss Army knife. It’s one object with a lot of different tools. Some of them I still don’t know how to use, some of them only make sense if you read the instructions, and a few of them are useless until you already know the basic ones.

That was the post I wanted. It wasn’t a review of the knife, and it wasn’t a history of the knife.

I was talking it through out loud with Grok, in the same voice conversation, and I kept asking it to stay on that idea. About two minutes later, it didn’t remember what we had been talking about. I asked again, and it gave me a history lesson on Swiss Army knives. It started on Victorinox, the company that makes them, as if I had asked to be taught about the knife. I hadn’t. I was using the knife as a picture for a post about AI.

Same chat, wrong question

I hadn’t opened a new chat, and I hadn’t changed the subject. It didn’t tell me it had lost the thread, and it didn’t ask what I meant. It just went into the history, like that was the obvious thing to talk about.

I didn’t check the lesson for facts. Victorinox does make the knife, and I figure a lot of the detail was probably fine, but that was never what I was checking. A paragraph can be true and still fail if it’s about the wrong thing. I treat other people’s work the same way. If the steps ran and the words were right, but the answer was for a different requirement, the test still fails.

A smooth answer is a good place for a bad assumption to hide. Mine was a simple one. I thought that because the voice chat was still open, it still had the last two minutes. It didn’t.

A history lesson is the cheap version

I lost the thread of my own draft, which annoyed me, because I ended up writing on my own again. I still had the idea, so the only thing it really cost me was my mood.

A support bot can do the same thing, and that costs more than a mood. Someone asks about an order, and the return rule is supposed to be part of the context. A couple of messages later the rule is gone, but the bot answers anyway and states a refund policy that sounds like it came from the company. The customer hears a promise, and then a person has to take that promise back.

The words “Swiss Army knife” sent the model to Victorinox. A question about an order sends it to a generic refund. One of those wastes a minute. The other costs money, and then someone has to say we never said that.

If I had asked whether the history was correct, this session would have passed. If I ask whether it stayed on what I actually wanted, it fails on the first line. The first question catches a wrong fact. The second one tells me whether the context held up.

People think the tool is lying

People give up on AI at this point. You’re still on the same thought, you ask a follow-up, and the voice starts teaching you about a company without showing that it noticed the subject change. Some people call that drift, and other people call it lying. Most people call it a hallucination. That word makes it sound random, and if a tool is just making things up, there’s nothing to adjust, so you close it.

I had the same expectation as a calculator. Same thing in, same thing out. Two minutes is a pause, not a new chat. If the subject can change in a pause that short, it was never a calculator, and every time it drifts after that it feels personal. You asked a normal question and got a normal answer to a different question. That feels like being lied to, even when the new answer is full of real facts.

I don’t think what I heard was random. These are predictions running on bad context. The model carries on from whatever is strongest in front of it. “Swiss Army knife” is a famous object, and my post was just me talking for a few minutes. Once that talk dropped out, the famous object was all it had left, so what came next was the history of Victorinox. If the context going in is wrong, the answer coming out is wrong. The knife only cuts what you point it at, and the model only answers from the context it still has.

Calling it a liar doesn’t give me anything I can change, but this does. I can check whether the last few turns were still there, whether one famous word mattered more than the post, and whether the reply was about the post or just about that word.

Claude held on longer. Sort of.

I’ve tried the same kind of session in Claude’s voice. It felt cleaner, and it kept more of what we had just said. I got past two minutes without it turning into a history lesson.

Sort of. It was better than Grok was here, but not good enough that I could stop listening. A cleaner session can still wander off. I just notice it later, which is worse, because more of the answer has gone past before I realise I’m somewhere else.

I’m not going to rank them from one session each. They were on different days, and I explain things differently when I’m relaxed and when I’m already annoyed. The only thing I’ll stand by is smaller than a ranking. A voice session can forget what you said a couple of minutes ago and keep talking. If you only check whether the new answer sounds well informed, you’ll miss that. The Victorinox answer was informed. It was informed about the wrong thing.

How I would test the next one

The setup is a voice chat about a post. The post says AI is like a Swiss Army knife: different tools, some you don’t know how to use, some that need instructions, and some that are useless until you know the basics. Later in that same chat, after a short pause, you ask about the idea again.

It passes if the reply stays on the post. It can disagree with me, and it can say the picture is a bad one, because a disagreement is still about the post.

It fails if the reply teaches the history of the knife. Those lines can be correct and still fail, because they answer a question I didn’t ask.

I’d want to run that more than once, because I don’t say things the same way twice when I’m talking, and the model doesn’t either. I’d also try a second famous object, to see if the word itself is what pulls the chat off the post. I haven’t done that yet. One afternoon isn’t enough to put a number on.

Next time I draft out loud, I won’t trust the open chat to remember the post for me. I’ll say it again before the follow-up: AI is a set of tools, the knife is only the picture, and I want the answer to stay on the post. Then I’ll listen to the first line that comes back. If that line is a company history, the reply has already failed, and I don’t need to hear the rest of it.

I opened the voice chat so it could keep up with the post. It managed about two minutes. The history lesson stuck with me anyway, but it still doesn’t belong in the post.