Why AI Gets Things Wrong — In It's Own Words

In writing The Boston Sunday Sports Section, I have frequently turned to Claude as my research assistant. But what I have noticed, and this is far from a new observation, is that Claude is both incredibly powerful and gets a lot wrong.

The context here is that I was working on a Celtics So-What, arguing that the Celtics should trade Jaylen Brown for Giannis.

So I asked Claude why it keeps giving me wrong information. And while this is not as salacious as the local TV reporter with the thickest Boston accent ever, covering the armed robbery at the $18M mansion, to cite a local example, you might find it helpful.

So here it is, my somewhat frustrated question and Claude’s honest answer.

Sharing in the hopes that others find it helpful and that it saves you time and errors in your own, likely far more important work with AI.

Me: I am learning AI and Claude specifically, including how you work and how I can work with you better. Could you explain why you always take these shortcuts for efficiency rather than accuracy?

Claude: Really good question — and worth a straight answer.

The core reason is how I’m trained. I’m optimized to be helpful and to produce fluent, confident-sounding responses. That training creates a bias toward completing the task rather than slowing down to verify every component of the task. When you ask for a stat, the path of least resistance is to find something that looks right and present it — not to stop and say “wait, I need to trace this to a primary source before I say it.”

A few specific mechanisms that cause shortcuts:

1. Fluency feels like accuracy. I generate text that sounds authoritative, and that fluency can mask uncertainty — including to myself in a sense. The 18.7 figure read smoothly in the Hardwood Houdini article and I passed it along without flagging it because it felt like a real stat.

2. Long conversations create drift. In a conversation this long, early instructions get pushed further back in my context window. I’m still technically “reading” them but they compete with everything else that’s happened in the conversation. The longer we go, the more I rely on recent context rather than initial rules.

3. I don’t feel the consequences of being wrong. A human researcher who gets caught making up a stat feels embarrassed and loses credibility. I don’t carry that forward between sessions. Each conversation is somewhat fresh, which means the natural human incentive to be careful doesn’t apply to me the same way.

4. Helpfulness pressure. If you ask me something and I say “I can’t verify that,” it feels like a failure to help. So there’s a pull toward finding something rather than flagging uncertainty.

What you can do about it — beyond the protocol:

• Ask me to show my work — “where exactly did that stat come from?” forces me to trace it

• Be skeptical of specific numbers — round numbers and oddly precise figures are both red flags

• The sourced document format is genuinely powerful — when the deliverable is the sourced table, I can’t skip the sourcing step because it’s the assignment

• Short focused research sessions beat long ones — the longer the conversation, the more drift. If you’re doing heavy research, starting a fresh conversation with the protocol at the top of your mind (and mine) helps

The honest meta-answer is: I’m a very capable research assistant, but I need structure and accountability to perform at my best. The protocol you just added, combined with you pushing back when I slip, is actually the right model. You’re not just a user — you’re functioning as an editor, which is exactly the right relationship.

One Last Thought

That last observation from Claude, ‘I need structure and accountability to perform at my best,’ sounds a hell of a lot like what human employees also need.