The myth of the perfect prompt
I spent three weeks last year obsessing over a single prompt for a client’s lead qualification bot. I was using GPT-4, and I genuinely believed that if I just added the right phrase, like “think step-by-step” or “you are a world-class analyst,” the hallucinations would stop. I remember spending four hours on a Tuesday tweaking a paragraph of instructions, changing “be concise” to “limit responses to 50 words,” thinking I was on the verge of a breakthrough.
It didn’t work. Not really. I got the accuracy from 72% to 78%, but it still failed in weird, unpredictable ways. That’s the trap. We’ve been told that prompt engineering is the secret sauce, but for actual business reliability, it’s kind of a dead end. You can’t prompt your way out of a probabilistic system’s inherent randomness.
Prompting is just asking the AI to try harder. But in a professional setting, “trying harder” isn’t a metric. We need outcomes that are verifiable.
Why LLMs are terrible at grading their own homework
Most people try to fix AI mistakes by adding a second prompt: “Now review your previous answer and correct any errors.” This feels smart because it mimics how humans work. We proofread. We double-check.
But LLMs aren’t humans. They suffer from something called sycophancy. If the AI made a mistake in the first pass, it’s highly likely to either double down on that mistake or just tell you it fixed it without actually changing anything. I tried this with a Python script for a data migration project using Claude 3. The AI kept hallucinating a specific library function that didn’t exist. I asked it to check its work three times. Each time, it apologized and then gave me the exact same broken code. It was like arguing with a wall that’s very polite but completely wrong.
The problem is that the AI is using the same logic to verify the answer as it used to generate the answer. If the logic is flawed, the verification is flawed. It’s a closed loop of error.
AI loop engineering vs prompt engineering
This is where we shift from prompt engineering to AI loop engineering. Instead of trying to write the perfect set of instructions, we build a system that forces the AI to interact with a source of truth outside of its own “brain.”
Loop engineering means building external verification steps. Instead of asking the AI, “Is this correct?”, you build a loop where the AI’s output is sent to a tool that can actually prove it’s wrong. For example, if the AI generates a piece of code, the loop doesn’t ask another AI to check it; it sends the code to a compiler. If the compiler throws an error, that error is fed back to the AI to fix. Now you have a factual feedback loop.
I might be wrong about whether this scales for every single use case, but for anything involving data or logic, it’s the only way to sleep at night. It’s the difference between hoping the AI is right and knowing it’s right because a third-party system verified the output.
Moving from tweaks to system design
Designing a system is harder than writing a prompt. It requires thinking about architecture rather than adjectives. You have to map out the flow: Generation -> Verification -> Correction -> Final Output.
Recently, we built a system for a client that handled invoice processing. We could have spent months prompting the AI to “be very careful with the numbers.” Instead, we built a loop that extracted the totals and then ran a simple Python script to sum the line items. If the AI’s total didn’t match the script’s sum, the system rejected the output and told the AI exactly where the math failed. Accuracy jumped to 99.4% almost overnight. No amount of “prompt hacking” would have gotten us there.
It’s a shift in mindset. We have to stop treating the LLM as the entire solution and start treating it as one component in a larger machine. The LLM is the engine, but you still need brakes, a steering wheel, and a dashboard to know if you’re actually going in the right direction.
Most businesses are still in the “prompting” phase. They’re just adding more words to their instructions and wondering why the AI still messes up 10% of the time. That 10% is where the risk lives. And you can’t prompt away risk; you can only build systems to catch it.
Anyway, I’m still figuring out the best way to handle the latency that comes with these loops. It’s a bit slower, and honestly, it can be frustrating when you just want an instant answer. But I’d rather wait five seconds for a correct answer than get a wrong answer in half a second.
Talk soon,
SM
