How I Fixed the Infinite Feedback Loop When Auditing Project Plans with Claude

This article is a reprint from Started Claude Code.

I always enjoy AI programming.

My usual workflow is to create a plan, have it audited, and then proceed with the implementation.

However, I felt that the auditing process hasn't been working well, especially since Opus 4.7. Perhaps it's because Opus has gained a broader perspective? It often brings up points that are irrelevant to my plan, and when I have it perform an automated loop of auditing and revising, the feedback often fails to converge.

Whac-A-Mole

Then one day, I realized it.

When writing a program with slightly complex logic, the AI keeps saying "this is wrong" or "that is wrong" every single time. It feels like an indirect loop, or more accurately, constant Whac-A-Mole.

It says, "B is weak from the perspective of A," so I fix B. In the next audit, it says, "B is excessive, and A is thin." When I add A, it then says, "C is inconsistent." When I fix C, it says, "The description of C is redundant." Once I fix that, it says, "C is insufficiently explained."

It's a seesaw. There is no exit in sight.

What I tried

Realizing this, I tried the following approach.

For a plan that was reasonably complete (having gone through 1 or 2 audits), I asked the AI, "Please audit the plan only for logical contradictions."

This worked perfectly. It diligently resolved the contradictions, and after a few rounds of auditing, it converged properly.

Since the number of contradictions is finite, it actually converges.

And when I let it start the implementation with a plan free of contradictions, it runs straight to the end without stopping (lol).

Of course, there are occasional implementation errors where it has to retry, but that's expected.

I wondered if this exists in the world

Having reached this point, I suddenly became curious. Surely, I'm not the first person to figure this out. Someone else must have thought of the same thing.

I looked it up.

There were similar concepts, but they felt different. To put it simply, they were complicated.

• Criteria Drift (Explanation by Hamel Husain, Shankar et al.): A phenomenon where evaluation criteria gradually shift when using an LLM for review. The countermeasure is "rescoring past scores while refining the evaluation axes." ...That's heavy.
• Oscillatory Convergence (Fractal Thought Engine): An observation that there is a certain number of sessions where the approach oscillates due to iterative feedback from the LLM. It's observed, but it's not about countermeasures.
• Moving the Goalposts (Microsoft Blog): The idea of not moving the rubric during evaluation and finalizing the rubric before starting the evaluation.

There are related topics. But what I did wasn't "finalizing the rubric"; it was "narrowing the evaluation axis to a single point: contradictions." As long as the scope is broad, points of criticism will spring up infinitely, so I'm trying to contain the scope to a finite set. I couldn't find existing research that clearly stated this.

I suspect it's written somewhere. It should be, but it's probably written in academic terms, and by the time I realized it, two months had already passed.

Simple once you're told

When I write it down, it sounds simple. "If you narrow the scope, it will converge." That's all there is to it.

But it took me two months to notice.

I kept trying to change how I wrote my prompts, thinking, "If I use Claude more intelligently, it will get better." Even when I wrote, "Don't give too many points" or "Be consistent with past feedback," it didn't work. Because the problem wasn't how I was writing the prompts.

It took time for the idea of narrowing the scope to "only finite items" to occur to me. When you audit normally, the scope is wide, so no matter what you fix, holes are found from a different angle. That was the whole story.

Conclusion

• When the audit of a plan doesn't converge, narrow the scope to "only contradictions."
• Contradictions are finite, so the process will converge.
• If you audit with a wide scope, criticism will spring up infinitely, leading to Whac-A-Mole.
• When you have the AI implement a plan that is free of contradictions, it will run until completion without stopping.

If there is anyone else caught in the same Whac-A-Mole trap, then writing this was worth it.

I feel good on days when I do something good.