AI is Changing Code Reviews

Before AI, there were 2 humans in the loop. The code reviewer and author.

The code reviewer would leave comments. The code author would read the comments and learn from them so that next time they are better. Or sometimes, the code reviewer would learn as well. Using ML terms, the reviewer and author would somehow revise the weights in their model (i.e. their brain).

This interaction incentivized me to add lengthy comments to code reviews. It made me believe, perhaps naively, that I was leveling up the team so that next time we make a similar change, it will be better, cleaner, faster, etc.

Code reviews have always felt like a long term investment to me. I read and leave comments and gradually my coworkers and I will produce more tasteful, performant, and correct code.

However, with the state of AI tools today, this long term play falls apart.

After AI, there is now only 1 human in the loop: the code reviewer.

The code reviewer would leave comments. The code author would then ask their AI agent to address the comments. Comments are resolved, the AI pushes new commits, and everyone is happy right?

The problem with this is that no weights are updated either biologically (in my coworker's brain) or logically (in the LLM). In other words, no one learns.

This is the frustration I've been feeling with code reviews lately. I feel more like a TSA border agent watching the display on the x-ray machine rather than a design partner aligning on mental models.

I have found myself adjusting to this new reality by using AI to combat AI.

After AI, there is 1 human in the loop: the code reviewer, but the AI model can learn too.

If there is only 1 human in the loop and my coworkers are so AI-pilled and outcome driven that they're not going to update their own biological weights from my comments, then screw it, I'm going to influence the AI model by providing it context.

I've been trying to update all the standard places that AI agents seem to consume (AGENTS.md, CLAUDE.md, etc.) with common feedback that I leave in code reviews like:

• useEffect isn't necessary here.
• Exhaustively handle all union variants with never type.
• Make sure you are following our API design guidelines.

I guess it's sort of like I'm creating some fuzzy lint rules. I'm also considering adding long-prose code architecture docs in the AI prompts.

I've been updating prompts to some code review bots so that I can play the UNO reverse card on AI-maxxed coworkers.

After AI, there are 0 humans in the loop producing and reviewing the code. But there are 2 humans in the loop influencing the AI.

I'm not at this stage yet but it feels like where things are naturally tending. I can see a team continuing to update the prompts to AI codegen and AI code review and building a compendium of instructions or rules for the AI agent, sort of like what exists today with company-specific lint rule packages.

Maybe we'll be back to 2 humans in a loop again as we argue over the correct way to prompt the AI agent or the correct best practices to add to the prompt.