Software Teams Should Require Two Peer Reviews For Code Merges

TL;DR: Analysis of 70 software peer reviews reveals that the first code reviewer contributed important feedback to 100% of code reviews, the second reviewer added important feedback to 65 (93%) of code reviews, but the third reviewer added significant feedback to only 3 (4%) reviews. Code reviews are an important part of the software development process, and for optimal returns on time invested, teams should use two approving reviewers per code merge.

Peer code reviews are a standard practice in software engineering, supported through lightweight (asynchronous) processes, such as GitHub’s pull requests (PRs). Despite wide adoption, the return on investment of peer reviews is not widely understood. In my discussions with hundreds of engineers, I discovered that most didn’t know why their team required one, or two reviews per PR, and whether increasing the number of reviewers would be a good time investment. Back in 2013, while working as a software engineer at Amazon, I attempted to answer this question for my team. Initially I researched various studies, articles, and books, but after failing to find metrics that would answer the question conclusively, I did a quick analysis of my team’s code reviews. Below is the summary of my findings.

Bad Merges vs. Number of Reviewers per Code Review
1. PR had 3 approving reviewers.
2. PR had 50 to 400 changed lines of code.
3. PR included business logic changes (no style, or config only changes).
1. Logic errors that result in unexpected behavior (ie: null pointer exception or not satisfying a business requirement).
2. Significant architectural issue requiring code reorganization (ie: to remove duplication of code), or re-design to address issues with implementation approach (ie: a non-scalable solution).
70 (100%) of PRs had at least one significant comment.
65 (93%) of PRs had a significant comment from the 2nd reviewer.
3 (4%) of PRs had a significant comment from the 3rd reviewer.

Technology leader. Passionate about software architecture and growing effective technology teams. Currently @ InstaCart. Previously @ Amazon.