How to measure developer productivity (ft. the productivity grader)

Note: Zenhub’s Productivity Grader has since been renamed to the Free GitHub Analytics tool and can be accessed for free here. 

At Zenhub, we spend a lot of time thinking about developer productivity – from automating workflows to reducing context switching to gaining insights into productivity through reporting.

As we continue on this productivity mission, we’ve also thought a lot about how to make measuring team productivity more effortless and accessible to everyone.

This month, we’ve made big strides in that vision: we’re excited to announce the release of our free-to-access Software Team Productivity Grader. Yes – free-to-access – meaning no Zenhub account or credit card is needed to use it. It’s our gift to you (aww, you shouldn’t have!).

The productivity grader is like a productivity measurement “hack,” if you will – it will make accessing your team’s productivity stats (like average # of issues completed, average review time, etc.) easy by uploading your GitHub repo URL. If you’re already eager to give it a go, you can try it here.

If you’d rather stick around, in this blog, we’ll discuss how to measure developer productivity and why. We’ll also get some insights on productivity measurement from Andrew Forward, lead developer on the productivity grader. Let’s get into it!

Why measure developer productivity?

Productivity measurement in software development is critical to a deeper understanding of how your team works and how to improve a team’s processes. This insight allows leaders, developers, and other product stakeholders to:

• Identify the need for more resourcing
• Make more effective process improvements by comparing previous performance against current performance
• Identify gaps and bottlenecks in the current process (e.g., Where in a pipeline the team is getting stuck/wasting the most time)
• More accurately predict release and project end dates
• Identifying wasteful tasks and streamlining them so devs can focus on innovation

Developer productivity metrics for measuring performance

The following is from the Merriam-Webster dictionary:

productive – adjective

pro·​duc·​tive | prə-ˈdək-tiv , prō-

: having the quality or power of producing, especially in abundance

If being productive is the “power to produce,” what metrics indicate this in software development? When creating the productivity grader, our team came to an agreement on what it all boils down to. Andrew Forward explains: “Measuring productivity comes down to how responsive your team is, how quickly a team moves from idea to production, and how their throughput measures in weekly cadences.”

Keeping the principles for productivity Andrew mentioned in mind, the team identified these key productivity metrics for representing performance:

• # of issues completed per week and the ratio of open to closed issues
• # of PRs merged per week and the ratio of open to closed PRs
• Issue cycle time (from issue creation to closure)
• Code review time (time between a PR being opened and merged)

Data for these metrics can be sourced from a project management or issue-tracking system that your team uses, ideally one that’s well-integrated with your codebase for accurate data. If you choose to use the productivity grader, it sources data directly from GitHub, so you never need to worry about accuracy!

When developer productivity metrics matter…and when they don’t

So, how important are metrics? The importance of metrics comes down to how the metrics inform and indicate team performance.

Before we get into the benefits of metrics, you should note that metrics can sometimes be counterintuitive: “It’s important that teams using productivity metrics aren’t changing the way they work to see an improvement in the metric rather than in their productivity,” says Forward. After all, we’ve all seen “vanity metrics” before – in software engineering, “the number of lines of code produced,” for example, might result in developers trading quality code for quantity.

When we created the productivity grader, we focused mostly on metrics teams could leverage as indicators they’re moving in the right direction or are blocked and metrics that measure a team’s overall capacity. “Something like cycle time is critical for creating project timelines and highlighting potential disappointment to your client base or other stakeholders. A metric like PR time is great for highlighting bottlenecks in your reviewing process and letting you know when it’s time to introduce processes like pair or mob programming,” says Forward.

Now, I know what some of you are thinking – Issues completed and PRs merged can be flawed metrics. What if a team’s issues are giant, or PRs vary in size based on who is working on them? What if Cycle Time is inaccurate due to old issues sitting in the backlog? These are all valid concerns – we know that measurements aren’t always perfect. The goal of the productivity grader is not to perfectly measure your team’s productivity (there’s no way we can do that without you working in Zenhub first). But instead, give your team an indication of how your historical trends rank relative to the broader industry. These high-level scores indicate whether you are directionally on track or have an opportunity to review your team’s process.

How to measure developer productivity using benchmarking

Measuring productivity is more than just tracking metrics. Metrics start to become valuable when placed within a context. For software teams, two contexts you will want to consider for benchmarking are:

• How the team performed previously
• How other software teams are performing in comparison

Measuring productivity = tracking + benchmarking

After a few sprints, teams can compare their current performance against previous ones. If you’ve been running a few sprints but have not kept track of prior performance – don’t fret! You can use a tool that gathers historical data already in GitHub, like the productivity grader, which collects data from the past 4 weeks.

For more robust productivity tracking, you can also try Zenhub’s velocity tracking and other agile reports, which are automatically generated when you’re working in Zenhub.

Finding software development productivity benchmarks from other teams

Using industry standards or benchmarking against other software teams can be useful for several reasons, such as:

• More accurate benchmarking for teams that lack historical sprint data
• Maintaining a competitive advantage against other tech companies
• Getting a “health check” of how effective your team is using an outside perspective

Despite being an essential element for reporting in many circumstances, gathering accurate data on how other software teams perform can be challenging. To make this easier for teams, we included a productivity score inside the productivity grader, which gives teams a weighted grade based on data collected from the top 100 most productive repositories.

Here’s how we identified the most productive software teams: “We started by looking at the top repositories in various categories – the top most starred, most active, most forked, etc. We then included the productivity grader metrics as a filter to identify top-performing teams,” says Forward.

The result? The productivity grader is one of the most accurate tools for benchmarking your team because it updates its list of top 100 repositories monthly using GitHub-derived data, so you’re constantly comparing your team against real data from the most productive teams.

Other resources for benchmarking software developer productivity:

Software Development Benchmarks and Metrics for Measuring Productivity, by Zenhub

The State of Scrum: Benchmarks and Guidelines, by the Scrum Alliance

The 2021 State of the Octoverse, by GitHub

Curious to see how productive your software team is? Test out the productivity grader

When it comes to measuring developer performance, it can be challenging to know where to start or how to benchmark it. Focusing on metrics that are functional in improving productivity, like Cycle Time or Review Time, can pinpoint blockers and help you accurately plan project timelines while avoiding the pitfalls of “vanity metrics.”

Ensuring both the accuracy of the metrics and of the benchmarkers is also critical for accurate productivity measurement. We always recommend using data derived from your codebase and vetting the source of benchmarking data before using it to measure your team’s performance.

To make reporting and benchmarking a lot easier, give the productivity grader a try! Simply input your public or private GitHub repo URL, and we’ll instantly give you access to your team’s productivity stats and benchmark them against a pre-vetted list of the top 100 most productive repos.