What’s the deal with engineering intelligence?

I remember the first time I came across “software engineering intelligence” as a tool category. I was on a call with one of our customers and our champion told me about a brand new solution they were evaluating to help them better track their engineering productivity. The tool was LinearB—now one of the most recognized names in the category—and they were just getting ready to sign-off on a multi-thousand seat pilot.

I couldn’t help but feel a twinge of envy.

These products seemed to have a magic key that we didn’t. They went straight to leadership’s desks, while we fought an uphill battle to expand from team-level adoption to leadership buy-in. They commanded impressive price tags because they were connected to the million-dollar question of measuring engineering performance and productivity. Meanwhile, we were often viewed as just another commodity solution in a crowded market.

The difference was stark: while we were celebrating getting engineering teams to adopt our tool, these metrics products were being championed in board meetings and executive reviews. They had somehow cracked the code on proving their value at the highest levels of the organization, often commanding six or seven-figure contracts while traditional project management tools like us fought over five-figure deals.

But over the last couple of years, I’ve started to notice a shift in this category. This crystallized for me during a CTO meetup we hosted last week focused on engineering productivity. The conversations revealed some fascinating trends about where we are with engineering metrics – and more importantly, where we might be headed.

The Data Hoarding Dilemma

There’s an ironic tension in the current state of engineering metrics: we’re simultaneously drowning in data and thirsting for actual insights. Companies have gotten remarkably good at measuring everything that moves in their engineering organizations. They’ve built or bought sophisticated dashboards that track every conceivable metric. Yet many engineering leaders confided that they feel like expensive data hoarders, sitting on mountains of numbers but struggling to extract meaningful value.

The CTOs we spoke with described dashboards showing sprint velocities, commit frequencies, deployment rates, and dozens of other metrics – all meticulously tracked and trending over time. But when asked how these metrics influenced their decision-making, many admitted to feeling overwhelmed. Some even described a kind of analysis paralysis, where the sheer volume of data made it harder, not easier, to make confident decisions about their engineering organizations.

From Metrics to Action

The most engaging discussions weren’t about what people were measuring – they were about what people were doing with those measurements. The room lit up whenever someone shared how they translated a troubling metric into concrete organizational changes. There’s a palpable hunger for practical examples of metrics driving real improvements in engineering organizations. Based on the conversations, it’s a hunger that most tools on the market aren’t satisfying.

Several leaders shared compelling stories about how they moved from measurement to action. One described how identifying a pattern of increasing code review times led them to implement a “reviews before meetings” policy, where engineers were encouraged to handle pull requests first thing in the morning before jumping into standups. The key insight wasn’t just about having the metrics – it was about having the organizational wisdom to interpret them correctly and the political capital to drive change.

The Narrow Lens Problem

A recurring frustration among CTOs at the meetup was how current engineering metrics focus narrowly on code delivery while missing the bigger picture. “We can tell you our deployment frequency down to the minute,” one leader noted, “but we can’t measure if we’re building the right things or if they’re maintainable.” Most tools completely miss critical phases like discovery, design, and long-term maintenance.

Some teams are starting to tackle this gap.

A few CTOs shared experiments measuring broader metrics like concept-to-value time, feature usage vs. maintenance costs, and system stability over time. But the consensus was clear – the industry needs tools that measure value delivery through the entire software lifecycle, not just code throughput. As one CTO put it: “We’re measuring the assembly line when we should be measuring the product lifecycle.”

The Rise of the Engineering Ops Role

One of the most intriguing trends from the conversations was the emergence of dedicated operations roles focused on metrics interpretation. More and more engineering leaders mentioned they were looking at hiring positions like “Chief of Staff to CTO” specifically to make sense of team data and recommend actions. This suggests that the challenge isn’t just gathering metrics – it’s having the organizational capacity to interpret and act on them.

These roles are becoming crucial bridges between raw metrics and organizational change, combining data analysis skills with deep engineering knowledge and organizational savvy. What’s particularly interesting is how this role is being structured differently across organizations – some creating dedicated engineering operations teams, while others embedding these capabilities within existing engineering excellence or platform teams.

The common thread is a recognition that turning metrics into improvements requires dedicated focus and specialized skills.

The Trust Factor

Trust in metrics emerged as a crucial concern. Engineering leaders are grappling with a fundamental question: if their underlying systems aren’t maintained properly, how can they trust the metrics they’re using to make decisions? When we dug deeper as a group, the challenges were multifaceted – from inconsistent issue tagging to incomplete git commits and varying definitions of what constitutes a “bug” versus a “feature.”

Several leaders shared horror stories of making decisions based on metrics that later turned out to be flawed due to data quality issues. It was only after a directional change had been made that they realized issues with the underlying data. There’s growing interest in how AI might help improve data accuracy and hygiene, ensuring that metrics actually reflect reality. While we’re still in the early days here, the appetite for a solution is real.

Where Do We Go From Here?

The engineering metrics category is at an interesting inflection point. We’ve mastered the art of collection, but we’re still learning the science of action. The next evolution won’t be about gathering more data – it will be about making that data truly actionable and ensuring it drives real organizational improvement that we were all promised during the demo.

For engineering leaders, the key questions have shifted from “What should we measure?” to “How do we act on these measurements?” and “Who in our organization should own this process?” It’s becoming clear that the tools that win in this space won’t be the ones with the most comprehensive dashboards or sophisticated measurements. They’ll be the ones that help engineering organizations bridge the gap between measurement and action, between data collection and actual improvement.

The million-dollar problem isn’t measuring engineering productivity anymore – it’s turning those measurements into meaningful change.