Metrics
Metrics and their critical place in the Product Manager's toolbox.
For those of you who took a high school shop class, the term “Measure twice, cut once” should sound familiar. The concept is that, by measuring the piece of wood you are preparing to cut to the exact length necessary for whatever thing you are trying to build, and then measuring it a second time, you dial in the certainty that the cut is at the exact right place. In my high-school shop class, failure to execute a successful measure twice, cut once exercise would almost certainly be followed by two things:
A board too short for your project, thus wasting the piece of expensive hardwood that you had already purchased and requiring additional funds to be sunk into your project.
The dreaded shop teacher dad joke “Looks like you better go get the board lengthener from the toolbox!”
Software product development feels similar.
Developing, amending, deprecating, or really doing anything to software without metrics is like setting off on an ambitious woodworking project without a tape measurer. You can create a beautiful design, get close enough when selecting rough lumber, and have a world-class shop at your disposal but as soon as you go to make that first cut, any hopes of an efficient and well-executed outcome are dead on arrival without the use of a tape measure.
Metrics are essential in every aspect of the process.
Financial metrics for comparative analysis when writing product or business requirements documents enable appropriately prioritized business objective roadmaps. They also help in the go, no-go decision-making for greenlighting products or features.
Consistent velocity metrics allow for accurate MVP capacity estimates. Roadmaps fall apart almost immediately without a solid understanding of resource capacity.
Standardized user interview questionnaires generate contextual data for usage and feature requirements. Understanding how the users use your product is much different than understanding that they use it in general.
Well-engineered usage metric dashboards allow for real-time user activity and trend data to continuously fuel product decisions. What percent of users enter this feature? What is the time to value? Is this feature worth the cost?
Rich, cumulative data sets empower Product Managers to pull the plug on features or products that have outlived their useful lives. Justifying behaviors and decisions gets much easier when you have numbers to back up your suppositions.
It is important to note that, as with all data-related endeavors, the age-old adage “garbage in – garbage out” applies. Creating sound, reliable, and useful metrics should start with sound, reliable, and useful questions. Understand the problem you are trying to solve and then go dig up the data to test your hypothesis. This conservative approach to data gathering brings up data collection methods, of which there are many.
Our shop uses Mixpanel, Google Analytics, PowerBI, some remnant Tableau dashboards, and numerous other tools at the SRE, data logging, and cloud computing levels that this post won’t get into. The sheer breadth of platforms can be overwhelming just to familiarize yourself with and justifies the embedding of the critical role of business analytics within the software development org. With so many tools and literally endless data to capture, process, and review, less is absolutely more. As with software development, start small when building out your metrics dashboards, stick to your MVP, and iterate your way to success. I have seen many dashboards and reports that have impressively cascading widgets and graphs only to find that most of the data is just noise.
Software is complex. Solving complex problems requires a fundamental understanding of the tools, system dynamics, and the problem itself to solve effectively. Asking the right questions will get you far but having metric-derived data to validate the answers to those questions will get you much farther.