Data is the decision-making engine behind every major organization. Whether evaluating next steps for your content strategy or designing a product roadmap, the metrics you choose to measure success will shape the culture and brand of your company. So, it's crucial to select the right kind of information for each decision.
Executives that value the wrong type of data can lead their organization down a perilous path. Just look at the Wells Fargo disaster. The leadership team focused singularly on cross-selling as an indicator of success, placing untenable pressure on employees to increase sales. This culture drove many to fraudulently open new accounts -- temporarily boosting revenue but ultimately resulting in a massive scandal.
When you choose the wrong metrics, you risk creating a perverse incentive (i.e., a system that causes unintended consequences by rewarding the wrong behaviors). It's quite easy to send your company in a negative direction by valuing inappropriate data.
While the following scenarios are high-level and aren't meant to cover the nuances of every situation, here are general guidelines for choosing between quantitative, qualitative, or gut-instinct data.
When you need to know why: Qualitative
Numbers are really great at telling you what is happening, but often fail to show you why something is happening.
Say, for example, your daily active users drop 20 percent in one month but tests show that everything is operating normally. If you reach out to inactive customers for feedback, you might discover that a competitor app is offering new features that your product lacks -- causing attrition. You can then find out why they like the features to design something better.
When you need to know how: Qualitative
In product development cycles, "how" is a critical part of the equation. When you're trying to solve a problem for customers, you need to know the steps they take to complete a task. If you want to outperform your competitor, you need to talk to customers to find out how you can make your product faster, easier, and friendlier.
When you need to choose between options: Qualitative and quantitative
If showing early concepts to potential customers, qualitative feedback is enough to guide decision-making. But after you launch a product or service, quantitative inputs must also factor in.
A/B testing is a great approach to refining an experience, releasing two versions of the same thing and seeing which one drives the best results. However, it's helpful to know why a certain version won -- especially before a full rollout -- which brings us back to qualitative data.
When you need to forecast performance: Quantitative
Any sort of predictive goal-setting will, inevitably, rely on quantitative data. Financial and engagement metrics are the two primary categories. Just make sure to emphasize user-centric targets as well, such as net performer score (NPS) for customer satisfaction. This will help you detect potential issues before they impact your bottom line or reputation.
When you need to test a concrete hypothesis: Quantitative
If you have a hypothesis such as "I believe X will achieve Y result," then you probably need a quantitative test. An example might be: "I think publishing 2x the number of articles per month will lead to a 40% increase in leads." By formulating your theory as such, you're able to get an objective conclusion to inform future strategy.
When you need a bold new idea: Gut instinct and qualitative
Big ideas often sound crazy. The concept of riding in a stranger's car (Uber) or staying in someone else's house on vacation (Airbnb) seemed unlikely to succeed just a decade ago. There will probably be limited quantitative data to inform a bold vision, so you have to trust your gut instinct and the feedback from early adopters.
When your values contradict the data: Gut instinct
The right choice isn't always about revenue or key performance indicators. A company's values should weight just as heavily as hard data -- if not more -- when making decisions. With board members breathing down your neck, it's hard to say no to activities that boost financial metrics. But if something feels wrong, it probably is. Trust your gut and advocate for values over short-term success.
When executives commit to decisions that align with values, it may lead to unanticipated rewards. Look at Bumble after removing sexual harassment and all imagery of guns, for example, and Nike after supporting Colin Kaepernick.
As I said in the beginning, the way you make decisions defines far more than your bottom line -- it shapes your culture and brand. So, consider the broader impact when you select your metrics for success.