Data, data everywhere…but not enough to decide!
With the strong emphasis on data driven insights, product analytics have been on the rise and rightly so. Product managers are expected to analyze the reams of data to derive insights and understand user behavior from the data. But do we always achieve the desired outcome for product decisions from these analysis?
Can we achieve the desired “nirvana” state of product management with data alone? The answer is “No”. Are you surprised?
While data is my first love for decision making, convincing my stakeholders, negotiations and prioritizations, it is only one side of the coin. The other, that we often tend to miss is the voice of the customer.
While data driven insight is quantitative and can be substantiated, the voice of the customer is often misconstrued to be a touchy-feely subject.
With the race towards data, the most important tenet — the voice of the customer is often forgotten. Data and voice of the customer are two peas in a pod and one cannot be complete without the other.
What does this translate to?
Numbers give us insights on the usage of a product and churn rates but only tell a complete story when combined with the qualitative counterpart user research (a.k.a talking to customers). Quantitative research is inferential. Possible important nuances could be lost and in the absence of information, could play tricks on our pattern-recognizing brains.
That does not make qualitative research any better. Qualitative analysis focuses on the subjective qualities, the nuanced actions that drive the numbers. This makes qualitative research very interpretive.Qualitative data can be messy as well, but proves to provide richer insights when combined with quantitative findings
The obvious question then is when do I use one over the other, qualitative vs quantitative?
Explore and understand a phenomena — Talk to your customers to understand the pain points and the jobs to be done.Often novel ideas get generated from customerthe qualitative interviews (qualitative).
Identifying product problems and tangible product improvements — Analytics is the best tool to validate hypothesis. Feature usage, improvements, conversion rate improvements can be prioritized well with data (quantitative).
Validate customer preferences — You find a clear winner between the “red” button and the “blue” one during one of your A/B tests. Data is clear but let your customers speak as to why one is better over the other (qualitative).
Understanding trends and usage patterns — Evolving industry/market trends are visually well represented by data (quantitative).
Product reviews, improving brand effectiveness — Assessing the customer sentiments, understanding the general likeability of the product is well done by customer feedback, be it from social forums, interviews or targeted surveys (qualitative)
Cost-benefit analysis — You have a feature hypothesis and need to prove a point to the executive leadership. While qualitative data is great, this case calls for hard numbers (quantitative).
Data can falsely sometimes simplify an issue while qualitative user research can magnify a trivial issue without the data. It is imperative to empathise with the user, understand the root of the problem and the context of the data acquired.
Marrying user research and product analytics together gives birth to a great product. Data cannot be undermined. It is one of the most important tenets, but only when combined with the voice of the user.