Product Discovery

Sample questions

Two questions from this practice test, in the same style as the certification exam. Answer before opening the explanation.

1. Question 1

   In a review meeting, an important customer asks for a button to export the order list to a spreadsheet. The team records the request in the backlog with the literal wording received and classifies it as a customer requirement.

   1. Build the button, because the request came from an important customer and the implementation is simple
   2. Investigate what the customer does after exporting, because the request describes a solution
   3. Refuse the request, because product decisions should not start from customer requests
   4. Put the request to the other customers in a vote, to measure demand for the same button
   Show answer and explanation

   Correct answer: B. Investigate what the customer does after exporting, because the request describes a solution

   What arrived was not a problem: it was a solution the customer formulated out of the tools they know. The information that matters sits one layer below, in what happens after the export. If the customer exports to calculate an indicator the product does not offer, the right answer is the indicator. If they export to send it to someone with no access to the system, the answer is sharing. If they export to check a number they do not trust, the answer is the reliability of the data, and the button would merely make the distrust cheaper to act on. Building it straight away is attractive because the request is specific, cheap and comes from whoever pays, which gives the sense of an easy, well-founded decision. Refusing on principle is attractive because it sounds disciplined, and it errs by discarding the only signal available: the request is a legitimate clue, it just is not the destination. Putting it to a vote is attractive because it seems to scale the evidence, and it only measures how many people recognise a familiar solution, telling you nothing about the problem behind it. Investigating the subsequent use costs one conversation and frequently reveals that the correct solution is not the one requested.

2. Question 2

   Two complaints appear in the support channel. The first occurs about 340 times a month and the user works around it in a few seconds. The second occurs about 12 times a month and, in the reports, leads the user to abandon the sign-up without completing it.

   1. Prioritise the first, because frequency indicates how many people are affected by the problem
   2. Prioritise the first, because contact volume is the most reliable signal available in the channel
   3. Treat the two as equivalent, because both were reported spontaneously by users
   4. Estimate the cost of each occurrence before prioritising, because frequency does not measure severity
   Show answer and explanation

   Correct answer: D. Estimate the cost of each occurrence before prioritising, because frequency does not measure severity

   Frequency and severity are independent dimensions, and the support channel only measures the first. A trivial annoyance repeated 340 times may cost less, in total, than 12 abandoned sign-ups, because each abandonment eliminates a user before they exist. The calculation that settles the comparison is simple: number of occurrences multiplied by the cost of each. Without that second factor, prioritising by volume is prioritising by what is easiest to count. There is also a recording asymmetry that worsens the error: workable problems generate contact because the person stays in the product and wants to complain, while problems that drive the user away generate silence, since whoever left does not open a ticket. Prioritising the first complaint by frequency is attractive because the number is large, verifiable and easy to defend in a meeting. Prioritising by the reliability of the channel is attractive because the data really is objective, and it confuses the precision of the measure with the relevance of what it measures. Treating the two as equivalent is attractive because it seems impartial, and it refuses exactly the comparison that prioritisation requires.