AI Product Development

Sample questions

Two questions from this practice test, in the same style as the certification exam. Answer before opening the explanation.

1. Question 1

   A marketplace receives 40,000 new products per month. In 94% of cases the supplier sends a category code that maps directly to the internal taxonomy. The remaining 6% arrive without a code. The team wants to automate categorization. Which approach fits best?

   1. Train a classifier on the full product history and apply it to 100% of the input, guaranteeing a single categorization criterion.
   2. Use a language model on the entire input, since it handles both the cases with a code and the cases without one.
   3. Apply the deterministic mapping to the 94% and use a model only on the 6% that arrive without a supplier code.
   4. Postpone automation until suppliers send a code in 100% of cases, avoiding the maintenance of two distinct paths.
   Show answer and explanation

   Correct answer: C. Apply the deterministic mapping to the 94% and use a model only on the 6% that arrive without a supplier code.

   The first question in AI product development is not which model to use, but which part of the problem already has a deterministic solution. When a reliable rule exists, it is cheaper, faster, easier to debug and introduces no variability between runs. The model should be reserved for the residual part the rule does not cover. Running a model over the 94% already solved by mapping adds inference cost, error risk and a source of inconsistency, with no corresponding gain. This pattern, solving the deterministic part with rules and the residual with a model, appears in most AI products that sustain production operations.

2. Question 2

   A support tool generates draft replies for agents. After three months, adoption is at 91%, agents edit 80% of drafts before sending, and average handling time fell 4%. Which reading best guides the next product decision?

   1. The 80% edit rate shows the model is unsuited to the task, and the feature should be discontinued before it consumes more budget.
   2. High adoption combined with frequent editing suggests the draft works as a starting point, and the gain lies in reducing the editing effort.
   3. The 91% adoption already proves the feature is a success, and the next decision should be expanding it to the remaining support channels.
   4. The 4% gain sits within the expected variation of the metric, so observation must be extended for three more months before deciding.
   Show answer and explanation

   Correct answer: B. High adoption combined with frequent editing suggests the draft works as a starting point, and the gain lies in reducing the editing effort.

   High adoption and high editing do not contradict each other; they tell you where the value is being created. If agents keep using the feature even while editing almost everything, the draft is saving the effort of starting from zero, which is usually the most costly part of the task. The common mistake is reading the edit rate as a measure of model quality and concluding failure. The correct reading is that the unit of value is not the draft that is ready to send, but the usable draft. That changes what the team should optimize: instead of chasing unedited sends, reduce the cost of editing, with structured edits, reusable snippets and learning from the most frequent corrections.