Sample report · Metrics and execution
Instagram Reels watch time dropped 10% week over week. How would you investigate?
This is what you get after every mock. It's a real report from Soreno, written for a sample answer we recorded, not a user's.
63 / 100
Fair
PM skills
Summary
You established a useful diagnostic framework, including data validation, a strong watch-time decomposition, and relevant segmentation. You also recognized that watch time is not an end in itself. The main weakness was adapting the investigation to new evidence: after performance and crash differences were ruled out, you proposed rollback rather than testing another explanation.
Your weakest PM skill
Finds the cause and the fix
Ranks the likely causes or options, says how to confirm one, and lands on a fix or a clear decision.
In this mock
You identified plausible internal and external causes and offered a concrete initial test using release notes, crashes, video start time, and version comparisons. However, after the interviewer ruled out performance and crash differences, you still proposed rollback or a hotfix without establishing a cause.The report then offers a question that trains this skill, and tracks it across your mocks.
What worked
- You correctly put measurement validation before diagnosing user behavior.
- You decomposed watch time into actionable drivers rather than treating the 10% decline as a single metric.
- You showed sound product judgment by distinguishing watch time from the broader value the product should deliver.
What to fix
- 1. This proposes a remedy before establishing a cause, right after the interviewer says performance and crashes show no difference. A version-level correlation would not yet justify rolling back the whole release.
- 2. The trade-off is relevant, but it does not answer the interviewer's request to isolate the feature or experiment causing the decline. Across the follow-ups, the investigation stalls rather than adapting to new evidence.
- 3. I looked for a concrete test of the initial data-quality hypotheses. You mention logging, pipeline delays and dashboard definitions, but not how you would rule them out.
The interview, out loud
Interviewer: Instagram Reels watch time dropped 10 percent week over week. How would you investigate?
Candidate: First I'd rule out data issues: a logging change, a pipeline delay, or a changed dashboard definition. Then internal changes: an app release, a ranking model update, or an experiment ramping up. Then external causes. Finally I'd break watch time down into daily users, sessions per user, reels per session and seconds per reel, and segment by platform, country, app version, and new versus existing users.
Interviewer: With that plan in mind, where would you start looking first, and why?
Candidate: Say it's concentrated on Android, on the latest app version. I'd check crash rates and video start time, and compare users on the old version against the new one over the same days.
Interviewer: What if you find no significant difference in performance or crashes between the two versions? Where do you go next?
Get your own report on your first PM mock.