Define the purpose of a product review
⏱ Untimed topicTolkin says a review may inform an audience and create accountability, but its primary aim should be making the product better. A frightening review can work against that aim. State what the team should gain from the discussion instead of treating it as a performance before executives.
Keep the discussion small and share the artifact
⏱ Untimed topicHe generally tries to keep the conversation under ten people while distributing the document more widely. The artifact can help the whole team and new hires understand prior decisions. Broad access to context does not require everyone to join the live discussion.
Use a template to improve problem framing
⏱ Untimed topicTolkin describes context, problem, possible solution, risks and success measurement in a product review. He also distinguishes early ideas from nearly-ready launches. The template helps people think, but rigid adherence is less important than clarifying the customer's job and the stage of work.
Check experimental power before promising an answer
⏱ Untimed topicAt Opendoor, lower transaction volumes make some A/B tests harder. Tolkin asks what effect a test could detect and how long it would take. A long experiment may be worthwhile, but pretending it will answer the question sooner creates false confidence.
Seek other evidence when a standard test cannot work
⏱ Untimed topicHe discusses customer conversations, observational approaches and other ways to increase conviction. These are not interchangeable with a well-powered randomized test. The point is to name the limitation and improve the decision rather than force a statistically empty result.
Use judgment with a feedback loop
⏱ Untimed topicIf testing is not feasible, Tolkin says consequential low-confidence decisions need more customer input or another perspective. When a team ships on judgment, it should still watch signals such as support issues or adoption. Intuition is not a reason to stop checking whether the choice was right.
Find the constraint a marketplace actually needs
⏱ Untimed topicTolkin describes an early uberPOOL commuter pilot that lacked enough matching liquidity. Limiting the beta by company did not create a viable pool of rides. His lesson is to identify the operating constraint rather than defend a neatly designed pilot; the historical account is unverified here.
