Test and Learn Cards

What is it

• A simple to use template that encapsulates the experiment to run against a defined hypothesis.

• The template offers a single view of what is being tested, how it will be measured, success criteria, results from the experiment and the actionable learnings deduced from the results

Who

• The test and learn card is typically drafted by a product manager or service designer. If the experiment is technically focused, an architect may draft the card

• Once drafted it is reviewed and baselined by the whole core team, and may be shared with key stakeholders

Running the technique

• Complete the left hand side of the card - write the target outcome you are hoping for in the format of a prediction. Start the statement with ‘We believe that by (action taken) … (resulting outcome seen) ‘. This is the Hypothesis statement

• Define the tests that will be run for the experiment to prove or disprove the hypothesis. Focus on identifying a causal relationship between action taken and resulting outcome to build a high level of confidence in the efficacy of the experiment.

• Make sure your hypothesis can be tested empirically, capturing the variables that need to be measured - this ensures that the team can meet the requirements as part of developing and running the experiment

• Define the success criteria, what the outcome should demonstrate to deem the hypothesis proven. The success criteria should be definitive, with suitable degrees of confidence(>95%) where large volumes of data may be involved.

• Capture a summary of the experiment result, and the key actionable learnings to take forward to prioritise future work in Validation