review · Evaluation, Problem Solving · 30–60 min · 2-10 people · medium energy

Searching for Disconfirming Evidence/Following Up Exceptions

Instead of treating outliers as noise, this method encourages facilitators to actively seek out and investigate data points or cases that don't fit the expected pattern. These exceptions can provide valuable insights into other causal factors or limitations of the intervention.

When to use Searching for Disconfirming Evidence/Following Up Exceptions

Use this method when you want to gain a deeper understanding of the factors influencing an outcome, beyond the main intervention. It's particularly useful when you suspect that there are other important variables at play.

What it solves

Oversimplified explanations, ignoring important contextual factors, lack of understanding of complex dynamics

How to run Searching for Disconfirming Evidence/Following Up Exceptions, step by step

  1. Identify outliers or exceptions to the expected pattern. (15 min)
  2. Investigate each exception to understand why it occurred. (Ongoing)
  3. Look for patterns or commonalities among the exceptions. (15 min)
  4. Revise your understanding of the causal factors based on the exceptions. (15 min)
  5. Document the exceptions and the revised understanding. (15 min)

Materials needed

  • Data
  • Analysis reports
  • Case studies

Facilitator tips

  • Encourage participants to be curious and open-minded.
  • Avoid making assumptions about the reasons for the exceptions.

Common pitfalls

  • Dismissing exceptions as irrelevant. Take the time to investigate each one.
  • Failing to revise your understanding based on the exceptions. Be willing to change your assumptions.

Variations

  • Use a fishbone diagram to explore the potential causes of the exceptions.
  • Conduct interviews with individuals involved in the exceptional cases.

Running it online or hybrid

Use a shared document or spreadsheet to track exceptions and disconfirming evidence. Assign owners to follow up on each item.

What it produces

A list of exceptions and disconfirming evidence, along with explanations for why they occurred.

Origin

Attributed to BetterEvaluation.org — source

Build a session around Searching for Disconfirming Evidence/Following Up Exceptions

METODIC places Searching for Disconfirming Evidence/Following Up Exceptions inside a complete, timed session plan — agenda, worksheets, facilitator guide and slides — matched to your group and goal.

Generate a session with Searching for Disconfirming Evidence/Following Up Exceptions · Try METODIC free