The Falsification Filter
Evaluate extraordinary claims by trying to disprove them, not by looking for reasons to believe them.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 6
- Confidence
- 75%
Keating repeatedly applies a consistent evaluation method to everything from UFO disclosure claims to his own team's cosmology results: try to falsify the claim rather than confirm it. The filter starts from the assumption that you are the most likely person to fool yourself, especially when a result would be exciting or beneficial to you. It requires that a claim be falsifiable at all — if no possible evidence could disprove it, it isn't a scientific claim. It then applies Occam's razor to rank competing explanations by simplicity rather than drama, and asks cui bono (who benefits) before accepting a claim's framing. Finally, it insists on physical or reproducible evidence over testimony, however credentialed the source, and requires updating the hypothesis rather than defending it when new data contradicts it. Keating used this same filter on his own team's 2014 gravitational-wave claim, which he later had to publicly retract.
Origin
Extracted from The Shawn Ryan Show; Keating attributes the core principle to physicist Richard Feynman's warning that scientists must treat themselves as the easiest person to fool.
Core principles
- 01Assume you are the easiest person to fool, including about your own hoped-for conclusions
- 02A claim that can't in principle be disproven isn't a scientific claim, it's a belief
- 03The simplest explanation is more probable, not automatically correct (Occam's razor)
- 04Always ask who benefits before weighing the evidence for a claim
- 05Testimony and credentials are not a substitute for physical, reproducible evidence
- 06Update the hypothesis when new data arrives instead of defending the original position
How to run it
- 1
Assume you're the easiest person to fool
Start every extraordinary claim, including your own, from the assumption that you are the most likely one to be wrong or biased toward a desired outcome.
- 2
Test for falsifiability
Ask whether the claim could, even in principle, be proven wrong. If no evidence could ever disconfirm it, treat it as belief rather than an evidentiary claim.
Watch out A claim that adjusts itself to explain away any counter-evidence (like the astrologer who kept the same 'reading' after Keating gave the wrong sign) has failed this test.
- 3
Apply Occam's razor
Rank competing hypotheses by simplicity and plausibility, not by which one is most exciting or dramatic.
Pro tip The simplest hypothesis isn't guaranteed correct, but it's more probable than an outlandish one.
- 4
Ask cui bono
Identify who benefits if the claim is believed — attention, funding, distraction, or credibility — before weighing the evidence itself.
- 5
Demand primary evidence, not testimony
Distinguish secondhand testimony ('I heard from someone who heard') from physical, reproducible, or documented evidence; testimony alone doesn't clear the evidentiary bar.
- 6
Update priors instead of defending the position
When new data arrives, revise the hypothesis rather than protecting the original claim, even if it's your own.
In the wild
During WWII, physicist Luis Alvarez built a system that spoofed U-boat radar by transmitting a signal that weakened exactly as a real retreating object would, tricking crews into thinking a threat was moving away right before an attack. To the U-boat crew, it looked like something defying the laws of physics.
→ Applying Occam's razor to a modern 'craft that defies physics' claim should include mundane possibilities like electronic countermeasures, not just exotic ones.
Common mistakes
Starting from the conclusion
Apologetics-style reasoning begins with the desired answer and reasons backward to support it, which the filter explicitly rejects in favor of falsification-first reasoning.
Treating testimony as proof
Accepting credentialed eyewitness accounts (pilots, whistleblowers, scientists) as sufficient evidence without demanding physical, reproducible data.
Is it for you?
Best for
Anyone evaluating a high-stakes or emotionally charged claim — UAP disclosures, viral conspiracy theories, or a surprising scientific result.
Not ideal for
Situations requiring fast, low-stakes decisions where the cost of being wrong is trivial.
From the transcript
“We look to falsify things, not to prove things.”
“Occam's razor suggests the simplest hypothesis isn't always correct, but it's more likely than an outlandish or less probable scenario, right?”
“You always have to ask a question cui bono, like who benefits from it, right?”
From the episode
#316 Brian Keating - Brian Keating - The First Object Ever Found From Another Solar System