TThe Shawn Ryan Show
← All frameworks
Strategy

The Falsification Filter

Test extraordinary claims by trying to disprove them before trying to believe them.

Difficulty
Easy
Time to result
~days to results
Steps
5
Confidence
90%

Keating applies a falsification-first approach to UFO claims, conspiracy theories, and his own scientific work. Start by assuming that desire, status, or excitement can make you fool yourself. State the claim in a form that evidence could disprove, then compare it with more ordinary explanations and use Occam's razor to rank probability rather than drama. Ask who benefits from the claim's acceptance, and distinguish eyewitness testimony from physical or reproducible evidence. The output is not permanent certainty but an evidence-weighted position that changes when the data changes. Keating's retracted BICEP2 result demonstrates why the method must apply most strictly to conclusions a researcher wants to be true.

Origin

Extracted from The Shawn Ryan Show; Keating connects the method to Richard Feynman's warning about self-deception in science.

Core principles

  • 01Treat yourself as the easiest person to fool
  • 02A useful claim must be capable of being disproved
  • 03Prefer simpler explanations before exotic ones
  • 04Physical and reproducible evidence outweighs testimony
  • 05Update the hypothesis when contrary evidence arrives

How to run it

  1. 1

    Make the claim falsifiable

    Write the claim precisely and identify an observation that would count against it.

    Watch out If every possible result can be explained away, the claim cannot be tested.

  2. 2

    Generate ordinary alternatives

    List plausible mundane causes before elevating an exotic explanation.

    Pro tip Include instrument error, environmental contamination, and deliberate deception where relevant.

  3. 3

    Rank by evidence and simplicity

    Use physical, documented, and reproducible evidence to compare explanations, with testimony treated as weaker support.

    Pro tip Use Occam's razor as a probability guide, not as proof.

    Watch out Credentials do not convert testimony into physical evidence.

  4. 4

    Check incentives

    Ask who gains attention, money, influence, or strategic advantage if the claim is believed.

    Watch out An incentive is a reason for scrutiny, not automatic proof of falsehood.

  5. 5

    Update the conclusion

    Change or withdraw the claim when stronger evidence supports a competing explanation.

    Watch out Protecting reputation by defending a failed claim compounds the original error.

In the wild

BICEP2 signal becomes galactic dust

Keating's team announced a signal interpreted as gravitational waves from cosmic inflation. Later analysis showed that galactic dust could mimic the sought pattern, so the extraordinary interpretation had to be withdrawn.

→ The team retracted the claim rather than treating the original announcement as untouchable.

Radar spoofing mimics impossible motion

Keating describes wartime electronic countermeasures that made a target appear to retreat on radar. The apparent violation was produced by manipulated signals, providing a mundane alternative when modern sensor reports seem physically impossible.

→ Electronic deception remains a hypothesis to test before concluding that a craft defied physics.

Common mistakes

Starting with the desired answer

Searching only for confirming evidence turns inquiry into a defense of the conclusion.

Treating testimony as proof

A credible witness can justify investigation without supplying reproducible physical evidence.

Is it for you?

Best for

People assessing scientific results, UAP claims, conspiracy theories, or other claims supported mainly by authority and testimony.

Not ideal for

Low-stakes choices where a full evidentiary review would cost more than a wrong decision.

From the episode

#316 Brian Keating - Brian Keating - The First Object Ever Found From Another Solar System

→