Previously I gave a poor man’s description of the concept of `falsifiability‘, which is a cornerstone of what most people consider to be good science. This is usually expressed in a handy catchphrase like `if it isn’t falsifiable, then it isn’t science’. For the layperson, this is a pretty good rule of thumb. A professional scientist or philosopher would be more inclined to wonder about the converse: suppose it is falsifiable, does that guarantee that it is science? Karl Popper, the man behind the idea, has been quoted as saying that basically yes, not only must a scientific theory be falsifiable, a falsifiable theory is also scientific . However, critics have pointed out that it is possible to have theories that are not scientific and yet can still be falsified. A classic example is Astrology, which has been “thoroughly tested and refuted” , (although sadly this has not stopped many people from believing in it). Given that it is falsifiable (and falsified), it seems one must therefore either concede that Astrology was a scientific hypothesis which has since been disproved, or else concede that we need something more than just falsifiability to distinguish science from pseudo-science.
Things are even more subtle than that, because a falsifiable statement may appear more or less scientific depending on the context in which it is framed. Suppose that I have a theory which says that there is cheese inside the moon. We could test this theory, perhaps by launching an expensive space mission to drill the moon for cheese, but nobody would ever fund such a mission because the theory is clearly ludicrous. Why is it ludicrous? Because within our existing theoretical framework and our knowledge of planet formation, there is no role played by astronomical cheese. However, imagine that we lived in a world in which it was discovered that cheese was naturally occurring substance in space and indeed had a crucial role to play in the formation of planets. In some instances, the formations of moons might lead to them retaining their cheese substrate, hidden by layers of meteorite dust. Within this alternative historical framework, the hypothesis that there is cheese inside the moon is actually a perfectly reasonable scientific hypothesis.
The lesson here is that the demarcation problem between science and pseudoscience (not to mention non-science and un-science which are different concepts ) is not a simple one. In particular, we must be careful about how we use ideas like falsification to judge the scientific content of a theory. So what is the point of all this pontificating? Well, recently a prominent scientist and blogger Sean Carroll argued that the scientific idea of falsification needs to be “retired”. In particular, he argued that String Theory and theories with multiple universes have been unfairly branded as `unfalsifiable’ and thus not been given the recognition by scientists that they deserve. Naturally, this alarmed people, since it really sounded like Sean was saying `scientific theories don’t need to be falsifiable’.
In fact, if you read Sean’s article carefully, he argues that it is not so much the idea of falsifiability that needs to be retired, but the incorrect usage of the concept by scientists without sufficient philosophical education. In particular, he suggests that String Theory and multiverse theories are falsifiable in a useful sense, but that this fact is easily missed by people who do not understand the subtleties of falsifiability:
“In complicated situations, fortune-cookie-sized mottos like `theories should be falsifiable’ are no substitute for careful thinking about how science works.”
Well, one can hardly argue against that! Except that Sean has committed a couple of minor crimes in the presentation of his argument. First, while Sean’s actual argument (which almost seems to have been deliberately disguised for the sake of sensationalism) is reasonable, his apparent argument would lead most people to draw the conclusion that Sean thinks unfalsifiable theories can be scientific. Peter Woit, commenting on the related matter of Max Tegmark’s recent book, points out that this kind of talk from scientists can be fuel for crackpots and pseudoscientists who use it to appear more legitimate to laymen:
“If physicists like Tegmark succeed in publicizing and getting accepted as legitimate mainstream science their favorite completely empty, untestable `theory’, this threatens science in a very real way.”
Secondly, Sean claims that String Theory is at least in principle falsifiable, but if one takes the appropriate subtle view of falsifiability as he suggests, one must admit that `in principle’ falsifiability is rather a weak requirement. After all, the cheese-in-the-moon hypothesis is falsifiable in principle, as is the assertion that the world will end tomorrow. At best, Sean’s argument goes to show that we need other criterion than falsifiability to judge whether String Theory is scientific, but given the large number of free parameters in the theory, one wonders whether it won’t fall prey to something like the `David Deutsch principle‘, which says that a theory should not be too easy to modify retrospectively to fit the observed evidence.
While the core idea of falsifiability is here to stay, I agree with Scott Aaronson that remarkably little progress has been made since Popper on building upon this idea. For all their ability to criticise and deconstruct, the philosophers have not really been able to tell us what does make a theory scientific, if not merely falsifiability. Sean Carroll suggests considering whether a theory is `definite’, in that it makes clear statements about reality, and `empirical’ in that these statements can be plausibly linked to physical experiments. Perhaps the falsifiability of a claim should also be understood as relative to a prevailing paradigm (see Kuhn).
In certain extreme scenarios, one might also be able to make the case that the falsifiability of a statement is relative to the place of the scientists in the universe. For example, it is widely believed amongst physicists that no information can escape a black hole, except perhaps in a highly scrambled-up form, as radiated heat. But as one of my friends pointed out to me today, this seems to imply that certain statements about the interior of the black hole cannot ever be falsified by someone sitting outside the event horizon. Suppose we had a theory that there was a banana inside the black hole. To check the theory, we would likely need to send some kind of banana-probe (a monkey?) into the black hole and have it come out again — but that is impossible. The only way to falsify such a statement would be to enter the black hole ourselves, but then we would have no way of contacting our friends back home to tell them they were right or wrong about the banana. If every human being jumped into the black hole, the statement would indeed be falsifiable. But if exactly half of the population jumped in, is the statement falsifiable for them and not for anyone else? Could the falsifiability of a statement actually depend on one’s physical place in the universe? This would indeed be troubling, because it might mean there are statements about our universe that are in principle falsifiable by some hypothetical observer, but not by any of us humans. It becomes disturbingly similar to predictions about the afterlife – they can only be confirmed or falsified after death, and then you can’t return to tell anyone about it. Plus, if there is no afterlife, an atheist doesn’t even get to bask in the knowledge of being correct, because he is dead.
We might hope that statements about quasi-inaccessible regions of experience, like the insides of black holes or the contents of parallel universes, could still be falsified `indirectly’ in the same way that doing lab tests on ghosts might lend support to the idea of an afterlife (wouldn’t that be nice). But how indirect can our tests be before they become unscientific? These are the interesting questions to discuss! Perhaps physicists should try to add something more constructive to the debate instead of bickering over table-scraps left by philosophers.
 “A sentence (or a theory) is empirical-scientific if and only if it is falsifiable” Popper, Karl ( 1994). “Falsifizierbarkeit, zwei Bedeutungen von”, pp. 82–86 in Helmut Seiffert and Gerard Radnitzky. (So there.)
 See the Stanford Encyclopedia of Awesomeness.