We live in very interesting times, especially if you are a theoretical physicist like me. To understand what kind of time we are living in physics-wise, it will be helpful to review some ideas of Thomas Kuhn, a famous philosopher of science. Kuhn described science as proceeding through a series of paradigms. A `paradigm’ is a sort of established framework in which scientists work to solve problems using an agreed-upon set of tools. The paradigm provides both the puzzles to be solved and the tools to solve them. Over time, scientists discover that the tools of the paradigm cannot solve every puzzle. The problems that lie beyond the reach of a paradigm are called anomalies. When enough serious anomalies are discovered, scientists begin to lose confidence in the existing paradigm and a crisis occurs. Historically, each crisis has been resolved by a subsequent scientific revolution, in which the old paradigm was replaced by a new paradigm that is capable of resolving the anomalies .
Interestingly, although the new paradigm solves more problems than the old paradigm, it also represents a complete change in perspective, so that even those problems that were solved by the old paradigm have to be `re-solved’ by the new paradigm, from a completely new point of view. As a result, there might be the odd puzzle that was solved by the old paradigm but suddenly cannot be solved by the new paradigm! This phenomenon is known as `Kuhn-loss’. The new paradigm is successful so long as it solves more important puzzles than the ones it loses through Kuhn-loss. I mention this only to illustrate how significant a change in paradigm is from Kuhn’s point of view: it is not merely a period of accelerated science, but a complete reworking of how scientists see the world.
We are currently in a period of crisis. Some physicists might disagree with me, but I think one can make a strong case that the paradigm that has taken us this far is showing cracks. In this post, I won’t directly compare current events to Kuhn’s description of a crisis, nor will I spend effort trying to define what the present paradigm is. For the moment I will content myself by pointing to some (just a few!) of the major puzzles that are facing us, and explain why they may represent `anomalies’ that require a new paradigm in order to solve them .
Dark matter / energy: One of the best-known puzzles of our time is the mystery of dark matter and dark energy in cosmology. Briefly, the matter that we can see in the universe (galaxies, nebulae and so on) is moving around as though it is being pushed and pulled by gravitational forces that have no visible source. In fact, there seems to be 95% more `stuff’ in the universe that we can’t actually see directly – we can only deduce its presence by its gravitational interactions with visible matter. The fact that we don’t know what this stuff is has been called the most embarrassing problem in physics for good reason: if somebody asks me what kind of matter and energy there is in the universe, I have to admit that, for the most part, I have no freaking idea.
Quantum gravity: Going by Kuhn’s picture of science, the key tool of the present paradigm is the Standard Model (SM) of particle physics. This model is impressively accurate down to really tiny scales and has been spectacularly confirmed time and time again in the world’s big particle accelerators, right up to the recent discovery of the Higgs Boson at the Large Hadron Collider (LHC). However, a major limitation of the Standard Model is that it does not tell us how gravity fits into the picture. While we have brought electromagnetism and the nuclear forces up to date with quantum mechanics, our theory of gravity is still straggling behind by over a hundred years. All the other forces have been given a quantum makeover, but gravity remains the shy stepsister, cloaked in a classical veil. Despite some pioneering attempts to get behind that veil, most notably String Theory and Loop Quantum Gravity, there is still no agreement among the community about which approach is correct or whether we have to try something else entirely .
Quantum foundations: It is often said that nobody understands quantum mechanics. This would be very worrying if it were true, since much of today’s technology is based on it! So what is the situation really? Well, obviously we understand the theory well enough to use it in practical applications. The trouble is more on the philosophical side: physicists can’t agree on why quantum mechanics works so well. In fact, we still can’t agree on why the universe should be quantum mechanical in the first place! John Wheeler’s famous question `why the quantum?’ still keeps many of us awake at night. There is an ongoing body of research on quantum foundations, whose goal is to improve our understanding of quantum mechanics to the point where most of us can agree on a single interpretation. This interpretation (it is hoped) would reveal quantum mechanics in such a way that nearly every physicist will reflexively slap their forehead and declare `of course! It had to be that way’! The interpretation should be so compelling that classical physics will look absurd by comparison and quantum mechanics will be the most natural way to describe the world.
As an example, since Einstein, the gravitational force is now widely interpreted as the curvature of space and time. However, technically it is possible to explain gravity in terms of fields operating in flat spacetime, in a way that agrees with current experimental data – yet if you ask any physicist what gravity is, nearly all of them will say `the curvature of space-time due to matter’. By contrast, if you ask them what the wave function of quantum mechanics is, you will get all kinds of different answers, and probably an invitation to a conference on foundations where such matters are still being hotly debated. Whereas curved space-time seems like an elegant, simple and compelling way of visualizing gravity, we have no similarly compelling paradigm for visualizing quantum mechanics.
One of the tasks a physicist faces during a crisis is to identify which anomalies deserve our attention and which ones are less important. This decision is guided by one’s intuitions and one’s chosen philosophy, hence a physicist must embrace some philosophy in order to make progress. For my part, I am most interested in the latter two anomalies: quantum gravity and quantum foundations. I think that the two are deeply connected. Since the regime of quantum gravity is still far from being accessible to experiments, the success of a theory of quantum gravity will be decided by the intuitive appeal of the physical principles on which it is based, as well as its elegance and explanatory power. We cannot hope to meet these demands all the way down at the level of quantum gravity (the Planck scale) if we still can’t do it up here on our home turf for quantum mechanics. Indeed, it is embarrassing that we cannot claim to have such a compelling picture of quantum mechanics, given that we have so much experimental data to guide us!
In upcoming blog posts I intend to elaborate on quantum gravity and quantum foundations and their possible connection to one another. I will also present my own ideas about how we should try to resolve the connected anomalies, using a philosophy based on a modern revival of operationalism and ideas from the exciting new field of quantum information. Stay tuned!
 This is a very rough version of Kuhn’s picture of scientific progress. The reader is encouraged to read the entry on Thomas Kuhn in the Stanford Encyclopedia of Philosophy: http://plato.stanford.edu/entries/thomas-kuhn/ . The less lazy reader is referred to Kuhn’s seminal work The Structure of Scientific Revolutions, University of Chicago Press, 2nd ed. (1970).
 There are of course far more anomalies in physics than the three listed here, although many of them can be linked to the same broad categories. For a more thorough list, see John Baez’s `Open Questions in Physics’: http://math.ucr.edu/home/baez/physics/General/open_questions.html .
 Some people have gone as far as to argue that String Theory is a failure. As an ignoramus, my own stance on this is more cautious, but that is a topic for another blog post.