GUILDENSTERN: ...Four: a spectacular vindication of the principle that each individual coin spun individually is as likely to come down heads as tails and therefore should cause no surprise each individual time it does.
—"Rosencrantz & Guildenstern Are Dead" by Tom Stoppard
There has been a lot of bru-ha-ha over the last few days about the much anticipated discovery of what looks to be the Higgs Boson at CERN. Among many other things that you have probably read is the statement that the confidence that the signal is real is 99.9999%. You might be wondering, why so many 9's? That is, they had a signal a while back that was already 99% or thereabouts certain. If I had 99% confidence in winning the lottery I would go out right now and spend $1000 on lottery tickets. Why was a 99% confidence limit not good enough to indicate discovery? Indeed, the announced discovery, with 99.9999%, is at the statistical confidence level that is considered the minimum for a particle physicist to announce a discovery. Why do they have to be so damn confident?
Rather than talking about the energy spectra of interaction cross sections, let's talking about flipping coins. At the opening of Tom Stoppard's play Rosencrantz & Guildenstern Are Dead, the two courtiers are flipping coins (and have been doing so for some time). They are approaching a streak of 100 flips of heads in a row. Rosencrantz (who wins a coin each time it comes up heads) is not concerned about this, but Guildenstern is so disturbed by the seeming violation of the laws of probability that he philosophizes at length about what it is that's going on. (The real thing that's going on is that he's a character in a play, not a real person.) Let's keep it more modest, though.
Suppose I were to walk up to you with a quarter, and flip it six times in a row. If the quarter is normal, and if I'm not cheating, the probability that all six flips of the quarter will come down heads is about 1.5%. In other words, if I do flip six heads in a row, you can be 98.5% sure that it was not due to random chance, that I must have been cheating somehow. (Ask me to show you this sometime.) You're not 100% confident, because there is a small chance that six heads will come up in a row just randomly, but it is a very small chance... and so you would be well within your rights to think that something was probably up. It may not be good enough to convict somebody in a courtroom, but it's certainly good enough to bet on.
Suppose instead, however, that 30 people come up to you, and each one of them flips six coins in a row. The probability that at least one of those people will flip six heads in a row is 38%. So, while it won't happen every time this crowd of coin-flippers accosts you, you shouldn't be particularly surprised that somebody flipped six heads in a row if a whole bunch of people tried it. Even though it's extremely unlikely that any given coin flipper will flip the coin six times, the probably that somebody somewhere will is entirely reasonable. Lightning has to strike somewhere. (See Randall Munroe's much more concise take on this, and on overreactions to it.)
This same principle applies to particle physics. The particle physicists looking for the Higgs Boson were not sure at exactly what energy the particle would show up. Here's one of the plots from the CMS collaboration:
The signature of the Higgs Boson is the extra bump of events at an energy of 125 GeV. There are lots and lots of events at all energies in the plot; there's a little something extra there, which indicates that something is going on there, and that something is probably the production of a short-lived Higgs boson. But they didn't know before they found it to look right at 125 GeV; it could have been at other energies, too. If all they were after was finding something that was "a little extra" at 95% confidence, they could have found it lots of places; indeed, there's a data point hanging out there at a bit over 135 GeV that is that far away from the background. But since there's 30 data points in the plot, I'm not the least bit surprised to see that. Randomly, you'd expect to see at least one of those more than 3/4 of the time somebody showed you a plot like this with 30 data points, even if there are no new particles.
The physicists in these collaborations were doing the equivalent of looking at a whole bunch of people flipping coins, and trying to find somebody who was flipping more heads than tails. If you look at 30 people who flip 6 coins and you find one person who has flipped 6 heads in a row, you have no right to declare that you've found a person who is cheating at flipping coins; the chances of that happening randomly are too high. Similarly, if you look at a whole bunch of different energies, and you see a single place where more is going on to 99% confidence than you'd expect from random fluctuations, you don't have much confidence that you've really found anything... because if you look at enough different energies, you will eventually find the unlikely random fluctuation. This is why for a particle physicist to be confident that she really has discovered something, she needs six nines in her confidence.
As for why the Higgs field (the "same thing" as the Higgs Boson... it's complicated) gives particles mass... that I really don't understand.