Why we can't predict the weather for more than 5 days (and even that's generous)...
As Irma bares down on Florida, the cone of uncertainty at the 5th day still has a width of hundreds of miles. We've also all seen the irritating phenomena of planning a BBQ in advance only to have it rained out. You would think that with all of the technology today, we'd be far better at weather prediction then we are, but you'd be wrong, and its not for lack of effort. The fact is that its extremely complicated. Weather prediction is predicated on essentially fluid dynamics and thermodynamics.
Suffice it to say that fluid dynamics is dependent upon partial differential equations, some of the most complicated math we have. These models that take into account variables such as humidity, temperature, and many others. But also, these models must take these variables into account at each data point of interest. These data points correspond to points on the earth and in the atmosphere. We are talking millions of data points giving thousands of variables. These equations that make up these models have terms that number in the hundreds of thousands or more had they are time dependent, which means every data point must be recalculated as time progresses in relation to each other as they all depend upon each other. You get a sense of how complicated this is. The processing power required is not the stuff of desktop computers. The complexity of these models is required to account for as much phenomena as possible but its not perfect, which gives rise to error.
However that doesn't explain exactly where most of the error comes from, just how complicated these models are. Most of the error in such models (even very simple ones) arises from a mathematical phenomenon called Chaos. If a system can be measured at each infinitely small point in space, with 100% precision and accuracy, there would be no error. However, we can't satisfy either of those conditions. With that being the case, these errors grow exponentially in time, and much faster then one might imagine. I mentioned previously that, in general, there are millions of data points all over the globe taking measurements for weather forecasters. Such data is fed into super computers used to crunch the math. As a illustration of just how powerful they are, and what it takes to crunch these numbers, I shall describe the Global Forecasting System (GFS) supercomputer.
The GFS supercomputer is the machine that is used to model hurricane paths, it is one of the two main models (the other is the euro model) that weather forecasters base their predictions on. The average processing power of a processor is measured in floating point operations per second (FLOPS). The average desktop processor, such as Intel's Haswell chip, performs at about 32 FLOPS. By comparison, the supercomputer used in the GFS system performs at 5.78 peta-flops. If your metric is rusty, over the average desktop, this supercomputer processes data about 1.8 TRILLION times faster. That is the type of processing power needed for these models. The error that arises from these models compounds to fast, that our weather models aren't very useful after 5 days, even with these super-powerful computers.
Despite the complexity of the models, in the end it is irrelevant. If we pretend that we have an infinitely power supercomputer, and covered the globe in sensors every cubic foot from the ground to the edge of the atmosphere, you still could not predict the weather in New Jersey with any certainty. The fundamental problem is how this error exponentially grows in complex systems. Its simply a problem that we will not be able to over come completely, as 100% accuracy and precision does not fundamentally exist.
The cone of uncertainty in a hurricane track is based upon ensemble models put out from the euro model and GFS systems. These are many potential tracks overlaid on top of each other as they vary certain parameters to see how the hurricane path varies in time. With the compounding error arising from Chaos theory, by the time 5 days has progressed, the endpoints of the tracks span hundreds of miles laterally. That is Chaos theory in action, the compounding variability of hurricane tracks as time progresses. You can see those for the current storms HERE. It's not a phenomenon we will ever correct for completely.
- - Chaos Theory. A good book on the topic for the every day reader is Chaos, by James Gleick. I highly recommend it.