Posts Tagged ‘Forecasting’

We learn about climate only when the models are wrong!

March 29, 2013

When a forecast based on a mathematical model is correct, we learn nothing.

A mathematical model is merely a theory, a simplification of reality or an approximation to the real world. By definition a mathematical model is a hypothesis.  When forecasts are incorrect, we can return to our model and improve it and make a new hypothesis. A forecast is then a test of the model but in just one particular set of circumstances. Being correct does not prove the theory behind the model. It does of course add to the body of evidence that the model may be a satisfactory representation of reality and it does allow further forecasts to be made without tweaking the model. For learning to take place the mathematical model must be the falsifiable hypothesis of the scientific method.

It seems to me that Solar Science has a much healthier (scientifically) attitude to models and forecasts than “Climate Science”. When observations don’t match a climate forecast, the observations are impugned rather than the models being improved. This is, I think, because the forecast climate results have been used to establish huge revenue flows in the political arena (whether as taxes or carbon credits or just as research funding). There has been a vested interest in denying the observations and calling the science “settled”. Once the science is “settled”  the climate forecast and its underlying model become sacrosanct and take on the certainty of prophecy. Instead of being falsifiable hypotheses, climate model forecasts have taken on the character of unfalsifiable prophecies!

No scientist would presume to claim that we know or understand all solar effects. Or that we know and understand the role of the oceans or of the water vapour and dust and aerosols in the atmosphere. “Climate” is contained in the thin, chaotic layer of atmosphere which surrounds us. Yet “Climate Science” makes the arrogant assumption that the effect of trace amounts of carbon dioxide on climate is known definitively. Filling a real greenhouse with higher concentrations of carbon dioxide does not make that greenhouse any warmer than one filled with normal air – but the plants do grow faster with access to the additional CO2!! But – claim the climate priesthood –  in the real atmosphere, carbon dioxide causes other forcings (clouds? aerosols? precipitation effects?) which maximise warming which means that our model is still valid. Why not just admit that we don’t know what we don’t know?

The behavioural issue of course is whether it is worth trying to control something as poorly understood as climate rather than ensuring that we have the wherewithal to adapt to whatever changes may come. Another ice age will surely come whether in 10 years or a 100 years or 2,000. It will then be our ability to harness all available energy sources around us which will determine our capacity to adapt.

Learning from forecasts when they are wrong – not just in science but also in business and project management and technology development – has long been a hobby-horse of mine and is why forecasts need to be wrong.

When there is no difference there is no learning.

  • I take prophecies to be a promise about the future  based primarily on faith and made by prophets , witchdoctors, soothsayers and politicians such as ”You will be doomed to eternal damnation if you don’t do as I say”,
  • I take “forecasts” to be an estimate of future conditions based on known data with the use of calculations, logic, judgement, some intuition and even some faith. They are extrapolations of historical conditions to anticipate – and thereby plan for -future conditions.

……. Over the last 30 years I have spent of a lot of time conducting and participating in reviews. Reviews of research projects, of construction projects, of organisations and processes, of designs, of strategies and action plans, of businesses and of companies. The common features  in all these different reviews, that I have found the most penetrating, have been the comparisons not only between forecast values  and actual values, (which may be any values indicating performance and capable of being extrapolated), but also between past forecasts and current forecasts.

Whether considering construction progress or costs or sales figures or cash flow or profit or number of patents applied for, it is the differences between forecast and actual values, or values forecast before and values forecast later which have led to learning. In all these fields we are in the area of the behaviour of complex systems; and where people and their behaviour is involved any system is inevitably a complex system.

When a forecast is fulfilled there is usually an air of congratulation, satisfaction and self-adulation and this leads to a deadly complacency that everything is “settled science” and well understood. In any enterprise of any kind, that kind of complacency is the kiss of death. It is the differences which lead to questioning, to proper scientific scepticism, to further investigation and ultimately to an increase of understanding and – perhaps – a better forecast. (Of course, ignoring all such differences  and to merely “continue as before” can be equally fatal).

Which brings me to climate (which is not a science by any stretch of the imagination) and solar cycles. They are both in the realm not only of where “what we know is a great deal less than what we don’t know” but they are also both in the region where “we don’t even know what we don’t know”. We do not even know all the questions to be asked. They are both complex systems where – by definition – the complexity lies in the multitude of the processes involved and their interactions.

When climate – which is contained in the 100 m of ocean and 20 or so km thick, turbulent and chaotic atmospheric layer (and which is dimensionally miniscule in relation to the 140 million km of the earth-to-sun system) – is so complacently considered to be “settled science” then we have shifted into the area of faith and soothsaying and prophecies. When climate modellers are smug enough to believe they have understood the climate system and believe that their models are complete, then the models produce outputs which are not forecasts but prophecies. (No doubt soothsayers and shamans have sometimes made accurate prophecies but I still would not buy a used car from one of them)! Weather is in the realm of forecast (though you could argue that the most accurate forecast is still that “the weather tomorrow will be like today”) but climate is not yet there.

This kind of “arrogance” which pervades some of the climate “scientists” is not so prevalent when it comes to the study of Solar Cycles. There is a clear understanding that “we don’t know what we don’t know”. In addition to the 11 year and 22 year cycles, other cycles are hypothesised for 87 years, 210 years, 2300 years (or maybe 2241 or 2500 years) and 6000 years. We have no idea what causes these cycles. Even the 11 year cycle which has been most studied produces  surprises every day but is properly in the area of “forecast” (and hopefully never again will be in the area of prophecy). ….

…… We seem to be in a solar minimum. We may be seeing a 210 year cycle – or maybe not. There are changes to the forecasts not only regarding the maximum level of sunspot activity but also about when it will occur and what the length of cycle 24 might be. There is speculation as to what effect the length of the solar cycle may have on climate – but we haven’t a clue as to what mechanisms may be involved.  This is not to say that there isn’t much speculation and hypothesising. There is a great deal of comment about the effect these changing forecasts may have on global warming or cooling or climate disruption.  In some quarters there is much glee that the forecasts have been “wrong”. Some comments question the intelligence of the forecasters.

But of course the forecasts themselves say nothing about how the behaviour of the sun may impact our climate. They do not pretend to be prophecies or to be statements of inevitable outcomes. All they do say is that we don’t know very much – yet – about the sun. But we do know enough to make some tentative forecasts.

But I am very glad that people continue to be brave enough to make forecasts and I am quite relieved that the forecasts are not spot on. That at least ensures we will continue learning.

UK Met Office fears ridicule from public more than from their paymasters!

January 5, 2011

The UK Met Office is busy spinning the story that it actually did forecast the coldest December in the UK in a 100 years but secretly informed only the cabinet of the UK government about this in October 2010. Secret forecasts for fear of being wrong! After all kings of old also had their own private soothsayers to study the entrails but they were usually executed if they could not spin their way out of wrong forecasts. But the Met Office story does not stand up and their credibility is in tatters.

The UK Met office (as an institution) is one of the most ardent supporters of Global Warming dogma which is concerned with climate not weather. This can only be a “political” choice or an act of faith since climate trends are of little significance for their main task of weather forecasting. While weather is only a subset of climate I find it difficult to believe that poor weather forecasting can be a sound bottom-up basis for forecasting climate.

I say that climate trends are of little significance for detailed weather forecasting because climate change considers temperature changes of about one degree per century or less whereas the daily variation at any particular location is typically 10 to 15 degrees, seasonal variations at any location are around 40 – 50°C over a year and geographical variation around the globe is also upto 50 °C at any moment in time. Any climate trends of the order of 1°C per century are then immaterial for the immediate weather forecast.

At a cost of some £200 million per year they do forecast the weather with some accuracy for upto about 5 days ahead but are notoriously poor with their long range forecasts (but I note that even their short term forecasts are not more accurate – statistically – than the simple statement that “the weather tomorrow will be the same as today”). As recently as October 2010 the Met Office published weather maps showing warm expectations for November, December and January – but they insist they did not make any long range forecasts. We are told by Harrabin of the BBC that this was because of their sensitivity to the ridicule poured on them after their forecasts of a “barbecue summer” in 2009 and a mild 2009/2010 winter proved spectacularly wrong. But , we are assured by Harrabin, they actually did forecast – correctly – a cold and snowy winter but only informed the UK cabinet secretly.

Presumably any future ridicule or budget cuts by the cabinet of the UK Government for being wrong would be preferable to any public ridicule!!

In any event the cabinet did not do very much with this confidential information (perhaps it was anonymous) and all the counties were woefully unprepared.


Why Forecasts need to be wrong

October 7, 2010


The Lorenz attractor is an example of a non-li...

Image via Wikipedia


This started yesterday as a short comment on the changing forecasts by Hathaway on solar activity in Solar Cycle 24 but has now become something else.

As clarification, I  distinguish here between prophecies and forecasts  where:

  • I take prophecies to be a promise about the future  based primarily on faith and made by prophets , witchdoctors, soothsayers and politicians such as “You will be doomed to eternal damnation if you don’t do as I say”,
  • I take “forecasts” to be an estimate of future conditions based on known data with the use of calculations, logic, judgement, some intuition and even some faith. They are extrapolations of historical conditions to anticipate – and thereby plan for -future conditions.


Habitable planet to be discovered in May 2011?

September 22, 2010

The only thing certain about forecasts is that they are more often wrong than right – and I exclude forecasts made entirely on known science or “laws” of nature where the level of uncertainty is insignificant (e.g. the sun will rise tomorrow). Nevertheless “Future History” which is a study of how forecasts evolve and how accurate they have been is a most powerful tool when making judgements about directions to follow and actions to be taken. In management “Future History” methodology is, I think, one of the most powerful tools for the development of corporate strategies and action plans.

The New Scientist reports that :

“Two researchers have used the pace of past exoplanet finds to predict that the first habitable Earth-like planet could turn up in May 2011”.

In 1965, Intel co-founder Gordon Moore observed that the number of transistors that fit on a chip doubles about once every two years – a trend now known as Moore’s law. Samuel Arbesman of Harvard Medical School in Boston wants to see if scientometrics – the statistical study of science itself – can similarly be used to not only study past progress but also to make predictions.

He and Greg Laughlin of the University of California, Santa Cruz, are testing the idea with exoplanets. Over the past 15 years or so, the pace of planet discoveries has been accelerating, with some 490 planets now known. “It is actually somewhat similar to Moore’s law of exponential growth,” Arbesman says.

To predict when astronomers might find the first planet similar in size to Earth that also orbits far enough from its star to boast liquid water, the team scoured the discovery records of 370 exoplanets.

They focused on two basic properties needed for habitability: a planet’s mass and its surface temperature. They used these two factors to assign each planet a ‘habitability metric’ ranging from 0 to 1, where 0 was uninhabitable and 1 is close to Earth’s twin.

A rough estimate of each planet’s habitability was then plotted against the date of its discovery. Using different subsets of the 370 planets, the researchers made curves from the individual points and extrapolated the curves to find when a planet would be found with a habitability of 1. They then analysed the range of discovery dates to determine which would be most probable.

Habitable planets:

Their calculations suggest there is a 50 per cent chance that the first habitable exo-Earth will be found by May 2011, a 75 per cent chance it will be found by 2020, and a 95 per cent chance it will be found by 2264.

In fact, exoplanet researchers have made forecasts of the future informally, plotting the mass of planets against the date of discovery to see how the field is progressing. “We’ve done that for many years at conferences,” says Eric Ford of the University of Florida in Gainesville. “The new aspect of this paper is putting an uncertainty on those predictions and unfortunately the uncertainty is quite large.”

One source of uncertainty is how factors like changes in funding and the development of new techniques and technology can alter the pace of discovery. “Like the stock market, past returns are no guarantee of future performance,” Ford says.

“There are always these complex factors of how science is actually done,” Arbesman agrees. But he says the forecasting technique could still prove useful, even if these factors are not accounted for directly. In part, that is because new technologies tend to take a while to ramp up, so they may not lead to sharp jumps in the number of discoveries made.

Previously, Arbesman has quantified how the ease of discovering new mammalian species, chemical elements, and asteroids affects the rate of their discovery. New species and asteroids are more difficult to find the smaller they are, and indeed larger ones are found first. For chemical elements, the opposite is true, since the bigger they are, the rarer and more unstable they tend to be.

Spanish citizenship for Pablo the Psychic Octopus?

July 10, 2010

Though Paul, who was born in the UK predicts that Germany will win tonight’s third place match, he has also predicted a win for Spain in tomorrows final.

In Germany there are now various suggestions of how Paul should be sliced, squashed, marinaded, fried, broiled, grilled or baked.

All of Paul’s predictions have been true in this World Cup. In Spain there are moves  to rechristen him Pablo and provide him with Spanish citizenship. Spain’s defensive midfielder Sergio Busquets said Friday that the Spaniards are in love with the octopus and and out of gratitude they want to re-christen it Pablo.

“We guess it is a Spanish octopus. We are in love with it and want to name it Pablo,” said Busquets.

Paul’s predictions require the choice of mussels but whether he will take to paella is not known. Paul’s religious and political affiliations are also not known.

But I think Paul is quite safe. His keeper’s at Sea Life Oberhausen are seeing a marked increase in the number of visitors to the aquarium and it it can be expected that Paul’s psychic abilities will be tested in new fields when the World Cup is over.

%d bloggers like this: