They got the broad shape OK – but temperatures turned out about 5 degrees cooler than the BoM expected.
Now the real world below.
The max temp Outlook still exaggerated expected warmth but was pretty fair for Outlooks, and the rain Outlook was realistic – you can make maps to compare at this BoM page.
Anyone else think there is something suspect about the BoM’s temps for the Perth area? Only this morning, my daughter and I were talking about what a cold spring it has been. Yet the BoM shows min and max anomalies above average for the last month and 3 months.
I have been recording night time minimum for months, and comparing it with the nearest (15k) official BOM site. The main purpose was to check on interior and exterior temperatures for my little greenhouse.
I don’t regard the readings as accurate, but sometimes they coincide with the BOM forecast. Other times one wonders if they used a dartboard to guess their predictions. For the last fortnight I’ve noted: -1.8, +1.6, +0.7, +1.3, +0.6, -0.9, +0.7, -2.3, -1.9, -4.3, +1.7, -0.9, -1.3, +2.2. Range between -4.6 and +4.4.
I’m not sure it’s as bad as 5 degrees, but the BoM is running far too warm all right.
Unless I am missing something, the BoM is still not putting an actual temperature figure on their temperature forecasts. All they say is there is x per cent chance of it being warmer than normal. That’s a pain in the neck, because it doesn’t really tell you anything definite. If it turns out 3 degrees colder than normal, and they predicted an 80 per cent chance of warmer than normal, they just say, oh well, the 20 per cent got up this time, bad luck. Really bad luck if frost just wiped out your seedlings.
Still I think it’s fair enough to infer that if they say there is more than a 50 per cent chance of warmer than average, then they fail if it’s cooler than average. If they say 70 or 80 per cent chance of warmer than average, and it turns out cooler, then they fail big time.
In this case, it was wamer than average over most of WA and SA, which they said it would be, and it was cooler than average in Far North Queensland, which they also said it would be. The rest of the continent – fail: they said warmer than usual, by up to a 75% chance, and it turned out cooler than usual.
Warwick, you look at these maps a lot. I remember seeing quite a few terrible comparisons in other posts, where the BoM got practically the whole continent wrong. But do you notice if they tend to predict too warm as a general rule, especially for night-time temps? That’s the impression I get. Wouldn’t surprise me if they have got some bulldust global warming model behind these forecasts somewhere telling them that with so much more CO2 in the air, the next 3 months just have to be warmer than the long-term average, practically all the time.
Your assertion that the outlook graph predicts temperatures exceeding the real world results by up to 5 degrees is a blunder based on a misreading of the scales. The colours of the outlook map refer to probabilities of temperatures exceeding the long term median. The colours on the real world map refer to the actual temperature anomily in degrees C.
You cannot subtract 0-1 degrees measured temperature covering the white area in central Australia from the red 75-80% probability that the temperature will be above average and come up with a difference of 5 degrees in temperature.
Ed. note: I know the two maps are contouring different data.
But my “about 5 degrees” is the only useful takeaway I can see out of it.
Readers can add their own comments.
I am sorry if you do not like the content here – but it is easy to move elsewhere or start your own blog if you think the BoM Outlooks need fairer reviews than the criticisms I have provided. – It is still a free country I think. IMHO the BoM Outlooks are utterly useless.
All the best.
You are misrepresnting the charts. In the area surrounding Perth, the BOM estimated an 80% chance of the daily minimum termperatures, in the 3 month period, exceeding the long term average. As you will appreciate, this did occur, ie the temperatures did exceed the long term average.
In the Canberra region, the BOM gave a 65% chance to the long term average being exceeded, but this did not occur. Their prediction was wrong, but forecasting, expcially over a long period, is an inexact science.
The point from all this is that there is no point. The authors have not misrepresented anything, they have not reported any incorrect figures, they just expected warmer minimum temperatures. This is *not* an error in IPCC climate science
This thread is a waste of cyberspace.
Ed., regarding your comment to my earlier post, I did not realise that comments were reserved on this site only for those who agree with your analyses. I thought it was for genuine discussion.
I am a scientist and my comment is no different to what I would say to a presenter in question time following a seminar or conference talk in which a slide with that data and interpretation appeared. That is how science works. The audience would be astounded by a response from the presenter that because I question that slide, I should go to other talks or conferences where I will find nothing to question. (How do know until I hear the talk and should I shut up there too?) Alternatively I should organise my own conference and only invite speakers and allow comments I agree with.
But hey, as you say it’s your blog and you can run it as you like.
Hey dude, you are not comparing apples with apples, or rather minimum temperature anomaly projections vs. real world anomaly.
So your headline is in error. And your thinking.
My replies to critical comments.
First what “the truth” says –
Re Perth the actual minimum anomalies were les than + 1 degree.
So the 80% predict obviously portended much warmer conditions. BoM fail.
Re Canberra the BoM 65-70% chance of exceeding normals portended a medium high anomaly – result was medium cool – BoM fail. If 3 months is too long a period – then the BoM should cease publishing Outlooks.
I know forecasting is inexact but the IPCC compliant main-stream-media publishes BoM Outlooks announcements as “written-on-tablets-of-stone” Gospel – they never examine the reliability of Outlooks – so I fill that gap.
I pay the bills for my share of cyberspace.
BoM and IPCC bad science leeches off taxpayers.
People are free to start blogs praising the BoM and IPCC – and I am sure they do.
Second what “Wadard” said –
It is obviously relevant to compare anomaly maps with the equivalent Outlook period – as I do – because that is what the BoM links to at the bottom of their Outlook pages.
See at bottom of this BoM page –
These links –
Maximum temperature departures from average for July to September 2012 – base period 1961-1990
Minimum temperature departures from average for July to September 2012 – base period 1961-1990
I respect the views of dissenters, and can see their point that you cannot directly compare percentage chances of warmer conditions with degrees Celsius above or below average. BUT:
1. Why doesn’t the BoM make the blooming things comparable in the first place? They must surely be able to predict 1 or 2 degrees above or below normal just as easily as this silly stuff about a 45% or a 65% per cent or whatever chance of above normal.
2. The point being examined here is whether the BoM predictions are reliable. There is a simple test of this, even given that the two metrics are different. Namely, on what share of the continent did the BoM get it right: what percentage of Australia was either warmer than usual or cooler than usual in line with their prediction? I make it pretty well dead on 50% in this case. Most of WA, SA and TAS are OK (predicted warmer, and were warmer), so is the Cape York Peninsula (predicted cooler, and was cooler). Virtually all the rest is wrong. This 50% success rate seems a typical result from other comparisons I have seen. But I can average 50% right any time I like by cutting the continent in two and tossing a coin for which bit will be warmer and which bit will be cooler. So Warwick’s point stands in my opinion: these forecasts are useless.
Warwick – do please pay attention here. The BOM chart shows its prediction of the probability of the daily minimum termperature being above the long term average minimum (LTAM). It does not say by how much. Are you clear so far ? Good. The temperature around Perth did exceed the LTAM. Your statement
>>Re Perth the actual minimum anomalies were les than + 1 degree.
>>So the 80% predict obviously portended much warmer conditions. BoM fail.
is incorrect, a fact that you and you alone seem to be having trouble with. To restate the obvious, there is *no* indication on the chart of how much warmer than average the BOM expects the termperature to be, that has come from your own fertile imagination.
You seem to be confused by the fact that the two charts use the same colours – if the area in the predicted chart is red, then you have decided that that area should be 5-6 degrees above the LTAM, but no, it’s just the BOM’s assessed probability that the area will be at least 0.01 degree above average. Look at the chart again, and then let’s just admit your mistake and move on.
Please note that I don’t want or need to set up a blog praising either the BOM or the IPCC, I am posting here to point out your mistake.
the truth –
A friendly tip: you will never convince anyone of anything with such a patronising attitude.
On the substance of the issue, I think Warwick is well aware, as we all are, that the charts show different things. The prediction is a percentage chance of warm weather, while the outcome is given in degrees warmer or cooler than normal.
The question being raised here is: how reliable are the BoM’s predictions? Do you have anything to contribute on that issue?
Warwick’s view is that they are useless. His logic is, if the Bureau is as sure as they can be (i.e. more than 80% sure, their highest confidence level) that it is going to be warmer than usual somewhere, then you would expect it to be noticeably warmer, not a few tenths of a degree. I think that’s a fair enough inference. After all, we feel actual temperature in degrees, not percentages of probability of warm weather. If we are to judge the reliability of the BoM’s predictions at all, then we have to use some comparison of their percentages with temperatures. The only alternative is to say the predictions are untestable, recognising that untestable predictions are by definition useless.
I have suggested another way of comparing the forecasts with the outcome. Namely, what share of the continent did the BoM get right, i.e. successfully predict to be either warmer or cooler than usual? If it’s only 50%, then that is no better than rolling dice. It seems to be around 50% pretty much all the time, including here. That’s why I agree with Warwick that these predictions appear useless.
Does anyone here think the BoM predictions are worth anything? If so, why?
Hey David Brewer,
I had to write gently for Warwick; if you look at his earlier rebuttals, he is impervious to people pointing out that he has misread the charts. Or perhaps he is just too proud to admit his mistake.
You might agree with his assumption that a high probability of above average temperatures therefore means that the temperature should be well over the LTAM
>> I think that’s a fair enough inference
but, and here is the point, this *is* an inference. The BOM makes no claims for it.
Note that he method of argument tha Hughes is using, and that you are condoning –
** I independently, and with no basis from the published work, assume that the paper means ** X, and X is incorrect, so I conclude that the paper is incorrect
has no validity whatsoever.
For the very last time, the BOM was predicting the probability that the minimum temperatures would be above the LTAM. That is *all* they were doing. The 5 degree
difference come from Hughes’s imagination.
Your own point is also incorrect, we are not debating
>>how reliable are the BoM’s predictions
we are looking at the two charts that Hughes has cited. As you will know, you cannot go from one specific example to a generality.
If you were to provide a number of these charts, taken over many years, and you could prove that they were all running at 50% or less accuracy, then you could make your argument. Attempting to dismiss all the BOM’s work on the basis of a single instance is intellectually dishonest.
ps – is ‘David Brewer’ a nomme de guerre for Warwick Hughes ?
Here is a published paper:
Verification and value of the Australian Bureau of
Meteorology township seasonal rainfall forecasts
in Australia, 1997–2005Note in the Abstract they mentions, “The results indicate that the forecasting system had low skill”.
and, “…uniformly close to the no skill value.”
For me it is telling that the BoM Outlook pages invite readers to compare Outlook maps with their anomaly maps in the case of temperature and decile maps in the case of rainfall. See towards the end of comment 10.
Seven years on from the above paper it would be useful to have an independent statistical expert assess and score BoM Outlooks against actual real world weather.
One could ask is “the truth” a misnomer for climate alarmist Dr David Jones of BOM whose predictions have been not only exaggerated but completely wrong. I suggest there are some at BOM like Dr Jones who either have no understanding of technology (eg fluid dynamics, heat transfer, thermodynamics etc) required to assess the mechanisms of climate and weather, or are just putting out misinformation for political purposes.
Warwick has a great record of assessing and presenting physical measured data (temperature, pressures, precipitation etc) to the extent that Dr Phil Jones of UAE in the climategate emails is shown to refuse to allow Warwick access to the HADCRUT original data in case Warwick discover errors in the compilation and manipulation. Nobody takes note of UAE anymore preferring the KNMI database.
Because of the political stance and interference in technical output by the likes of Dr David Jones and the present chair(person) BOM has lost its reputation. BOM are in breach of their Charter. Until the top management is swept aside and honest scientists are allowed to operate without interference they should just collect and openly provide weather and climate data for others (more competent) to assess.
“the truth” – you have contributed nothing useful here.
Gosh, cementafriend, thanks for the ringing endorsement.
I have no views whether Mr Hughes does or does not have a great record of assessing and presenting physical measured data, but, and please note what I am saying, in this post, he has misread the data he has presented and he has reached a false conclusion. If you cannot see this, then I would suggest that it is you who has contributed nothing to the thread.
The premise ‘Warwick has done some really good things’ does not lead to the conclusion ‘therefore he is aways right’.
ps – no, I am not David Jones.
Beachgirl
Interesting paper, thanks for the link, but, alas, not relevant to this thread. What you suggest would indeed be interesting, and perhaps Mr Hughes might include this in his busy independent research schedule. As I tried to point out to David Brewer, you can’t just criticise one BOM publication, (especially when you misinterpret it), and then conclude that everything that they, the BOM, do is rubbish.
ps I am failrly sure that you are not Hughes, although I think that cementafriend might well be. 🙂 I am wondering if Warwick, either as himself, or perhaps as cementafriend or even as David Brewer, could elaborate on the title ‘earth scientist’ that he claims in the limited references that I can find for him on the web.
Just asking …
Interesting stuff coming out here. I didn’t know Warwick had been banned from the UEA site, but it does seem to be an unfortunate habit of some of the more enthusiastic warmers to respond to critics with “mushroom treatment”.
However, back to the point. Warwick, the truth seems to concur with my argument that if the BoM has a persistent 50% or so success rate, then that is evidence that their predictions are useless. Could you give us the links to all your earlier posts on this so we can see how many comparisons you have made and what they show? Or have you done any overall assessment, or a string of them over a substantial period?
Very interesting also that the BoM rainfall predictions were assessed years ago – by two blokes in Vet Science departments? – and found to have “low skill”, i.e. useless. So why keep putting them out years later with minimal caveats? The main likely users would be farmers making decisions on the basis of likely rainfall. Seems they would be just as well advised to ask the local tea-leaf reader. You could not expect temperature estimates to be any better, since the same models will be involved, and if rainfall is wrong temperature is bound to be wrong also.
One more point. Let’s go back for a moment to how the temperature predictions are made. The BoM will produce, say, 20 model runs of temperatures across Australia for the next three months. They subtract the normals to get anomaly maps. But they don’t publish those anomaly maps, which would be directly comparable with the outcome maps and show exactly how accurate their prediction was for anywhere on the continent.
Instead they further process the model runs to get probabilities of warmer or cooler than average. I don’t know exactly what method they use. The simplest would be to just report the percentage of the models that predicted warmer weather for each spot, so if there were 20 model runs, 80% chance of warmer means that 16 runs predicted it. But there could be bells and whistles – excluding outliers, ignoring predictions close to normal etc.
But why go to percentages at all? It chucks out information and the resulting probability maps are far less useful than predicted temperature departures. We don’t want to know what chance they estimate of warmer than usual if that can mean anything between 0.1 degrees warmer and 5 or 8 degrees warmer. We want to know if it’s going to be average, a bit warmer, very warm, very cold or whatever.
Call me a cynic but the only reason I can see for the BoM not predicting temperature anomalies is that they know their predictions are no good. Percentages are an ideal way of keeping the modelling in the public eye while concealing the fact that it’s useless. If this is the strategy, it’s working. Look at all the time we are spending arguing over whether the percentages can be lined up with temperatures, instead of discovering whether we are getting anything useful here for our tax dollars.