date: Wed Dec 3 13:09:24 2008 from: Phil Jones subject: Re: NASA to: Tom Wigley Tom, Here is a paragraph I got from Tom Peterson at NCDC overnight The story I've gotten about the Oct. CLIMAT is that we received two sets of CLIMAT messages from Russia. The first had September's data in the October message. The second had October's data in October's message. When I was in charge of GHCN, we had a policy to always replace with the latest data as that should be the correct value. However, there were some problems with this, for example, when a country would transmit a correction to SLP and only transmit that - letting the rest of the fields just be set to missing. Apparently the system as it was working last month did not replace with more recent data so these September values stayed in the data set. NASA was blameless as they just use GHCN. GHCN has been in serious need of maintenance and improvements (e.g., to incorporate all the CLIMAT messages, to account for changing station numbers, etc.) for quite some time. But instead of spending time and energy improving GHCN version 2, the plan was to create GHCN version 3 instead. Unfortunately, GHCN version 3 has taken longer than anticipated. Cheers Phil PS Russia is the only country to send its CLIMAT data in large groups (as opposed to altogether for the whole country). This is why it was only part of Russia that was affected. This also explains why the skeptics think this is all down to GHCN. NCDC must injest the climate messages into GHCN. It isn't GHCN that is at fault, but there logic. What I do is that if new data come in it replaces old. If it comes in missing then you keep what you might have had! Seems fairly trivial to me! Tom, This is all a storm in a tea cup. GISS shouldn't have got it wrong, but far too much is being made of it by skeptics and their journalist friends in the US. Much of what this report states is garbage. GISS had some of the Russian data in wrong for Oct 2008. This comes about from trying to do things too soon after the month. GISS pick up the data from NOAA/NCDC and it is based on a mixture of CLIMAT (the monthly averages) and SYNOP messages off the GTS. Attached is what MOHC produced from these two sources for October for the CLIMAT and for the SYNOPS. The latter is dodgy as you have to decide how many days are needed to get a complete month. The CLIMAT comes through on the 4/5th of the month and again later between the 16-20th (in this case during November). GISS try to get this out just after the 5th - as the UAH/RSS get theirs out quickly. We wait till the 20th or 21st. People shouldn't be looking at individual months, but the skeptics do to keep saying that there has been no warming since 1998. To counter this you just have to look at the 1991-2000 average versus the 2001 to 2008 average. GISS blamed NCDC for not doing enough QC on what was coming in, but I reckon this isn't right. It is clear that many of the Russian data for October were September's data. When they spotted the mistake they reran the gridding and put the revision up. The skeptics then said some other parts of the world had altered and GISS got worried. What they had forgotten was that CLIMAT data keeps coming in during the first 20 days of the month, so the second time they ran the gridding they had more data in other parts of the world - so that is why other areas appeared to change. What also happens is that the SYNOPS come in first (they do each day during October), GISS must replace these with the better CLIMATs as they come in during November. CRU/MOHC don't use the SYNOPS. MOHC are doing some experiments with these data to update daily series for extreme temperature and precip analyses - and one of checking these is the comparison on a monthly basis with the CLIMATs. The real issue is why did NOAA/NCDC pick up the wrong CLIMAT or SYNOPS for October (September's values for 90+ stations in Russia). According to MOHC, the Russians put the right data out. Countries often put the previous months data out by mistake - mainly in Africa and South America. None of the GISS or NOAA/NCDC software tests for this, but it's easy to see if you look at the precip amounts, which they won't be doing. Gavin says something went wrong at NOAA/NCDC, which is reasonable. It is hard for GISS to pick up September's data just for 90 stations in Russia, when they get the whole month's data from NOAA/NCDC. I've made a note to ask Russ Vose, Dave Easterling or Tom Peterson when I next email one of them. It is probable it was in DC though and not in Asheville. Our numbers are on our web site. CRUTEM3 (land only) for Glob/NH/SH they were 0.78/0.86/0.71 wrt 61-90. So October was quite warm. Globally only 2004-2007 were warmer - since 1900. The blogs had a lot about telling NOAA/NCDC and GISS where they can pick up more data off NMS web sites. These are only for a few countries though. The blogs seem to have no idea of the GTS (never mentioned anywhere). There seems to be a belief that NOAA produces the monthly averages - but the CLIMATs are produced in the countries. NOAA does from the synops, but replaces these as the month progresses. It all stems from trying to do things too quickly after the month. Our (MOHC and mine) are all automated. MOHC do look at the maps - like those attached. Cheers Phil At 03:35 02/12/2008, you wrote: Phil, I presume you have seen the attached. Can you comment? I/we have never been very keen on what GISS does or produces, but their results are still in reasonable accord with CRU and NOAA. Is this just luck? Why, I wonder do they go off half-cocked like this? What do/will CRU/NOAA get for Oct. 2008? Tom. Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ----------------------------------------------------------------------------