date: Mon Apr 25 11:19:18 2005 from: Phil Jones subject: Re: Report to: "Brohan, Philip" , Ian Harris Philip, Harry and I are meeting at 2pm today and you'll get the answers and/or additional material later today or tomorrow. As I'm away tomorrow, I'll answer 3 now and give my thoughts on 6. 3. Each month the HC updates the 1991-2005 file with the latest CLIMAT messages. I also do this and pick up earlier CLIMAT messages (for the last 18 months) or so in case any late data get added in. I do this monthly and add in any months of MCDW (Monthly Climatic Data for the World - a file I get from NCDC) that have been received in the past month. MCDW is currently about 2-3 months behind real time. About 100 stations get added through this - most are additional stations in the US that don't appear in the GTS CLIMAT messages. I also from MCDW add in their back data. This is approximately an additional 50-100 station months worth of data - for a random selection of countries. This often (but not always) includes Pakistan, a few sites in India, Sudan, Brazil, a few sites in Thailand plus others. These listed have been regulars in the last 12 months. How NCDC gets this data isn't clear to me. It is likely to be back data received after the cut-off each month. It could be mailed in reports. If it is coming by the GTS your system could be losing it. In assessments done by WMO every year, there is a +/- 2-5% difference in CLIMAT reports received at the main monitoring centres around the world. Each month I also manually add in monthly temperatures for some sites in the Antarctic. These are for the South Pole and Russian sites on the continent (3 sites, Bellingshausen, Vostok and Mirny). South Pole never comes in over the GTS and the Russian sites are very sporadic. I get these by email from the SP and by email from a colleague in St. Petersburg (Victor Lagun). Victor is on the attached paper which came out in IJC recently. All the Antarctic data in this paper are in the dataset. Every now and then I go to the BAS web site of Gareth Marshall and pick up and manually add in back Antarctic data (this is mainly for the French site at Dumont D'Urville) and the NZ site (Scott Base) as reporting from other countries has been relatively good recently. I don't keep any sort of record of any of this, but all the manually entered data is checked by me for outliers. All the MCDW data and back CLIMATs entered automatically get their anomalies listed out and I look for largish values. I am happy to continue doing all this after HadCRUT3 starts. The Antarctic stuff is important as coverage is poor down there. NCDC have produced maps of recent trends for the AR4 from their dataset and their Antarctic coverage is much poorer than HadCRUT2. We are currently trying to figure out if this is because of missing data or the their plotting software. It may be that the single 5 by 5 boxes get omitted when plotting. 6) What Harry has plotted is the annual average corrections in the histograms. The CRU averages appear small, but the stations are all over the world, so averaging makes them appear near zero - adjustments don't have a preferential sign (which might be a point worth making, if you gridded with and without the adjustments, differences would be very small). For Canada, although a big country, it is still only a small part of the world. The adjustments they have applied tend to make sites warmer or colder by about 0.7 deg C. There is less cancelling going on and the adjustments fall into two camps. Maybe if we split the CRU adjustments into continents (even just NH and SH) we might get the bimodal shapes that Canada shows. The different seasonal cycles around the world partly produce the CRU result as for winter months adjustments tend to be larger. I'll see if Harry can try this for USA+Central America/Caribbean, S. America, Europe, Asia Australasia and Africa (+NH and SH). Some of these may not have many samples though. Cheers Phil At 10:07 25/04/2005, Brohan, Philip wrote: Hi Harry. Thanks for the report, it is basically fine - clear and concise. It describes well all the good work you've done on the data. There are a few points that still need clearing up, mostly details needed for the HadCRUT3 paper I am writing: 1) I am using revision 10 of the data files (data, normals and SDs). Is this the final version? If not, please send me the latest revision. 2) Have you any references for the data changes you have made? We'll need these for the paper. 3) Have any other station data changes or additions been made since the production of HadCRUT2? (Antarctic data?) This isn't really part of your report, but we'll need to put details in the paper. 4) Please send me a list of the station IDs you deleted. I can then delete them from the recent data file as well. 5) You've done a good job cleaning up the normals, but I still need estimates of the expected uncertainty of the station normals where they are inferred from limited data or taken from WMO. You will know this from the work you describe in section 3 on process improvement. Can you add a couple of figures like those in section 4, showing the difference between the WMO and new normals for your 617 stations, and the effect of using only 15 of the 30 years in generating a climatology? 6 I'm a bit confused by section 4: Additional information. If I interpret it correctly, for both the CRU and Canadian data, a bit under half of the measurements have been corrected. But the CRU corrections are most likely to be small (~.2C), while the Canadian corrections are never small, and are most likely to be medium sized (~.6C). Is this right? Can you shed any light on this discrepancy? (At first glance it looks as if the uncorrected Canadian data has large inhomogeneities which are approximately normally distributed, but only those larger than a threshold have been identified and corrected; while the CRU corrections are an attempt to correct for the remaining small inhomogeneities. But this is pure speculation on my part.) Thanks, Philip On Fri, 2005-04-22 at 18:56, Ian Harris wrote: > Philip, > > Please find attached a final draft of the report, "ENHANCEMENT AND > QUALITY CONTROL OF CRU MONTHLY TEMPERATURE STATION DATA SET'. > > All comments, etc will be dealt with as quickly as possible. > > Cheers > > Harry > Ian "Harry" Harris > Climatic Research Unit > University of East Anglia > Norwich NR4 7TJ > United Kingdom -- Philip Brohan, Climate Scientist Met Office Hadley Centre for Climate Prediction and Research Tel: +44 (0)1392 884574 Fax: +44 (0)1392 885681 Global climate data sets are available from [1]http://www.hadobs.org Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ----------------------------------------------------------------------------