date: Fri Mar 18 11:02:33 2005 from: Phil Jones subject: RE: HadCRUT2 problems - diagnosis and possible solution to: "Folland, Chris" Chris, [[[redacted: personal]]] Peter is and will be a very good scientist - he just needs to be able to write clear English in emails, and also to know a little of the history of Had/CRU work. David's email helped there. You'll both be a big loss to the HC when you go, but glad to hear it won't be until end of next year. This date will be fine for IPCC. As for IPCC, I'm sure we can get later submitted work in quite easily. I am towing the WGI line though for the time being, which is submit by May05. This is a ridiculous date, but I suspect Susan has it to allow a slippage to the end of 05. There will still be many relevant papers appearing in 2006, that we won't know about, that we should be referring to. Govt. Reviews should enable us to get these in. Hopefully DEFRA will have some HC people in on these. Did you get involved in these with the TAR - or did you stay totally separate? What is your home #? Is 9am OK on Monday? Cheers Phil At 10:23 18/03/2005, you wrote: Phil Thanks. I have not seen any HadCRUT3 stuff yet. [[[redacted: personal]]] It would be a good idea to have a chat about IPCC. I cannot believe IPCC will allow itself to be 18 months out of date when it appears. If so, long live the TAR! Surely the Government Reviewers like DEFRA wont allow that! Can you ring me Monday at home any reasonable time. Working at home then. Am would be best. Heard no more about Ilarionov. We are ready if he pounces! No feedback from the "Sceptic". But at least I did not get misquoted by the Wall St Journal - I was not even quoted despite my hour of discussion. A good sign probably. So far, touch wood, I have stayed on the "right" side of the sceptics over the last decade. Certainly spoken to enough of them, often at length, and found out a bit about what makes them tick (different in every case). There may come a moment when this is useful. A long standing problem worth mentioning now is my potential successor. A month ago it was formally agreed that I can stay on till 31 Dec 2006, so the problem recedes for now. We might want to chat about that. DEP goes about the same time on present projections. Chris Prof. Chris Folland Met Office Fellow and Head of Climate Variability Research Global climate data sets are available from [1]http://www.hadobs.org Met Office, Hadley Centre, Fitzroy Rd, Exeter, Devon, EX1 3PB United Kingdom Email: chris.folland@metoffice.gov.uk Tel: +44 (0)1392 886646 Fax: (in UK) 0870 900 5050 (International) +44 (0)113 336 1072) [2]http://www.metoffice.gov.uk Hon. Professor of School of Environmental Sciences, University of East Anglia, Norwich -----Original Message----- From: Phil Jones [[3]mailto:p.jones@uea.ac.uk] Sent: 18 March 2005 08:58 To: Folland, Chris Subject: RE: HadCRUT2 problems - diagnosis and possible solution Chris, I'm going to send a brief email to Philip to see where we are with HadCRUT3/v. There needs to be a submission fairly soon wrt AR4. The Beijing meeting is May 10-13 with rewriting the 2-3 months after, before we get a FOD for general release to any who want to comment. As an aside, I'll be interested to see how WGI plays this wrt the FOD. I can see bits of text appearing on web sites if Susan isn't very careful. I'm sure there will be safeguards to stop this. As for HadCRUT2/v, I was only suggesting what seemed most practical given the imminent arrival of HadCRUT3/v. Sticking with the current coverage percentages we are using seems the best thing to do with 2v. It would have been better for Peter to have just compared where the differences were rather than looking at the whole world. Tasmania is either in the right or wrong position, after all. Even if it is wrong it won't make much difference. Also trying to stop key files (those that will be used in HadCRUT3/v) being altered just seemed common sense to me. I've only seen so far one brief outline of the proposed HadCRUT3/v paper. It would be good to see more of the whole thing rather than bits and bobs and potential diagrams that Simon keeps sending around. Although the contract we've done for you has ended, the person here is still doing some work. We've not sent everything back yet. We've still not sent the new SD file, which is necessary to weed out some typos that creep in from time to time. We have removed some duplicate stations that came in last time, and checked/altered/deleted some of the 1961-90 normals files. Philip should have these. Let me know of any relevant decisions made at the meeting. The next time I'll be down in Exeter will be for the HC Review meeting (May 17-19), the week after Beijing. Any further news on Iliaronov? Also was the person happy with you response to the question to John Houghton about paleo in the TAR? This seemed fine with me. Also, I'd be happy to ignore Iliaranov's piece. Another aside, NCDC have decided in their new gridding (which might not be in time for AR4) to forget their first difference method. They say it is more trouble than it is worth. They are also going to try to use SYNOP-calculated CLIMATs, which seems more trouble than it's worth. Cheers Phil At 22:21 17/03/2005, you wrote: >Phil > >Except for the one hopefully calming email I sent, I have kept out of >this so far. I have now asked for a briefing next week (likely >Tuesday). If anything silly looks like happening please do not hesitate >to contact me! > >Chris > >Prof. Chris Folland > >Met Office Fellow and Head of Climate Variability Research > >Global climate data sets are available from [4]http://www.hadobs.org > >Met Office, Hadley Centre, Fitzroy Rd, Exeter, Devon, EX1 3PB United >Kingdom >Email: chris.folland@metoffice.gov.uk >Tel: +44 (0)1392 886646 >Fax: (in UK) 0870 900 5050 > (International) +44 (0)113 336 1072) >[5]http://www.metoffice.gov.uk > >Hon. Professor of School of Environmental Sciences, University of East >Anglia, Norwich > > > >-----Original Message----- >From: Phil Jones [[6]mailto:p.jones@uea.ac.uk] >Sent: 17 March 2005 16:59 >To: Thorne, Peter >Cc: Thorne, Peter; i.harris@uea.ac.uk; Chris Folland; Tett, Simon; >Parker, David; John Kennedy; Brohan, Philip; Jennifer Hardwick; Jen >Teignmouth >Subject: Re: HadCRUT2 problems - diagnosis and possible solution > > > > Dear All, > Maybe I've misinterpreted Peter's email, but UEA doesn't want >the whole of > HadCRUT2v/HandCRUT2 back. If the coverage percentage goes back to >what it > was then none of HadCRUT2/v will change except for Jan 2005. Unless >I've misunderstood, > we've been using the same coverage since 2v started, so there is no >need to > send the whole thing back. Also, if the old version goes back, then >there >shouldn't be > any error messages !!! Why will there be any if we are doing what we >were. THERE IS NO > NEED TO CHANGE ANYTHING IF YOU GO BACK TO THE ORIGINAL COVERAGE > FILE. There won't be any grid boxes that are different. I don't want >any >changes. I keep > saying this and it keeps falling on deaf ears. > > As an aside, the coverage percentage file isn't mine. When I >started doing HadCRUT2v, > I got this from David Parker, so it's yours - not mine ! > > Finally, although there are differences they only have an impact > if > >they are between > 25 and 75% land/ocean coverage. > > The USGS version isn't what you want anyway. Part of the >differences come about from > setting ocean squares to land where there is permanent sea ice. This >was >a simpler > way of allowing some extrapolation over sea ice from land stations >without bringing in > any SST and also stopping SST getting into squares where it shouldn't. > > Merging on errors isn't right either. It still makes no sense. >Merging by area, maybe > incorporating errors might be possible. Merging should also consider >the >number of > samples not just the error. If a box has 75% land with two stations >but >large error, but the > 25% ocean has small error, the merged value should also reflect the >greater % land. I > still need to be convinced of this, and the only way I'll be convinced >is >to see the > old and new methods (and little difference between them). > > Cheers > Phil > > > Cheers > Phil > > >At 15:38 17/03/2005, Thorne, Peter wrote: > >All, > > > >attached is an excel spreadsheet (sorry linux people!) showing the > >percentage coverage from IGBP > >([7]http://lpdaac.usgs.gov/glcc/globdoc2_0.asp) resolved to 5 by 5 and > >the > > >version that Phil has sent of the coverage percentage. Top is Phil's > >data which is to nearest whole percent (x100), next is IGBP as a > >fractional coverage and then last is the difference field as a > >fractional measure (Phil minus IGBP). There are some fairly large > >differences throughout. In the poles these are because of different > >classification procedures I should think. Differences beyong the 3rd > >d.p are probably rounding (obviously). The others look kinda scary so > >I'm advocating adopting an Ostrich position ... > > > >As Simon intimated earlier in this thread in HadCRUT3 the merge is > >going to be based upon estimated error rather than fractional > >coverage. > > > >Path of least resistance is install Phil's dataset back in which > >we've been using for a long time and run and just ignore any error > >messages, send UEA everything (complete rather than solely post-1990 > >merged hadcrut2 and hadcrut2v files) this month and then keep running > >it with a mask we know to be sub-optimal because the whole thing is > >going to be > > >replaced on a timescale of months. We can let you know which > >gridboxes have changed in the merged product for the website Phil. > > > >I haven't got time to chase this further right now so if you do want > >it > > >chased you'll have to scrape together the time from somewhere else > >I'm afraid. > > > > > >All those not in favour shout now or forever hold your peace. > > > >Peter > > > >-- > >Peter Thorne Climate Research Scientist > >Hadley Centre for climate prediction and research > >Met Office, FitzRoy Road, Exeter, EX1 3PB > >Tel:+44 1392 886552 Fax:+44 1392 885681 [8]http://www.hadobs.org > >Prof. Phil Jones >Climatic Research Unit Telephone +44 (0) 1603 592090 >School of Environmental Sciences Fax +44 (0) 1603 507784 >University of East Anglia >Norwich Email p.jones@uea.ac.uk >NR4 7TJ >UK >----------------------------------------------------------------------- >- >---- Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ------------------------------------------------------------------------ ---- Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ----------------------------------------------------------------------------