date: Mon Aug 24 09:34:17 2009 from: Phil Jones subject: Re: Data Loss- a couple of questions to: Daniel Youmans Dan, [1]http://blogs.nature.com/news/thegreatbeyond/2009/08/climate_researcher_vs_foi_part.html This alludes to another story (below) where the issues are similar. This is from a skeptic email listing. It obviously depends on what people think of as Environmental Information. It is useful to look at the EIR and see what it refers too. It doesn't refer to climate data. In this case with Queens, the tree-ring data come from ages ago, so to my mind it isn't current EIR. One issue is that FOI and EIR probably shouldn't apply to some things Universities do. Researchers should be free to determine who they want to work with. In the tree case there are numerous people in central Europe (alpine countries and Germany especially) who have chronologies for dating buildings and paintings. Many make reasonable livings out of doing this work. If their data were made available, they would be out of a job. Keenan is a nasty person. I did comply with one of his FOI requests [2]http://www.informath.org/apprise/a5620.htm Keenan documents things here, but misses out several important facts. SUNYA (State University of NY at Albany) investigated the claims, and found them to be groundless. Keenan agreed not to do anything about his claim until SUNYA investigated things, but he then published them in a journal (Energy and Environment) whilst their investigation was ongoing. There is no way of dealing with these people. A number of US Universities have been beset with claims over the last few years. The datasets CRU make available on its web site ( eg [3]http://www.cru.uea.ac.uk/cru/data/) are used by countless climatologists around the world. I'd just look at the number of citations for this paper. Mitchell, T.D. and Jones, P.D., 2005: An improved method of constructing a database of monthly climate observations and associated high-resolution grids. Int. J. Climatol. 25, 693-712. Also have a look at my citation record on World of Science. Because of my common surname you have to remove one from biology, but it is still very high. This is a pain to do, so have a look at this site. [4]http://www.eecg.utoronto.ca/~prall/climate/ Person obviously motivated by green politics, but it shows how high many CRU people are in the list. My position varies on his list, but he's probably worth contacting. With the way the UK govt assesses research for RAE (and the new REF) citation counts are one metric. Proposals and PhD students are other metrics. Names of people who were in CRU (Tom Wigley, previous Director, Mark New, Jean Palutikof Robert Marsh, Sarah Raper and Dick Warrick). People still in CRU are me, Keith Briffa, Tim Osborn who are in the list. Mike Hulme was in CRU, but is now in ENV. This web site has been updated with citation counts for many of the skeptics. It is sometimes difficult to classify people as easily as this though. Finally [5]http://www.uea.ac.uk/env/ has a quote from Sir David King, the former govt Chief Scientist. [6]http://www.uea.ac.uk/env/annrep2008 [7]http://www.uea.ac.uk/polopoly_fs/1.94246!env_ann_rep_2007.pdf Have a look at the 2007 Report and the number of ENV people involved in the IPCC Reports. ENV (including CRU) involvement is more than other University Dept anywhere in the world. Only the Met Office in the UK and NOAA (American Met Service) have more. Away Aug 27-Sept 5. Cheers Phil Peiser, Benny wrote: CCNet Xtra, 15 August 2009 2009 -- Audiatur et altera pars ANOTHER UK CLIMATE DATA WITHHOLDING SCANDAL IS EMERGING -------------------------------------------------- They are my data. --Peter M. Brown, President of the Tree-Ring Society, April 2007 In other words, even if the research and the researcher's salary are fully paid for by the public - as is the case at QUB - the researcher still regards the data as his or her personal property. -- Doug Keenan, 14 August 2009 I also offered to visit QUB with the case officer, to demonstrate how quickly the data could be copied (e.g. from floppy disks), and to copy the data myself. The officer, though, declined my offer, again saying that she was satisfied with QUB's explanation. There is a mechanism to appeal an ICO decision, to a tribunal. I told the case officer that I wanted to do so. The officer replied that, in order to file an appeal, I would need a formal Decision Notice from the ICO. I requested a Decision Notice. The officer then informed me that the ICO would send a Notice, but that, because they were busy, it would take about two years to do so. -- Doug Keenan, 14 August 2009 The main problem with dendrochronolgy is one of small sample size. As a forester, who specialises in forest measurement, I can state that within any stand of trees, managed or not, there is always variation within the stand with competition between trees being the main player. Sure annual weather events will influence the stand as a whole, but often this will be masked by the within stand competition. If the dendro gets the sample wrong then any inferences that they might make about the past weather are also incorrect. I shake my head every time someone quotes from another dendro study, when will they ever learn? --Pnadanus, WUWT, 14 August 2009 I speak as a forester who has examined tree rings on zillions of stumps and cores. Weather events and conditions are not at all obvious in tree rings. Heck, even fire scars are difficult to discern and fires often leave no mark at all on individual trees. An unusually mild winter, dry spring, or a hot summer are all virtually impossible to detect, even if thousands of contemporaneous ring series are measured to a gnats eyebrow and compared. --Mike D., WUWT, 14 August 2009 -------------------- ANOTHER UK CLIMATE DATA WITHHOLDING SCANDAL IS EMERGING Watts Up With That, 14 August 2009 <[8]http://wattsupwiththat.com/2009/08/14/another-uk-climate-data-scandal-i> s-emerging/#more-9913> As many WUWT readers know, Steve McIntyre's tireless quest to get the raw data that makes up the gridded Hadley Climate Research Unit HadCRUT dataset has been fraught with delays, FOI denials, and obvious obfuscation. In some cases the "dog ate my homework" is the excuse. The UK Register has an excellent summary of the issue. A similar issue has been brewing in parallel over tree ring data in the UK. Doug Keenan tells us the story of getting the "ring around" for over 2 years trying to obtain what many would consider a simple and non controversial data request. - Anthony Guest Post by Doug Keenan Queen's University Belfast is a public body in the United Kingdom. As such, it is required to make certain information available under the UK Freedom of Information Act. The university holds some information about tree rings (which is important in climate studies and in archaeology). Following discusses my attempt to obtain that information, using the Act. Scientists study tree rings for two main purposes. One purpose is to learn something about what the climate was like many years ago. For instance, if many trees in a region had thick rings in some particular years, then climatic conditions in those years were presumably good (e.g. warm and with lots of rain); tree rings have been used in this way to learn about the climate centuries ago. The other purpose in studying tree rings is to date artefacts found in archaeological contexts; for an example, see here. Tree-ring data from Northern Ireland One of the world's leading centers for tree-ring work is at Queen's University Belfast (QUB), in Northern Ireland. The tree-ring data that QUB has gathered is valuable for studying the global climate during the past 7000 years: for a brief explanation of this, see here. Most of the tree-ring data held by QUB was gathered decades ago; yet it has never been published. There is a standard place on the internet to publish such data: the International Tree-Ring Data Bank (ITRDB), which currently holds tree-ring data from over 1500 sites around the world. QUB refuses to publish or otherwise release most of its data, though. So I have tried to obtain the data by applying under the UK Freedom of Information Act (FoI Act). I have submitted three separate requests for the data. Each request described the data in a different way, in an attempt to avoid nit-picking objections. All three requests were for the data in electronic form, e.g. placed on the internet or sent as an e-mail attachment. The first request was submitted in April 2007. QUB refused the first request in May 2007. I appealed the refusal to a Pro-Vice-Chancellor of QUB, who rejected the appeal. The primary reason that the Pro-Vice-Chancellor gave for rejection was that some of the data was in paper form and had not been converted to electronic form. The Pro-Vice-Chancellor additionally claimed that after data was converted to electronic form, "It is then uploaded to the International Tree Ring Data Base". There might indeed be some small portion of the data that is not in electronic form. My request, though, was for a copy of the data that is in electronic form. So, is all data that is in electronic form available at the ITRDB, as the Pro-Vice-Chancellor claimed? QUB has in the past published the results of various analyses of its tree-ring data (most notably its claim to have sequences of overlapping tree rings extending back in time many millennia). In doing the analyses, the sequences of tree-ring data are analyzed statistically, and the statistical computations are done by computer. This is well known, and moreover has been stated by QUB's former head tree-ring researcher, Michael G.L. Baillie, in several his publications. (Indeed, Baillie and his colleague Jon R. Pilcher, also at QUB, wrote a widely-used computer program for tree-ring matching, CROS.) Obviously the data that was used for those computations is in electronic form-and it has not been uploaded to the ITRDB. Thus the claim by the Pro-Vice-Chancellor is untrue. The Pro-Vice-Chancellor further claimed that to organize the data in "the very precise categories which [I] have specified" [in my request] would entail a vast amount of work. My request, though, was merely for the tree-ring data that had been obtained and used by the university; that hardly seems like precise categorization. Moreover, I later submitted a second request for "the data about tree rings that has been obtained by [QUB] and that is held in electronic form by the university". That request was also refused. And a third request that was very similar to the second was refused. All three requests were refused in whole, even though the university is required to make partial fulfillment when that is practicable. The UK Information Commissioner's Office After half a year of trying to obtain the information from QUB, I appealed to the Information Commissioner's Office (ICO). The ICO is charged with ensuring that the FoI Act is enforced. My appeal to the ICO was submitted on 24 October 2007. The ICO notified me that an officer had been assigned to begin investigating my case on 14 October 2008. Such a long delay is clearly incompatible with effective working of the Act. The ICO then contacted QUB, asking for further information. QUB then admitted that almost all the data was stored in electronic form. Thus QUB implicitly admitted that its prior claims were untruthful. QUB now asserted, however, that the data was on 150 separate disks and that it would take 100 hours to copy those disks. (These were floppy disks-the type that slide into computers and, prior to the internet, were commonly used to carry electronic data.) It takes only a minute or two to copy a floppy disk, however; so the claim of 100 hours to copy 150 floppy disks is an unrealistic exaggeration. QUB also said that it considered photocopying a printed version of the data, but that this would take over 1800 hours. As noted above, all my requests were for data that is in electronic form; moreover, I have repeated this point in subsequent correspondences with QUB. The statement from QUB about photocopying is thus not relevant. On 22 December 2008, the ICO sent me a letter rejecting my appeal, on the grounds that the time needed by QUB would exceed an "appropriate limit" (as stipulated in the FoI Act). The ICO had accepted QUB's explanation for refusing to release the data without question, and without discussing the explanation with me. I telephoned the ICO to raise some objections. To each objection that I raised, the ICO case officer gave the same reply: "I'm satisfied with their [QUB's] explanation". I also offered to visit QUB with the case officer, to demonstrate how quickly the data could be copied (e.g. from floppy disks), and to copy the data myself. This seemed particularly appropriate because the officer had told me when she started on the case that she would visit QUB as a standard part of investigation, yet she had not made such a visit. The officer, though, declined my offer, again saying that she was satisfied with QUB's explanation. There is a mechanism to appeal an ICO decision, to a tribunal. I told the case officer that I wanted to do so. The officer replied that, in order to file an appeal, I would need a formal Decision Notice from the ICO. I requested a Decision Notice. The officer then informed me that the ICO would send a Notice, but that, because they were busy, it would take about two years to do so. Environmental Information Regulations I discussed the above with a colleague, David Holland. Holland said that my request should not have been processed under the FoI Act. His reasoning was that the information I was requesting was about the environment: environmental information is exempt from the FoI Act and requests for such information should instead be processed under the Environmental Information Regulations (EIR). He pointed out that the tree-ring data clearly fits the definition of "environmental information" given in the EIR. It also clearly fits the common (dictionary) definition. I had been aware that the EIR existed, but had assumed that the EIR was essentially the same as the FoI Act. After the discussion with Holland, though, I checked and found that there is one major difference between the EIR and the FoI Act: under the EIR, there is no limit on the amount of time that a public institution requires to process a request. In other words, even if QUB's original claim that some of the data was only available on paper were true, or even if QUB's revised claim that copying data from disks would take 100 hours were true, that would still not be a valid reason for refusing to supply the information. I am not an expert in how to apply the EIR or FoI Act, though. So I telephoned the ICO headquarters to ask for guidance. There I spoke with a Customer Service Advisor, Mike Chamberlain. Chamberlain told me the following: that the information seemed obviously environmental; that there was no limit on processing time that could be used to refuse a request for environmental information; that I could freely visit a site where environmental information was held in order to examine the information; and that it was the duty of the public authority (i.e. QUB) to determine whether the EIR or the FoI Act was applicable. Chamberlain also confirmed everything that he told me with someone more senior at the ICO. It is regrettable that I had not realized the above earlier. My initial request to QUB, in April 2007, had stated the following. It might be that this request is exempt from the FOIAct, because the data being requested is environmental information. If you believe that to be so, process my request under the Environmental Information Regulations. QUB, however, had not processed my application correctly. I should have caught that. There is another issue. I had described the information to the ICO case officer by telephone and also by e-mail (on 24 November 2008). Hence the case officer must have known that the information was environmental, and thus exempt under the FoI Act and only requestable under the EIR. Why did the ICO not act on that? On 29 January 2009, I e-mailed the case officer, citing the above-quoted statement from my request to QUB and saying "I would like to know the reasoning that led to my request being processed under the Freedom of Information Act, instead of EIR". Initially, there was no reply. The EIR was enacted pursuant to the Aarhus Convention, an international treaty on environmental information that the UK promoted, signed, and ratified. Failure to implement the EIR would constitute a failure by the UK to adhere to the Convention. So, a few weeks after e-mailing my question to the ICO, and with no reply, I contacted the Aarhus Convention Secretariat (ACS), at the United Nations in Geneva. The ACS has a mechanism whereby individuals can file a complaint against a country for breaching the Convention. I had an initial discussion with the ACS about this. That turned out to be unnecessary though. The Assistant Information Commissioner for Northern Ireland contacted me, on 10 March 2009: he was now handling my case and, moreover, he had visited QUB and seen some of the data. On 22 April 2009, I received a telephone call from the Assistant Information Commissioner for Northern Ireland. The Assistant Commissioner said that he was preparing a Decision Notice for the case, and he made it clear that the Notice would hold that the data should be released under the EIR. The next I heard anything was on 13 July 2009, when it was announced that the Assistant Commissioner had been suspended. On 13 August 2009, I telephoned the ICO: I was told that a new officer would be assigned to the case within the next few days and that a draft Notice, which had been written by the Assistant Commissioner, was in the signatory process. I am presently awaiting further word. Another example-Gothenburg University I have previously been involved with obtaining tree-ring data from another institution: Gothenburg University, in Sweden. Sweden has a law that is similar to the UK's Freedom of Information Act (the Swedish law is the Principle of Public Access). In 2004, Swedish courts ruled that the law applied to research data held by universities. In a famous case known as the "Gillberg affair", a researcher at Gothenburg University refused to obey the law. As a result, both the researcher and the rector of the university were convicted of criminal malfeasance. (The researcher received a suspended sentence and a fine; the rector received a fine.) Gothenburg University does substantial tree-ring research. On 10 April 2007, I requested their tree-ring data. The university's lead tree-ring researcher repeatedly resisted, claiming that it would take weeks of his time, and that he was too busy to do it. On 22 April 2008, I sent a letter to the (new) rector of Gothenburg University, saying that if the data was not supplied, I would file complaints with both the Court and the Parliamentary Ombudsmen of Sweden. The next day, all the data was submitted to the ITRDB. What transpired with Gothenburg University exemplifies the importance of laws on Freedom of Information for tree-ring data. Motivations for withholding data Some people have asked why QUB does not want to release the data. In fact, most tree-ring laboratories do not make their data available: it is not just QUB and Gothenburg that have been reluctant. The reason for this was elucidated by Peter M. Brown, in April 2007. At the time, Brown was president of the Tree-Ring Society, which is the main international organization for tree-ring researchers. Following is an excerpt (the full explanation is here). ... they ARE my data. Funding agencies pay me for my expertise, my imagination, and my insights to be able to make some advance in our understanding of how nature works, not for raw data sets. ... It is the understanding and inferences supplied by the scientist that funding agencies are interested in, not her or his raw data. In other words, even if the research and the researcher's salary are fully paid for by the public-as is the case at QUB-the researcher still regards the data as his or her personal property. There are only a few tree-ring laboratories where attitudes are different. One example is the University of St Andrews, in the UK. Almost all tree-ring data held by St Andrews is freely available in the ITRDB. It is notable that QUB continues to withhold its data even though, in 2009, the tree-ring laboratory at QUB was effectively closed. The closure was primarily due to the lab lacking funds, which presumably resulted from having almost no research publications (i.e. the lab had not been producing anything; so funding agencies declined to support it). The dearth of publications occurred even though the lab has some extremely valuable data on what is arguably the world's most important scientific topic-global warming (as outlined here). This problem arises because the QUB researchers do not have expertise to analyze the data themselves and they do not want to share their data with other researchers who do. ------------------------------------------------------------------------ -------- Correspondences Date Sender Summary (with link) 2007-04-10 DJK My first request for the information held by QUB (sent by e-mail). 2007-04-16 QUB Acknowledgement of request. 2007-05-11 QUB E-mail saying that there will be a delay in responding to the request (which is required to be within 20 business days of my request), but that QUB would respond by May 18th. 2007-05-21 DJK E-mail to the ICO, about the lack of response from QUB; Cc'd to QUB. 2007-05-22 QUB First refusal of the request for information. 2007-05-22 QUB E-mail with attached description of how to appeal the refusal (this was sent following a telephone call to QUB in which I noted that they are required to send me such). 2007-05-23 DJK E-mail to the ICO, noting that QUB had responded. 2007-05-24 DJK Appeal of the refusal, submitted to a Pro-Vice-Chancellor of QUB. 2007-05-25 QUB Initial response to the appeal, saying that appeal to a Pro-Vice-Chancellor should be resorted to only if QUB's Information Compliance Officer and I are unable to resolve things ourselves. 2007-06-02 ICO Acknowledgement of my prior e-mails, correctly noting that the ICO should not act at this time. 2007-06-21 QUB Second refusal of the request for information. 2007-07-13 DJK Appeal, submitted to a Pro-Vice-Chancellor of QUB. 2007-07-26 QUB Notification that a response to my appeal to a Pro-Vice-Chancellor will be delayed until the second half of August. 2007-08-15 QUB E-mail from a Pro-Vice-Chancellor, saying that a response to my appeal will be sent by September 30th (i.e. about seven weeks after the four-week limit). 2007-09-28 QUB Rejection of my appeal by a Pro-Vice-Chancellor. 2007-10-24 DJK Appeal to the ICO. 2007-11-27 ICO Acknowledgement of appeal, saying that it might be several months before I hear from the ICO again. 2008-05-15 DJK Second request for the information held by QUB. 2008-05-15 DJK Third request for the information held by QUB. 2008-06-12 QUB Rejection of my second and third requests, by the Head of the Registrar's Office. 2008-06-19 DJK E-mailed reply to the Head of the Registrar's Office at QUB. 2008-07-01 QUB Acknowledgement of my last e-mail, by the Head of the Registrar's Office. 2008-07-15 DJK E-mail to the ICO, notifying them of the rejected second and third requests for the information. 2008-10-14 ICO E-mail notifying me that an officer has been assigned to my case and asking me to confirm that I would like to proceed. 2008-10-15 DJK E-mail to the ICO, confirming that I would like to proceed. 2008-11-24 DJK E-mail to the ICO, briefly synopsizing how tree-ring dating works (this followed a telephone call, in which the case officer had said that might be helpful). 2008-11-24 ICO Acknowledgement of my last e-mail and notice that QUB's response had been slightly delayed. 2008-12-22 ICO Rejection of appeal. 2009-01-21 DJK E-mail summarizing telephone call with the ICO; during the call I was informed that it would take roughly two years for the ICO to issue a Decision Notice on the case. 2009-01-21 ICO Letter acknowledging my request for a Decision Notice (this was sent 43 minutes after my last e-mail, and its content appears to be independent of that). 2009-01-29 DJK E-mail to the ICO, asking why my requests for information had been processed under the FoI Act, instead of the EIR. 2009-03-02 DJK E-mail to the ACS, alleging a breach of the Aarhus Convention by the United Kingdom. 2009-03-10 ACS Reply from the ACS Legal Support Officer, concluding that the evidence substantiating the allegation is sufficient for the ACS to proceed with a review. 2009-03-10 ICO E-mail from the Assistant Information Commissioner for Northern Ireland, saying that he was handling the case and he had visited QUB (this was concomitant with a telephone call, which elaborated). 2009-03-11 DJK E-mail to the ACS, saying that the ICO was now progressing things, and so my allegation was unnecessary. 2009-03-11 ACS Acknowledgement of prior e-mail. 2009-03-25 ICO E-mail from the Assistant Information Commissioner, saying that he was continuing with work on the case and would contact me again after Easter. EDITOR'S NOTE: Background information: Why Irish tree-ring data is important, see: <[9]http://www.informath.org/apprise/a3900/b910.htm> ---------------- CCNet is a science policy network edited by Benny Peiser. To subscribe, send an e-mail to ("subscribe cambridge-conference"). To unsubscribe send an e-mail to ("unsubscribe cambridge-conference"). Information circulated on this network is for scholarly and educational use only. The attached information may not be copied or reproduced for any other purposes without prior permission of the copyright holders. DISCLAIMER: The opinions, beliefs and viewpoints expressed in the articles and texts and in other CCNet contributions do not necessarily reflect the opinions, beliefs and viewpoints of the editor. <[10]http://www.staff.livjm.ac.uk/spsbpeis/> At 22:25 21/08/2009, you wrote: Phil, Thanks very much for that link- it definitely clears up a lot of the questions the article raises. Have papers/websites reported a similar story to this before? Have similar issues plagued other institutions? Lastly, would you be able to give us a short quote on the importance of the work the CRU does, and the benefits UEA gains from being associated therewith? Many thanks, Dan Youmans Dan Youmans News Editor (UEA) The Project e: [11]danielyoumans@theprojectonline.org w: [12]http://theprojectonline.org On 21 Aug 2009, at 10:13, Phil Jones wrote: Dan, There is no story here. We have lost no data. [13]http://www.cru.uea.ac.uk/cru/data/availability/ We will be contacting Met Services around the world to see if we can release their data. There is an awful lot of history to the story that is doing the rounds on skeptic websites. They are just stirring things up - in some attempt to stop politicians doing anything climate change. Cheers Phil At 22:04 19/08/2009, you wrote: Dear Professor Jones, I am contacting you as UEA News Editor of The Project, a new, free, student newspaper set to launch in Norwich next month. I am writing to you about the story, published on The Register's website, about the CRU's loss of some data. Would it be possible for you to provide a reaction to this story for our paper, perhaps mentioning the pre-eminence of the CRU and the School of ENV that are so vital to UEA's reputation, or even better, if we could perhaps speak on the phone? I appreciate any help you can give me, Regards, Dan Youmans News Editor (UEA) The Project e: [14]danielyoumans@theprojectonline.org w: [15]http://theprojectonline.org Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email [16]p.jones@uea.ac.uk NR4 7TJ UK ---------------------------------------------------------------------------- Dan Youmans News Editor (UEA) The Project e: [17]danielyoumans@theprojectonline.org w: [18]http://theprojectonline.org Prof. Phil Jones Climatic Research Unit Telephone +44 (0) 1603 592090 School of Environmental Sciences Fax +44 (0) 1603 507784 University of East Anglia Norwich Email p.jones@uea.ac.uk NR4 7TJ UK ----------------------------------------------------------------------------