cc: p.jones@uea,t.osborn@uea date: Thu Sep 3 12:33:47 1998 from: Keith Briffa subject: Re: NERC proposal review to: "Dr. Myles Allen" Myles thanks indeed for the trouble you have taken and particularly for your willingness to discuss the issues you raise openly. On a quick first reading I have to say that I think I agree with almost everything you say - certainly I agree that the complex methods of analyses we mention do not necessarily hold out greater promise than simple regression. However, you are absolutely correct when you assume that this is just the message I intended to relay. The truth of this must be explicitly demonstrated and considerable efforts to compare methods with the same input data will no doubt be required. I , and I know Tim and Phil, will be really delighted to collaborate with you on this if we are successful . I believe Mike Mann will do likewise and I believe that together we would likely make a useful contribution to the science of this area. At this moment , I am (almost literally) up to my neck in bits of scrawl trying to put together a draft NERC Thematic Proposal and I must defer discussion 'til next week. In the meantime I will pass this message and your comments on to Tim and Phil and , with your permission (?) , later forward them to Mike Mann. Again thanks for the review and let's hope we are funded. See you soon. best wishes Keith At 11:53 AM 9/3/98 +0100, you wrote: >Dear Keith, Phil and Tim, > >I just completed a (rather late) review of your proposal, "Improving our >understanding of natural climate variability...", enclosed. Since NERC have a >habit of selectively editing these reviews before passing them back to the >proposers, I wanted you to see the whole thing. As you can see, I'm strongly >recommending funding, although I did have some reservations about your proposed >methodology. I think these should be straightforward for you to reply to -- in >any case, the committee might just see this as a difference of opinion between >us. If you are interested in pursuing the explicit modelling approach and would >like to discuss it further before replying, we could perhaps talk it over at the >conference next week. > >Regards, > >Myles >-------- >Review of GR3/12107, Briffa et al, "Improving our understanding of >natural climate variability..." > >This is an excellent project which I rate overall as highly recommend >for funding. The resources requested seem very reasonable for a >significant extension of what is already an internationally successful >research programme addressing a topic of central significance to the >mission of NERC. Drs. Osborne, Jones and Briffa are all leading >experts in the field of centennial-timescale climate reconstruction, >eminently well-qualified for this project. This recommendation is >based on general expertise in the analysis of climate variability and >its role in the the detection of anthropogenic climate change, some >work on secondary analysis of paleoclimatic indicators but no direct >experience in the acquisition or primary analysis of paleodata. I >have a number of methodological queries, detailed below, which it >might be helpful for the authors to clarify, although I stress these >are relatively minor details which should not be taken as >justification for denying funding. > >The authors' key advantage over many others working in this field is >direct familiarity with the primary acquisition of paleodata and, with >it, an appreciation of the central problem of potential >non-stationarity in the functional relationship between >paleo-indicators and climate variables (see, for example, Briffa et >al, 1998a). Thus, despite the fact that the project is extremely >ambitious, I am confident that they will not, as others may have done, >fall into the trap of overinterpreting the paleorecord. A key output >of this project should be an assessment of the implications of such >non-stationarity on the used of the 20th century instrumental record >as a calibration period for paleodatasets. > >Part of the problem in much recent work in this area has been an >over-reliance on apparently sophisticated analysis techniques which >are not obviously motivated by the problem and which make it extremely >difficult to evaluate the significance (physical or statistical) of >any relationships observed. Spectral techniques (whether or not they >are evolutive) are particularly dangerous, since they rely explicitly >on an assumption of stationarity over a certain timescale the >implications of which are often obscure. Lagged-window eigenvector >techniques like EEOFs/MSSA looked like a solution when first >introduced, but turned out to be subject to many of the same sorts of >problems. Thus I don't fully understand why the authors feel that >EEOFs/MSSA, evolutive spectral analysis (methodology part i) and >frequency-domain transfer function estimation (part ii) are the >techniques of first resort. If the aim is to critically evaluate some >of the more extravagant claims which have recently been made on the >basis of these techniques, then I fully support it. For the purposes >of actually advancing our understanding of the relationship between >past climate and paleoindicators, I suggest that a simpler and more >direct approach is likely to be much more informative. > >The hypothesised relationships between hemispheric-scale and grid-box >climate variables and between grid-box variables and local >paleoindicators must be written down explicitly (this doesn't need to >mean an explicit model of tree growth -- very likely nothing more >complicated would be needed in either case than a linear regression >model with a red noise residual term) and a standard uncertainty >analysis provided on all unknown parameters. If no such relationships >can be written down, or no consistent model found, then we must >question the point of the whole exercise, but I am prepared to believe >this is not the case. If they can be written down, then techniques >like EEOFs/MSSA should only be brought in if there is a clear argument >that they are likely to provide a better estimate of model parameters >than simpler techniques like regression. I have not seen this >argument made, although this may simply reflect my relative >unfamiliarity with the primary paleoclimate literature. > >Having worked quite extensively on techniques like EEOFs/MSSA and >frequency-domain EOF analysis (the precursor of the Park-Mann >approach), I think I can state with some confidence that interdecadal >and centennial timescale modes of climate variability will be found, >which will appear coherent with similar modes extracted from the >paleorecord, and which will possibly even bear a striking resemblance >to modes extracted from the GCM control integrations, quite >irrespective of the true relationship, if any, between these various >datasets. I think, therefore, it would be reasonable for the >committee to request reassurance that parts (i) and (ii) of the >scientific methodology will not simply consist in applying these >analysis techniques to these datasets and inspecting the results, but >will include at least an element of explicit modelling along the lines >described above. This may be precisely what the authors mean by >"exploring the spatial congruence and sensitivity in spectral >character of the major modes" and comparing "the major spatiotemporal >components of this paleomatrix ... with those of the instrumental >matrix and model datasets", but I am afraid these sentences are too >compressed for me to understand exactly what they mean. If the >authors wish to correspond over this point in replying to this review, >I am quite happy for you to disclose my identity. I would also, if >the authors are interested and feel it would be helpful, be very >interested in advising them on their analysis approach as (as I assume >it will) the project proceeds. > >Please accept my apologies for the late return of this review. > >Myles Allen >