Climate Research Unit email scandal

Hackers got access to a trove of private emails from the University of East Anglia's Climate Research Unit that is being trumpeted by those who disbelieve in anthropogenic global warming as proof of scandal. I've looked through the data a bit myself--you can find a searchable archive of the emails here. I suspect this collection of emails may end up being put to good research use as the Enron email corpus was. While I found a few embarrassing things, I found no evidence of outright data fabrication or fakery.

The main email that has been cited as such evidence is an email from Phil Jones that says:

I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.

Gavin Schmidt at RealClimate explains:

The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem”–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.

In other words, "hiding" in this case is using temperature measurement records instead of tree rings as a proxy for temperature records for a period of time where the tree rings are known not to be an accurate proxy, for whatever reason.

It's also claimed that these emails show a concerted effort to subvert the peer review process and stop publications by climate change skeptics, but most of those emails seem to center around an issue where the scandal was actually from the skeptics--the publication of a 2003 paper by Willie Soon and Sallie Baliunas in the journal Climate Research that was considered by 13 authors of papers cited to have misrepresented their work. Subsequently, half of the editorial staff of the journal resigned in protest at what they saw as a failure of peer review, and the managing director of the journal's parent company issued an apology (see Wikipedia's summary). The emails show that these scientists were upset by Climate Research's publication of bad science and encouraged protest and those resignations.

A few blog posts that seem to have good overviews of the issues:

• Skeptical Science, "What do the hacked CRU emails tell us?"
• Real Climate, "The CRU Hack" (followed by over 1,000 comments)
• Real Climate, "The CRU Hack: Context" (gives background on some of the issues, including the above Climate Research issue).
  
• Greenfyre's, "Climate change Deniers hoax themselves... again."
• Millard Fillmore's Bathtub, "Smoking guns in the CRU stolen emails: A real tale of real ethics in science" (what the CRU emails show about treatment of an erroneous climate science paper)
  

An interesting comparison to past scientific controversy is:

• Carbon Fixated, "Newtongate: the final nail in the coffin of Renaissance and Enlightenment 'thinking'"

And, to compare to the climate change skeptics:

• Bishop Hill, "Climate cuttings 33" (summary of the apparent worst issues)
• Essex County Conservative Examiner, "Hadley [sic] CRU hacked with release of hundreds of docs and emails"
• Climate Audit, "CRU Correspondence"
• Watt's Up With That, "Mike's Nature Trick"
• Watt's Up With That, "CRU Emails 'may' be open to interpretation, but commented code by the programmer tells the real story"
• Watt's Up With That, "Spencer on elitism in the IPCC climate machine"

The last of these posts, from Univ. of Alabama climate scientist and skeptic Roy W. Spencer, notes that:

If all of this sounds incompatible with the process of scientific investigation, it shouldn’t. One of the biggest misconceptions the public has about science is that research is a straightforward process of making measurements, and then seeing whether the data support hypothesis A or B. The truth is that the interpretation of data is seldom that simple.

There are all kinds of subjective decisions that must be made along the way, and the scientist must remain vigilant that he or she is not making those decisions based upon preconceived notions. Data are almost always dirty, with errors of various kinds. Which data will be ignored? Which data will be emphasized? How will the data be processed to tease out the signal we think we see?

Hopefully, the scientist is more interested in discovering how nature really works, rather than twisting the data to support some other agenda. It took me years to develop the discipline to question every research result I got. It is really easy to be wrong in this business, and very difficult to be right.

Skepticism really is at the core of scientific progress. I’m willing to admit that I could be wrong about all my views on manmade global warming. Can the IPCC scientists admit the same thing?

Another noteworthy comment, from Real Climate, is this one from caerbannog and Gavin Schmidt's reply:

Just a reminder: CRU is just one of many organizations focusing on climate research. The fact that its director has reacted badly (i.e. appearing to go for the “bunker” mentality) to repeated scurrilous attacks has no bearing on the validity of the science.

Hansen’s approach has been quite different — he’s basically said to his detractors, “here are all of the source code and data — go knock yourselves out”.

Under Hansen, the NASA/GISS data and source code have been freely available on-line for years. And all of the sceptics’ scrutiny of said data has uncovered only one or two minor “glitches” that have had minimal impact.

Just a quick question (or two) to Gavin, if you feel the need to spend even more of your weekend downtime answering questions here.

Given that all of your climate-modeling source-code has been available for public scrutiny for quite a long time, and given that anyone can download and test it out, how many times have climate-model critics have actually submitted patches to improve your modeling code, fix bugs, etc? Have you gotten *any* constructive suggestions from the skeptic camp?

[Response: Not a single one. - gavin]

I think this illustrates that it's far better to be completely open with your data and methods.

UPDATE (November 26, 2009): There's now an official response from the Univ. of East Anglia, the Climate Research Unit, and Phil Jones. Jones notes, regarding the Freedom of Information requests:

We have been bombarded by Freedom of Information requests to release the temperature data that are provided to us by meteorological services around the world via a large network of weather stations. This information is not ours to give without the permission of the meteorological services involved. We have responded to these Freedom of Information requests appropriately and with the knowledge and guidance of the Information Commissioner.

We have stated that we hope to gain permission from each of these services to publish their data in the future and we are in the process of doing so.

UPDATE (December 4, 2009): The journal Nature has weighed in on the controversy.

Climate scientist Judith Curry makes good points of criticism about climate scientists' behavior.

UPDATE (December 6, 2009): Univ. of East Anglia climate scientist Mike Hulme (author of Why We Disagree About Climate Change, a book that I read several chapters from in a class on human dimensions of climate change this semester) on the issue:

The key lesson to be learned is that not only must scientific knowledge about climate change be publicly owned — the I.P.C.C. does a fairly good job of this according to its own terms — but the very practices of scientific enquiry must also be publicly owned, in the sense of being open and trusted. From outside, and even to the neutral, the attitudes revealed in the emails do not look good. To those with bigger axes to grind it is just what they wanted to find.

This will blow its course soon in the conventional media without making too much difference to Copenhagen — after all, COP15 is about raw politics, not about the politics of science. But in the Internet worlds of deliberation and in the ‘mood’ of public debate about the trustworthiness of climate science, the reverberations of this episode will live on long beyond COP15. Climate scientists will have to work harder to earn the warranted trust of the public - and maybe that is no bad thing.

But this episode might signify something more in the unfolding story of climate change. This event might signal a crack that allows for processes of re-structuring scientific knowledge about climate change. It is possible that some areas of climate science has become sclerotic. It is possible that climate science has become too partisan, too centralized. The tribalism that some of the leaked emails display is something more usually associated with social organization within primitive cultures; it is not attractive when we find it at work inside science.

It is also possible that the institutional innovation that has been the I.P.C.C. has run its course. Yes, there will be an AR5 but for what purpose? The I.P.C.C. itself, through its structural tendency to politicize climate change science, has perhaps helped to foster a more authoritarian and exclusive form of knowledge production - just at a time when a globalizing and wired cosmopolitan culture is demanding of science something much more open and inclusive.

UPDATE (December 11, 2009): PolitiFact gives its analysis of the CRU emails, which is fairly balanced.

UPDATE (December 12, 2009): Deep Climate catches Stephen McIntyre engaging in quote mining of the CRU emails in order to mislead.

UPDATE (December 24, 2009): David Douglass and John Christy, in "A Climatology Conspiracy?", argue that the CRU emails show a concerted effort to delay the publication of their paper, publish another paper criticizing it along side of it, and deny them the right of final reply. Their case is somewhat weakened by the fact that the second paper points out a significant error in their paper and they have apparently not tried to publish a reply or correct the error.