The U.N.’s Intergovernmental Panel on Climate Change holds some of the greatest responsibility when it comes to communicating climate science, if only because it is so high-profile and regarded as the gold standard of climate science. Every five to seven years or so, the IPCC releases an assessment report reviewing the recent research of thousands of climate scientists around the world. Each assessment report is released in a series of sections devoted to specific topics, and each section is accompanied by a “summary for policymakers” (SPM), which is intended to summarize the findings for a non-scientific audience, particularly government officials who can use the information to help create new climate policies.
The reports also tend to receive extensive coverage in the media, and more so as the years go on and the international focus on climate change continues to sharpen. This means that the IPCC’s assessments are reaching a bigger audience than ever before.
But although the IPCC’s reach has been expanding, its reports have not necessarily become easier for the layperson to understand — in fact, just the opposite, argues a new study, published today in Nature Climate Change. The study uses analysis software to find that the readability of the IPCC’s SPMs has generally deteriorated over time, even as media coverage of it has become increasingly readable.
“We started looking into IPCC communications because we (in line with many other people) had the feeling that IPCC summaries for policymakers are quite simply difficult to read and to understand,” said Ralf Barkemeyer, an associate professor at the KEDGE Business School and the study’s lead author, in an e-mail to The Post. Although the organization does employ a communications staff, the IPCC has been criticized in the past for problems with communication, not only when it comes to the language of the reports, but also when it comes to relaying its internal decision-making processes to the public.
Barkemeyer and his colleagues used two different tools to analyze the texts of the SPMs and their corresponding media coverage: an algorithm which assesses readability and a separate software which assesses how optimistic a text is in tone. They applied these analysis tools to four types of publications: the IPCC reports and related articles in popular science publications (namely, the journals Science and Nature), newspapers and tabloids.
In general, the IPCC reports were the least readable and newspapers and tabloids were the most readable, with popular science publications occupying a middle ground between them. For instance, one of the SPMs from the latest assessment report includes such complex phrasing as, “Mitigation scenarios reaching concentration levels of about 500 ppm CO2eq by 2100 are more likely than not to limit temperature change to less than 2 °C relative to pre-industrial levels, unless they temporarily ‘overshoot’ concentration levels of roughly 530 ppm CO2eq before 2100, in which case they are about as likely as not to achieve that goal.”
Media coverage was also generally more pessimistic in tone than the IPCC reports themselves — perhaps unsurprisingly. As the authors note, “Newspapers need to turn a piece of scientific information into a piece of news, which among other aspects requires bringing future climate change consequences into the sphere of immediate interest of the reader. Using emotive language is one of the journalistic strategies for bringing the future into the immediate.”
What was surprising, according to Barkemeyer, was that there was no improvement in the SPMs’ readability as more reports were released over time. “Given the huge amount of attention that has been paid to this topic in recent years, we would definitely have expected to see some improvements over time,” Barkemeyer said in his e-mail — but in fact, some sections of the reports seemed to become less readable as time went on.
The assessment reports are typically released in different sections, which address specific topics and are published by three designated committees, or “working groups,” each of which also provides its own summary for policymakers. The researchers found that while the readability of working group I’s SPMs remain fairly stable over time, working groups II and III deteriorate over time. In contrast, readability for newspapers and popular science magazines did increase as time went on, peaking in 2007.
The authors also found that political tensions may play a role in an SPM’s readability. Every time an assessment report is released, it goes through a round of edits known as the plenary process. As the authors note, “The plenary process is important to the SPM because its ‘approval’ means that the material has been subjected to detailed line-by-line discussion and agreement between government delegates and authors.” Yet when they compared the SPMs’ readability pre- and post-plenary, they found that the plenary process actually lowered readability in five out of eight cases.
“We found a strong relationship between political mood and SPM readability,” the authors write. They assessed political mood by examining reports on the plenary processes from the Earth Negotiations Bulletin reporting service. When political tensions were running high, the readability went down after the plenary process was complete.
Low readability is a problem if it means that the reports aren’t clear to the policymakers they’re meant to inform, Barkemeyer said. “Hard-to-understand summaries are more likely to be misunderstood – and it will then be necessary that intermediaries ‘translate’ information from these summaries into a language that policymakers can understand,” he wrote in his e-mail. “Thus, the communication process becomes more complicated, with more actors involved, and a higher likelihood of misinterpretations and disagreements along the way.”
“There is no doubt that more needs to be done to make IPCC reports more readable and accessible,” said an IPCC statement on the study e-mailed to The Post. “The newly elected Chair of the IPCC, Hoesung Lee, has made this point and we are determined to tackle it. The challenge is to do it in a way that does not damage the scientific rigor and robustness of the reports, or allow important nuances in them to be lost.”
The IPCC has already made some steps toward improving its communication, the statement noted. “The previous Session of the IPCC, in Nairobi in February this year, took decisions to enhance the usability of IPCC reports, for instance by using digital technology to share and disseminate information, and to draw on specialists (such as science writers and graphical designers) to enhance the readability of IPCC reports. The IPCC is holding an expert meeting in February 2016 to discuss lessons learned from communicating the Fifth Assessment Report, and when the scoping process for the Sixth Assessment Report starts next year the Panel will have an opportunity to tackle this.”
But it may be that the low general readability scores indicated in this study don’t actually pose that much of a problem for the documents’ intended audiences, said Michael Oppenheimer, a professor of geosciences and international affairs at Princeton University who was a lead author in the IPCC’s fourth assessment report and a coordinating lead author on the IPCC’s special SREX report on extreme climate events and disasters.
“This and other reviews of IPCC communication don’t emphasize enough that the target is primarily governments (for example, the staff of EPA or DoE or an environment ministry), not the average newspaper reader (or even a scientifically attuned reader),” Oppenheimer said in an e-mail to The Post. “The main point of the four-day plenaries which produce the SPMs is so that the material drafted by scientists can be converted to a form that is understandable to governments while still accurately reflecting the science.”
He added: “One might ask this: If, as the study finds, newspapers and other intermediaries are doing a progressively better job of communicating IPCC findings to the larger public, and if governments are happy with the SPMs, is there really a problem?”
Still, it remains unclear whether the science in the documents is actually being interpreted correctly by the policymakers and journalists who read them. On this front, Oppenheimer agrees with the authors that improvements could still be made to the IPCC’s writing teams in order to ensure that the documents are likely to be understood correctly by their readers.
One option would be to hire professional science communicators to help write the reports, Barkemeyer said. But, he cautioned, “potential benefits could be outweighed by the addition of yet another set of actors in the process, potentially distorting and politicizing the original voice of the scientific panel.” A simpler course of action could be to simply provide science communication training for the reports’ existing authors. A U.S. based climate communication group has already begun providing such a service for U.S. climate reports. A team of experts from the organization served as communication advisers on the 2014 U.S. National Climate Assessment in order to improve its readability and accessibility to the general public.
In any case, the increasing urgency of international efforts to combat climate change, along with corresponding growth in media coverage and public interest, means the need for effective science communication will only become more relevant as time goes on.
And while the authors note that the IPCC is already aware of the challenges associated with translating climate science to a lay audience, and has taken steps over the years to improve its communication, Barkemeyer concludes: “Our findings illustrate that there is still a lot of room for improvement.”