It's been done before, or it's wrong. Part I*
(* Aug 28th: After receiving comments from several of the authors I have made a few revisions to the original post. I also added some details about the author's comments at the end, along with extra thoughts from me.)
That's how it feels sometimes when looking at new science results. Either the result seems like old news, or it's hard to believe. In practice things are usually more complicated than this simple dichotomy. In part one of this two-part blog post I'll give a recent example from astronomy of a result that was advertised as new, but reallywasn't may not have been.
In the middle of July several press releases went out about a blue exoplanet. For example, the title of the European Space Agency (ESA) release was: "Hubble spots azure blue planet - true color of exoplanet measured for the first time". At the Knight Science Journalism Tracker science writer Charlie Petit wrote a summary of this story and the news coverage. Many writers emphasized this was the first time that an exoplanet's color had been determined.
But was it? In the paper, "The Deep Blue Color of HD189733b: Albedo Measurements with Hubble Space Telescope/Space Telescope Imaging Spectrograph at Visible Wavelengths", the authors show that the albedo of this exoplanet decreases from the ultraviolet to the red part of the optical spectrum. Albedo is the fraction of incident light that is reflected by a surface, so higher albedo at shorter optical wavelengths means that HD 189733b would appear to be blue.
In the discussion section, the authors say:
A search in ADS for the name Berdyugina uncovers these two conference proceedings papers from 2011: "Polarimetry of Hot Inflated Jupiters Reveals Their Neptune-like Blue Appearance" and "Exoplanets as blue as Neptune".
The press release from NASA and Space Telescope Science Institute (STScI) does not say this was the first time that an exoplanet's color had been determined, and they include this statement:
[Being a skeptic, I tried to see if the NASA/STScI release had been edited and fixed afterwards, by looking at independent postings of the release. I didn't find evidence for a change.]
The ESA press release, on the other hand, uses the word "first" four different times - in the sense described above - including the following:
an unjustified first that may not be justified.
How much would the press coverage have been affected if none of the press releases had claimed this was a first? That question cannot be answered, but I'm guessing it would have taken a hit. Science writer Nancy Atkinson, from Universe Today, tweeted that she already knew this exoplanet was blue before the release:
which prompted me to read the paper and research this blog post.
Atkinson still wrote about the story, but with the headline: "Hubble Confirms Exoplanet Has a Blue Atmosphere". Kudos to Nancy Atkinson.
It seems like Svetlana Berdyugina noticed problems with the publicity claims. I noticed this graphic on Wikipedia Commons about her 2011 paper. The graphic has a date of July 11th 2013, the day when the publicity for the new paper occurred.
One reason I was interested in describing this result correctly is that we recently released a new Chandra result for HD 189733b and we wanted to mention this recent work. Thanks to Nancy, we got it right. Coincidentally, our release mentions ESA because XMM-Newton was used in some of the author's X-ray analysis.
The importance of trust
Several good, experienced writers missed the mixed messages in the press releases. If they'd read the paper they would have seen the recognition of previous work and the lack of a claimed first. Of course, many writers don't have time to read papers carefully - or at all - because they have enormous workloads. But, there was a possible shortcut here, taking advantage of a public link to the paper provided by most of the releases. A quick search of the paper for "first" with control-F quickly uncovers the issues I've explained here (I do lots of searches like this myself, thanks to the ubiquity of PDF format). This shortcut won't work as well in many cases, but it can still be useful for quick comparisons between the press release and the paper.
With writer's time shortage it's more important than ever for reliable publicity by PIOs, scientists & their colleagues. We work hard at this. For example, if you check our press release for the X-ray transit by HD 189733b you will see two different uses of "first". Both of these are taken straight from the Astrophysical Journal paper that we based the release on. I don't think it's absolutely essential for claims to be in the science paper, but it's a good foundation. Even if you stick closely to the paper, there are no guarantees. Peer review is a useful filter but it isn't infallible, as well-known cases like arseniclife have shown us.
I'll note that we've made our own mistakes in doing publicity. However, our goal is to keep improving, in part by learning from the successes and mistakes made by others. Occasional mistakes are impossible to avoid, but if scientists see that we've consistently exaggerated or distorted the science then they may be less likely to want to work with us. As PIO Matt Shipman explains in this excellent blog post about a study in Germany, good, active PIOs make scientists more likely to publicize their results. As Matt says:
End-note #1: part II of this blog post is about a result that is so new I found it difficult to believe at first, and comes with a different set of lessons for science communication. Stay tuned.
Story updates: For some details about responses from the authors after I sent them my original blog post, and for extra thoughts from me, please read on. As a warning, my scientist's love of detail is on full display.
Update (Aug 20th):
I sent this blog post to Svetlana Berdyugina who agreed with it and provided some more background information. She and her collaborators put out a press release in late 2007 that already pointed to the possible blue color of HD 189733b. Note the blue color of the planet in the illustration. This work was then confirmed by the paper discussed above, Berdyugina et al. (2011). She also provided this presentation from 2011 that discusses her more detailed work on HD 189733b from her 2011 paper and gives evidence that another planet, Upsilon And b, is also blue.
Extra update (Aug 28th):
After contacting Svetlana Berdyugina and posting her comment, I sent the post to Frederic Pont and Tom Evans, the first two authors of the Evans et al. (2013) paper, "The Deep Blue Color of HD189733b: Albedo Measurements with Hubble Space Telescope/Space Telescope Imaging Spectrograph at Visible Wavelengths" (arXiv) and now published in The Astrophysical Journal Letters. I received a polite, interesting and very detailed response from them, explaining what they were thinking with the publicity for their new paper. I'll mention part of their response here, for the comments involving publicly available information about HD 189733b work:
(1) Pont and Evans mention skepticism in published papers about the polarization work of Berdyugina and collaborators, the authors of the 2011 paper mentioned in my post. One is a paper in Monthly Notices of the Royal Astronomical Society from early 2009 (arXiv) by Lucas et al. that criticizes a paper in The Astrophysical Journal (ApJ) from 2008 (arXiv) by Berdyugina et al. The second is a 2009 ApJ paper (arXiv) by Wiktorowicz that fails to detect the polarization signal claimed by Berdyugina et al. in the same 2008 ApJ paper criticized by Lucas et al.
(2) They mention that the color inferred in their 2013 paper differs from the Neptune-like blue color for HD 189733b claimed by Berdyugina et al. (2011). Instead, they are claiming a "deep blue" color, as their paper's title and the text clearly states.
(3) They also point out that the Universe Today article mentioned by Nancy Atkinson refers to an HST press release in 2012 that already stated HD 189733b is blue. This was based on work that did not include Berdyugina, but included one of Evan's and Pont's collaborators.
Additional update from me (Aug 28th):
Point (1) above by Pont and Evans is the most important comment. If the Berdyugina et al. (2011) paper is not robust, it has a strong effect on claims of priority. This is something for experts to decide and I'm certainly not one of them, however I will make a few general comments. The papers mentioned above in (1) are both from 2009 and though they criticize a related paper (Berdyugina et al. (2008)) they cannot directly address Berdyugina et al. (2011) since they predate it. Also, Berdyugina et al. (2011) discuss the Wiktorowicz (2009) paper's non-detection in detail and say in the abstract, for example: "Therefore, the nondetection by Wiktorowicz, based on a measurement integrated within a broad passband covering the V band and partly covering the B and R bands, is inconclusive and consistent with our detection in B." Apparently the ApJ referee accepted this argument. I looked at the papers citing Berdyugina et al. (2011) and see that several express caution and reference one or both of the 2009 papers cited above, but I did not find subsequent papers that challenged Berdyugina et al. (2011) in detail.
In comparing Berdyugina et al. (2011) and Evans et al. (2013) while researching my original post I noted that the latter say in their paper:
Given this difficult situation, I think the STScI/NASA press release took the best approach by simply following the science paper and not claiming a first.
I seem to have stepped into a controversy and what may be an ongoing debate. As a relatively inexperienced blogger I feel like I'm working above my pay-grade here, but I have learned a few things. Since I wasn't on any sort of deadline, I had plenty of time to ask Evans et al. (2013) why they made their publicity claims before publishing this blog post, and I should have done so. Because they've raised unanswered questions about a key paper, I changed the wording in my post to be more circumspect in a few places. Ironically, this post was held up because I was waiting for authors of a different paper in another field to answer my questions about apparent inconsistencies in numbers in their publicity. Eventually I decided to split that discussion off to a separate blog post.
Members of both teams are welcome to give their own comments on this post if they feel I've left out crucial information (I'm not comfortable posting their full comments). However, I suggest they communicate directly with each other and with others in their field to discuss any unresolved issues. If they make progress I would be happy to host a guest post or two from them in the future.
There will be no further updates to this post.
That's how it feels sometimes when looking at new science results. Either the result seems like old news, or it's hard to believe. In practice things are usually more complicated than this simple dichotomy. In part one of this two-part blog post I'll give a recent example from astronomy of a result that was advertised as new, but really
In the middle of July several press releases went out about a blue exoplanet. For example, the title of the European Space Agency (ESA) release was: "Hubble spots azure blue planet - true color of exoplanet measured for the first time". At the Knight Science Journalism Tracker science writer Charlie Petit wrote a summary of this story and the news coverage. Many writers emphasized this was the first time that an exoplanet's color had been determined.
 |
| An illustration showing HD 189733b. Credit: NASA, ESA, M.Kornmesser |
In the discussion section, the authors say:
"We are not the first to claim that the albedo of HD 189733b decreases across the visible wavelength range"and later they say:
"Berdyugina et al. (2011) hypothesize that HD 189733b might have a reflection spectrum similar to Neptune."Here's the paper by Berdyugina et al. (2011) and here's a picture of Neptune:
 |
| Neptune. Credit: NASA, Voyager 2 |
A search in ADS for the name Berdyugina uncovers these two conference proceedings papers from 2011: "Polarimetry of Hot Inflated Jupiters Reveals Their Neptune-like Blue Appearance" and "Exoplanets as blue as Neptune".
The press release from NASA and Space Telescope Science Institute (STScI) does not say this was the first time that an exoplanet's color had been determined, and they include this statement:
"Earlier observations have reported evidence for scattering of blue light on the planet. The latest Hubble observation confirms the evidence."Kudos to STScI and NASA.
[Being a skeptic, I tried to see if the NASA/STScI release had been edited and fixed afterwards, by looking at independent postings of the release. I didn't find evidence for a change.]
The ESA press release, on the other hand, uses the word "first" four different times - in the sense described above - including the following:
""This planet has been studied well in the past, both by ourselves and other teams," says Frédéric Pont of the University of Exeter, UK, leader of the Hubble observing programme and an author of this new paper. "But measuring its colour is a real first - we can actually imagine what this planet would look like if we were able to look at it directly.""They then say:
"Earlier observations using different methods have reported evidence for scattering of blue light on the planet, but these most recent Hubble observations give robust confirming evidence, say the researchers."So, they say it's a real first and it confirms previous work. It's not just the ESA release. Press releases from University of Exeter and University of Oxford also try to hedge their bets or claim a
How much would the press coverage have been affected if none of the press releases had claimed this was a first? That question cannot be answered, but I'm guessing it would have taken a hit. Science writer Nancy Atkinson, from Universe Today, tweeted that she already knew this exoplanet was blue before the release:
OK, researching Hubble obs of the "blue" exoplanet...what is new here? We knew it had a blue atmosphere last year: http://t.co/fylYPpGy0z
— Nancy Atkinson (@Nancy_A) July 11, 2013
which prompted me to read the paper and research this blog post.
Atkinson still wrote about the story, but with the headline: "Hubble Confirms Exoplanet Has a Blue Atmosphere". Kudos to Nancy Atkinson.
 |
| A comparison of the albedo from Berdyugina et al. (2011) with that of Neptune. Credit: Svetlana Berdyugina, Wikipedia. |
One reason I was interested in describing this result correctly is that we recently released a new Chandra result for HD 189733b and we wanted to mention this recent work. Thanks to Nancy, we got it right. Coincidentally, our release mentions ESA because XMM-Newton was used in some of the author's X-ray analysis.
 |
| An artist's impression of HD 189733b, with an inset showing the Chandra image. For more details see our long caption. Credit: X-ray: NASA/CXC/SAO/K.Poppenhaeger et al; Illustration: NASA/CXC/M.Weiss |
The importance of trust
Several good, experienced writers missed the mixed messages in the press releases. If they'd read the paper they would have seen the recognition of previous work and the lack of a claimed first. Of course, many writers don't have time to read papers carefully - or at all - because they have enormous workloads. But, there was a possible shortcut here, taking advantage of a public link to the paper provided by most of the releases. A quick search of the paper for "first" with control-F quickly uncovers the issues I've explained here (I do lots of searches like this myself, thanks to the ubiquity of PDF format). This shortcut won't work as well in many cases, but it can still be useful for quick comparisons between the press release and the paper.
With writer's time shortage it's more important than ever for reliable publicity by PIOs, scientists & their colleagues. We work hard at this. For example, if you check our press release for the X-ray transit by HD 189733b you will see two different uses of "first". Both of these are taken straight from the Astrophysical Journal paper that we based the release on. I don't think it's absolutely essential for claims to be in the science paper, but it's a good foundation. Even if you stick closely to the paper, there are no guarantees. Peer review is a useful filter but it isn't infallible, as well-known cases like arseniclife have shown us.
I'll note that we've made our own mistakes in doing publicity. However, our goal is to keep improving, in part by learning from the successes and mistakes made by others. Occasional mistakes are impossible to avoid, but if scientists see that we've consistently exaggerated or distorted the science then they may be less likely to want to work with us. As PIO Matt Shipman explains in this excellent blog post about a study in Germany, good, active PIOs make scientists more likely to publicize their results. As Matt says:
"Further, the authors suspected that, since scientists are often wary of reporters, the influence of university PR “is presumably also dependent on PR’s ability to create an impression of competence. If scientists perceive expertise and feel well supported, they will probably be more willing to cooperate."If I can't be competent, at least I like to create an impression of competence! More seriously, this gels with my own experience. I recall one scientist saying she never wanted to work with her home institution again because of a bad experience doing publicity. And, I recently had a scientist tell me he had no idea why anyone would want to do publicity because the science just gets exaggerated. Even in a field like astronomy, scientists can be deeply suspicious of publicity efforts. One of our goals is to publicize the wonderful results in astronomy without losing the trust of the scientists who do the work.
End-note #1: part II of this blog post is about a result that is so new I found it difficult to believe at first, and comes with a different set of lessons for science communication. Stay tuned.
Story updates: For some details about responses from the authors after I sent them my original blog post, and for extra thoughts from me, please read on. As a warning, my scientist's love of detail is on full display.
Update (Aug 20th):
I sent this blog post to Svetlana Berdyugina who agreed with it and provided some more background information. She and her collaborators put out a press release in late 2007 that already pointed to the possible blue color of HD 189733b. Note the blue color of the planet in the illustration. This work was then confirmed by the paper discussed above, Berdyugina et al. (2011). She also provided this presentation from 2011 that discusses her more detailed work on HD 189733b from her 2011 paper and gives evidence that another planet, Upsilon And b, is also blue.
Extra update (Aug 28th):
After contacting Svetlana Berdyugina and posting her comment, I sent the post to Frederic Pont and Tom Evans, the first two authors of the Evans et al. (2013) paper, "The Deep Blue Color of HD189733b: Albedo Measurements with Hubble Space Telescope/Space Telescope Imaging Spectrograph at Visible Wavelengths" (arXiv) and now published in The Astrophysical Journal Letters. I received a polite, interesting and very detailed response from them, explaining what they were thinking with the publicity for their new paper. I'll mention part of their response here, for the comments involving publicly available information about HD 189733b work:
(1) Pont and Evans mention skepticism in published papers about the polarization work of Berdyugina and collaborators, the authors of the 2011 paper mentioned in my post. One is a paper in Monthly Notices of the Royal Astronomical Society from early 2009 (arXiv) by Lucas et al. that criticizes a paper in The Astrophysical Journal (ApJ) from 2008 (arXiv) by Berdyugina et al. The second is a 2009 ApJ paper (arXiv) by Wiktorowicz that fails to detect the polarization signal claimed by Berdyugina et al. in the same 2008 ApJ paper criticized by Lucas et al.
(2) They mention that the color inferred in their 2013 paper differs from the Neptune-like blue color for HD 189733b claimed by Berdyugina et al. (2011). Instead, they are claiming a "deep blue" color, as their paper's title and the text clearly states.
(3) They also point out that the Universe Today article mentioned by Nancy Atkinson refers to an HST press release in 2012 that already stated HD 189733b is blue. This was based on work that did not include Berdyugina, but included one of Evan's and Pont's collaborators.
Additional update from me (Aug 28th):
Point (1) above by Pont and Evans is the most important comment. If the Berdyugina et al. (2011) paper is not robust, it has a strong effect on claims of priority. This is something for experts to decide and I'm certainly not one of them, however I will make a few general comments. The papers mentioned above in (1) are both from 2009 and though they criticize a related paper (Berdyugina et al. (2008)) they cannot directly address Berdyugina et al. (2011) since they predate it. Also, Berdyugina et al. (2011) discuss the Wiktorowicz (2009) paper's non-detection in detail and say in the abstract, for example: "Therefore, the nondetection by Wiktorowicz, based on a measurement integrated within a broad passband covering the V band and partly covering the B and R bands, is inconclusive and consistent with our detection in B." Apparently the ApJ referee accepted this argument. I looked at the papers citing Berdyugina et al. (2011) and see that several express caution and reference one or both of the 2009 papers cited above, but I did not find subsequent papers that challenged Berdyugina et al. (2011) in detail.
In comparing Berdyugina et al. (2011) and Evans et al. (2013) while researching my original post I noted that the latter say in their paper:
"We are not the first to claim that the albedo of HD 189733b decreases across the visible wavelength range. Berdyugina et al. (2008, 2011) used polarimetry to infer albedos of A_g = 0.61±0.12 in the B-band (390–480 nm) and A_g = 0.28±0.16 in the V-band (500–590 nm). Our results are systematically ∼2σ lower than these values."In other words there is only a hint (at most) that the albedo values determined by Berdyugina et al. (2011) and Evans et al. (2013) are different. I'll take an extreme position by assuming this is a fluke and that the Berdyugina et al. (2011) paper is actually seriously flawed. In this case, the claim in some of the press releases that Evans et al. (2013) have detected the color of an exoplanet for the first time is probably robust. However, even if this is true the contradictory claims in some of the press releases are still a problem. Also, the mention by Evans et al. (2013) of the Berdyugina et al. (2011) work without criticism is polite but potentially misleading, given the conflict with the press release statements.
Given this difficult situation, I think the STScI/NASA press release took the best approach by simply following the science paper and not claiming a first.
I seem to have stepped into a controversy and what may be an ongoing debate. As a relatively inexperienced blogger I feel like I'm working above my pay-grade here, but I have learned a few things. Since I wasn't on any sort of deadline, I had plenty of time to ask Evans et al. (2013) why they made their publicity claims before publishing this blog post, and I should have done so. Because they've raised unanswered questions about a key paper, I changed the wording in my post to be more circumspect in a few places. Ironically, this post was held up because I was waiting for authors of a different paper in another field to answer my questions about apparent inconsistencies in numbers in their publicity. Eventually I decided to split that discussion off to a separate blog post.
Members of both teams are welcome to give their own comments on this post if they feel I've left out crucial information (I'm not comfortable posting their full comments). However, I suggest they communicate directly with each other and with others in their field to discuss any unresolved issues. If they make progress I would be happy to host a guest post or two from them in the future.
There will be no further updates to this post.
Comments
Post a Comment