The Nearest Exoplanet Discovered. Probably.
The astronomy news last week was dominated by one story: a planet may have been discovered around the Sun-like star Alpha Centauri B, as reported in Nature. This is exceptional because, if confirmed, it will be the nearest exoplanet to the Earth. The mass of this exoplanet candidate is about the same as the Earth, which would make it the lightest exoplanet ever found around a star like the Sun. This is an exciting result.
The details of the astrophysics have been well described by many different authors, including several that I cite below. Instead of covering this well-trodden ground, I'll discuss how the result was communicated to the public, including the issue of expressing uncertainty in describing new results. I'll also mention the embargo break that presented some challenges for those doing the publicity and for science writers.
Did you notice how carefully I worded the description of this result? I used "may", "if confirmed" and "planet candidate". The exoplanet candidate has not been observed directly. Its existence is inferred from the tiny, rhythmic wobble apparently seen in the star Alpha Centauri B, after several, much stronger signals have been subtracted away. No other team has reported evidence for planets in the Alpha Centauri system, and if they had, this new paper wouldn't be such big news.
I've read the paper and I think there's a good chance that a planet has been found, but I think confirmation is needed (the reader might ask how I even have a clue about this: see a footnote [1] for a brief explanation). I'm not the only person who thinks this. Artie Hatzes, an astronomer from Thuringian State Observatory, in Tautenburg, Germany wrote a News and Views article for Nature where he expressed similar views. He uses words like "possible extrasolar planet", and "if it is confirmed" and uses the phrase "candidate exoplanet" four times. He also uses the "extraordinary claims require extraordinary evidence" line, from Carl Sagan and says that although a "planet-like" signal is found, the "discovery does not quite provide the "extraordinary evidence"".
Another planet expert, Debra Fischer, agrees that confirmation is needed. In a Planetary Society interview with Bruce Betts, she compliments the authors of the Nature paper for their new analysis techniques, but says: "Nevertheless, because these corrections essentially constitute a new approach, confirmation is critical."
All of the science writers on Nature's embargo list were sent a copy of the News and Views article in advance. So, most of them would have been aware of Hatzes' reservations. How much of this doubt made it into the media coverage of this paper? A good amount of it did. Dan Vergano from USA Today wrote one of the more carefully worded articles beginning with this summary:
Nadia Drake, writing at Science News includes a good quote from Debra Fischer: "It's a tough detection, there's absolutely no question about it," Fischer says. "A detection that's as technically difficult as this one requires confirmation." Drake then segues into an excellent, detailed discussion of plans to confirm the result.
John Timmer, writing for Ars Technica, gave a good, detailed explanation of the technique used by the planet hunters, as they eliminated other signals until a "a hint of a signal with a periodicity of 3.3 days" was left. His statement that "it's probably a planet" (his emphasis) rings true.
Brian Vastag from the Washington Post quoted Artie Hatzes about the need for confirmation by other teams. Dennis Overbye, from the New York Times, wrote that astronomers were electrified by the news of the planet but they also cautioned that it needed to be confirmed by other astronomers. I was interested in seeing more here, but he doesn't give any details about the doubts or the astronomers who expressed them, because he goes on to quote Debra Fischer saying it's the story of the decade. Maybe there was an overzealous editor at work here.
Phil Plait, at his Discover Magazine blog "Bad Astronomy" wrote an extremely enthusiastic description of the result along with an impressive level of technical detail and some science fiction perspective. He doesn't provide any obvious qualifiers, but he's an experienced astronomer so this can be considered an opinion piece as well as a report. He read the paper and found it "pretty convincing".
Other writers also decided not to include qualifiers about the result, or had editors remove them. Instead of calling these out, I'll give a few possible explanations for these omissions. First, the authors addressed concerns about the significance of the result on a press conference held by ESO. The first question was from Seth Borenstein, of the Associated Press, who asked
1000 5000 chance that this is a "false-alarm". Other writers asked questions about seeking confirmation.
Second, the paper itself does not use qualifiers. For example, the abstract includes: "Here we report the detection of an Earth-mass planet orbiting our neighbour star Alpha Centauri B". Presumably, most or all of the authors of the paper along with the two or three referees agreed that qualifiers weren't necessary. So, adopting the view that peer-reviewed papers can offer a useful approximation of The Truth, one can argue that qualifiers were not essential for a press report.
Third, there was no shortage of exoplanet experts who were very excited about the result and who weren't expressing doubts, at least in quotes made public. This includes people like Geoff Marcy, Alan Boss and Sara Seager. Science writer Ross Andersen, writing for the Atlantic, asked Sara Seager whether this is a candidate or if it has been confirmed, and she responded "To answer your second question, we consider this particular finding a detection. It's not a candidate; it doesn't need to be confirmed. It's a detection." What's going on here? It's not a shock to see scientists disagree about the credibility of a new result, especially if they have different backgrounds. In this case, Hatzes and Fischer are mainly observers, while Seager does more theory. It's possible that there are differences being used in the definition of key words like "detection" and "candidate", with the observers demanding a higher threshold for detection.
It's well known that people can have subtle biases that make it difficult to be objective, and scientists are not exempt from these problems. I won't attempt to speculate about other people's point-of-view, but I'll give you a personal example. Do I like Artie Hatzes' News and Views article because I'm convinced it's the most objective and carefully-considered reaction to this new result, or because he happened to give a very positive referee's report to an early paper of mine? I don't think it's the latter, but it's difficult to say I don't harbor *some* bias. I *will* openly admit my experience doing publicity and following others doing it is an issue for me, as I've seen results come and go or be seriously contested, some recently.
Other challenges for writers might be connected to any word limits they have. If you have described the basics of a result and only have a hundred words left, you might see a choice between describing the need for confirmation, which some might find boring, and talking about the exciting possibility of sending probes to Alpha Cen B, or even traveling there. Bloggers don't have word limits - as I'm reminding readers with this article - and so they can fit more details in. However, as Vergano's article shows, it's possible to be concise and conservative. Another issue for writers was likely a lack of time, exarcerbated by the shorter than usual embargo time given by Nature and eventual embargo break, as I describe in the next section.
The science media, especially the mainstream media, is good at describing the latest big discovery (spurred on by press releases produced by people like me, but that's a story for another blog post). However, this can give a distorted view of how science really works. As Andrew Revkin said in his very first blog post at dotearth:
I asked for comments from planet expert Dave Charbonneau who passed along, via email, what he told others:
Does Sagan's principle of "extraordinary claims require extraordinary evidence" really apply here? A less vague way of applying that requirement might be: is an unusually high statistical significance required for a result of unusually high public interest? I would assert that it isn't. But, when there's such widespread interest I think there's a greater need to explain potential concerns with reliability and the steps required for confirmation.
Given the challenges involved with this story and the other explanations given above, I think writers did their jobs well. However, I think that serious discussion of uncertainty and the need for confirmation can make the difference between a good story and a great story. Critics might say this talk is too nuanced for readers. Well, some of them want nuance, as Emily Willingham has recently
written.
As a late addition to this post, I just saw that astrophysicist Lawrence Krauss strikes an excellent, skeptical tone in this Newsweek article where he says: "That astronomers can deduce this kind of motion in a star four light years from us is so amazing, in fact, that we should be cautious about accepting the validity of the new claim, which was published in Nature last week, until it is confirmed using another independent set of observations. The observed signal is so small that it must be extracted from far bigger random jitter in the star’s light. Many of my colleagues have expressed skepticism on this point, but if we weren’t skeptical we wouldn’t be scientists." Well said! I'll repeat that this was Newsweek!
EMBARGO MISADVENTURES
There was one obvious misadventure with the publicity effort. To give some background, Nature sent their summary of the result, along with a copy of the paper and the News and Views article by Hatzes, to science writers who agreed to honor an embargo. This means the writers could talk to the authors and to experts to research their stories, but couldn't publish their articles until 1pm EDT on Wednesday, October 17th, on Nature's regular schedule. Nature normally sends their press package out the week before the paper is published, but in this case they sent their package out on Monday, October 15th. Apparently they do this for their best results. Presumably, their thinking is that the embargo only has to hold for two days.
To publicize the result, the outreach people at the European Southern Observatory (ESO) organized an embargoed press conference at 10am EDT on Tuesday, October 16th. To participate, writers had to agree to honor the embargo. As an aside, I have had one science writer recommend several times that NASA should do embargoed press conferences, to give writers more time to prepare their stories. I asked J.D. Harrington, a public affairs officer at NASA about their policy on embargoes and here is his emailed response:
Nadia Drake explained some more details to me about what happened. Someone saw the story on the Croatian website and posted it to Facebook, then someone saw that and did a blog post, and then someone saw that and tweeted about it. Once it reached Twitter it bounced around and news inevitably got back to Nature about the embargo break, as mentioned above. With the embargo lifted, stories could be published straight away, and the rush was then on to get stories out quickly. Some writers, like Drake, were anticipating an embargo break, so they got their stories finished early, just in case and then monitored sites like Twitter to see if anything slipped out. You can see that most of the articles mentioned above were published on Tuesday, October 16th.
This is a good case study in observing the big effect that social media, especially Twitter, is having in communicating science. Excitement built up, while the embargo held, and then the embargo break was quickly reported, so writers could publish their stories. One might say that the social media buzz indirectly contributed to the embargo break, but that would be speculation. Afterwards, the many stories were quickly spread around, with the best ones being deservedly celebrated. All of this happened in just a few days, like a Dan Brown novel, and it was fun to watch. However, the story of Alpha Centauri Bb is not yet over and many mysteries remain about the system.
[1.] My PhD thesis at the University of Sydney involved searching for oscillations on stars like the sun, including Alpha Centauri A & B. The technique involved radial velocity measurements just like these exoplanet hunters, so there's a lot of overlap, except I was looking for wobbles *in* the star and they are looking for wobbles *of* the star. Also, I was searching for lower amplitude variations on a much shorter timescale.
 |
| An artist's impression of the planet around Alpha Centauri B. Credit: ESO/L. Calçada/Nick Risinger (skysurvey.org) |
Did you notice how carefully I worded the description of this result? I used "may", "if confirmed" and "planet candidate". The exoplanet candidate has not been observed directly. Its existence is inferred from the tiny, rhythmic wobble apparently seen in the star Alpha Centauri B, after several, much stronger signals have been subtracted away. No other team has reported evidence for planets in the Alpha Centauri system, and if they had, this new paper wouldn't be such big news.
I've read the paper and I think there's a good chance that a planet has been found, but I think confirmation is needed (the reader might ask how I even have a clue about this: see a footnote [1] for a brief explanation). I'm not the only person who thinks this. Artie Hatzes, an astronomer from Thuringian State Observatory, in Tautenburg, Germany wrote a News and Views article for Nature where he expressed similar views. He uses words like "possible extrasolar planet", and "if it is confirmed" and uses the phrase "candidate exoplanet" four times. He also uses the "extraordinary claims require extraordinary evidence" line, from Carl Sagan and says that although a "planet-like" signal is found, the "discovery does not quite provide the "extraordinary evidence"".
Another planet expert, Debra Fischer, agrees that confirmation is needed. In a Planetary Society interview with Bruce Betts, she compliments the authors of the Nature paper for their new analysis techniques, but says: "Nevertheless, because these corrections essentially constitute a new approach, confirmation is critical."
All of the science writers on Nature's embargo list were sent a copy of the News and Views article in advance. So, most of them would have been aware of Hatzes' reservations. How much of this doubt made it into the media coverage of this paper? A good amount of it did. Dan Vergano from USA Today wrote one of the more carefully worded articles beginning with this summary:
"Our nearest star-system neighbor, Alpha Centauri, may have an Earth-sized planet orbiting one of its three stars. If confirmed, the planet orbits too close to the star for life."He then quoted Artie Hatzes' caution later in the article. Vergano later says
"Dumusque says that more looks at the star should strengthen the finding and notes that there is a possibility that the planet may slightly eclipse, or transit, in front of the star, another tell-tale sign of its existence measurable by astronomers."So, not only does Vergano mention that the finding needs to be strengthened, he discusses ways to confirm the result. Nicely done.
Nadia Drake, writing at Science News includes a good quote from Debra Fischer: "It's a tough detection, there's absolutely no question about it," Fischer says. "A detection that's as technically difficult as this one requires confirmation." Drake then segues into an excellent, detailed discussion of plans to confirm the result.
John Timmer, writing for Ars Technica, gave a good, detailed explanation of the technique used by the planet hunters, as they eliminated other signals until a "a hint of a signal with a periodicity of 3.3 days" was left. His statement that "it's probably a planet" (his emphasis) rings true.
Brian Vastag from the Washington Post quoted Artie Hatzes about the need for confirmation by other teams. Dennis Overbye, from the New York Times, wrote that astronomers were electrified by the news of the planet but they also cautioned that it needed to be confirmed by other astronomers. I was interested in seeing more here, but he doesn't give any details about the doubts or the astronomers who expressed them, because he goes on to quote Debra Fischer saying it's the story of the decade. Maybe there was an overzealous editor at work here.
Phil Plait, at his Discover Magazine blog "Bad Astronomy" wrote an extremely enthusiastic description of the result along with an impressive level of technical detail and some science fiction perspective. He doesn't provide any obvious qualifiers, but he's an experienced astronomer so this can be considered an opinion piece as well as a report. He read the paper and found it "pretty convincing".
Other writers also decided not to include qualifiers about the result, or had editors remove them. Instead of calling these out, I'll give a few possible explanations for these omissions. First, the authors addressed concerns about the significance of the result on a press conference held by ESO. The first question was from Seth Borenstein, of the Associated Press, who asked
"Can you address Hatzes' issue in his N&V that this an extraordinary claim and if you follow Sagan's rule, it requires extraordinary proof and he doesn't think this is enough".One of the authors answered by saying that they estimated there's a 1 in
Second, the paper itself does not use qualifiers. For example, the abstract includes: "Here we report the detection of an Earth-mass planet orbiting our neighbour star Alpha Centauri B". Presumably, most or all of the authors of the paper along with the two or three referees agreed that qualifiers weren't necessary. So, adopting the view that peer-reviewed papers can offer a useful approximation of The Truth, one can argue that qualifiers were not essential for a press report.
Third, there was no shortage of exoplanet experts who were very excited about the result and who weren't expressing doubts, at least in quotes made public. This includes people like Geoff Marcy, Alan Boss and Sara Seager. Science writer Ross Andersen, writing for the Atlantic, asked Sara Seager whether this is a candidate or if it has been confirmed, and she responded "To answer your second question, we consider this particular finding a detection. It's not a candidate; it doesn't need to be confirmed. It's a detection." What's going on here? It's not a shock to see scientists disagree about the credibility of a new result, especially if they have different backgrounds. In this case, Hatzes and Fischer are mainly observers, while Seager does more theory. It's possible that there are differences being used in the definition of key words like "detection" and "candidate", with the observers demanding a higher threshold for detection.
It's well known that people can have subtle biases that make it difficult to be objective, and scientists are not exempt from these problems. I won't attempt to speculate about other people's point-of-view, but I'll give you a personal example. Do I like Artie Hatzes' News and Views article because I'm convinced it's the most objective and carefully-considered reaction to this new result, or because he happened to give a very positive referee's report to an early paper of mine? I don't think it's the latter, but it's difficult to say I don't harbor *some* bias. I *will* openly admit my experience doing publicity and following others doing it is an issue for me, as I've seen results come and go or be seriously contested, some recently.
Other challenges for writers might be connected to any word limits they have. If you have described the basics of a result and only have a hundred words left, you might see a choice between describing the need for confirmation, which some might find boring, and talking about the exciting possibility of sending probes to Alpha Cen B, or even traveling there. Bloggers don't have word limits - as I'm reminding readers with this article - and so they can fit more details in. However, as Vergano's article shows, it's possible to be concise and conservative. Another issue for writers was likely a lack of time, exarcerbated by the shorter than usual embargo time given by Nature and eventual embargo break, as I describe in the next section.
The science media, especially the mainstream media, is good at describing the latest big discovery (spurred on by press releases produced by people like me, but that's a story for another blog post). However, this can give a distorted view of how science really works. As Andrew Revkin said in his very first blog post at dotearth:
"The strength of science lies in the trajectory of understanding more than in any single finding, and the most durable ideas emerge from conversation, not monologue."Although he's describing his motivation for starting a blog, his comment summarizes a larger goal of scientific research. In the case of Alpha Centauri Bb, one can ask what needs to be done to satisfy skeptics like Artie Hatzes and Debra Fischer about whether a planet has really been detected. One study doesn't do the job. The authors have already offered assistance by providing their velocity data so that people can check their analysis. In other words, different teams are being invited to check their work, and I'm very confident that multiple teams are taking up this offer. It's possible that a different team will find a weaker signal, but it's also possible that a similar strength signal or a stronger one will be seen.
I asked for comments from planet expert Dave Charbonneau who passed along, via email, what he told others:
"Yes, the analysis makes sense to me. The authors present a long list of items they need to worry about for their data, and how they correct for them. But their description of these makes sense, they have done what I would do, and (most importantly) they are releasing their data. You can bet that within hours many scientists around the world will analyze these data and attempt to confirm the signal, and we will know shortly the result. This is the strength of a really exciting field -- the correction timescale is short. But I have no worries here, although I too will be keen to analyze the data. Who wouldn't?"This is about the difference between confidence and certainty. Charbonneau is confident that the planet has been detected but he's not certain, and he wants to find out for himself, not just rely upon what others say. It's also important for completely independent work to be performed. Debra Fischer and her team are doing their own observations, as outlined here. Others may have been observing it, including some of my fellow Australians.
Does Sagan's principle of "extraordinary claims require extraordinary evidence" really apply here? A less vague way of applying that requirement might be: is an unusually high statistical significance required for a result of unusually high public interest? I would assert that it isn't. But, when there's such widespread interest I think there's a greater need to explain potential concerns with reliability and the steps required for confirmation.
Given the challenges involved with this story and the other explanations given above, I think writers did their jobs well. However, I think that serious discussion of uncertainty and the need for confirmation can make the difference between a good story and a great story. Critics might say this talk is too nuanced for readers. Well, some of them want nuance, as Emily Willingham has recently
written.
As a late addition to this post, I just saw that astrophysicist Lawrence Krauss strikes an excellent, skeptical tone in this Newsweek article where he says: "That astronomers can deduce this kind of motion in a star four light years from us is so amazing, in fact, that we should be cautious about accepting the validity of the new claim, which was published in Nature last week, until it is confirmed using another independent set of observations. The observed signal is so small that it must be extracted from far bigger random jitter in the star’s light. Many of my colleagues have expressed skepticism on this point, but if we weren’t skeptical we wouldn’t be scientists." Well said! I'll repeat that this was Newsweek!
EMBARGO MISADVENTURES
There was one obvious misadventure with the publicity effort. To give some background, Nature sent their summary of the result, along with a copy of the paper and the News and Views article by Hatzes, to science writers who agreed to honor an embargo. This means the writers could talk to the authors and to experts to research their stories, but couldn't publish their articles until 1pm EDT on Wednesday, October 17th, on Nature's regular schedule. Nature normally sends their press package out the week before the paper is published, but in this case they sent their package out on Monday, October 15th. Apparently they do this for their best results. Presumably, their thinking is that the embargo only has to hold for two days.
 |
| This photograph shows a view of the Milky Way behind the dome of the ESO 3.6-metre telescope at La Silla in Chile, where the observations of Alpha Centauri B were performed. Alpha Centauri is visible towards the upper right of this photo. Credit: S. Brunier/ESO |
To publicize the result, the outreach people at the European Southern Observatory (ESO) organized an embargoed press conference at 10am EDT on Tuesday, October 16th. To participate, writers had to agree to honor the embargo. As an aside, I have had one science writer recommend several times that NASA should do embargoed press conferences, to give writers more time to prepare their stories. I asked J.D. Harrington, a public affairs officer at NASA about their policy on embargoes and here is his emailed response:
"NASA stepped away from putting anything out under embargo almost two years ago. The primary reason was that they no longer make sense in today's 24-hour news cycle coupled with the competitive nature of the media market (always wanting to be first). We've have numerous problems supporting embargos with many being broken. As such, we no longer support them. We also pushed hard against the major journals to get them to stop, but as you know, they vehemently resist change..."Getting back to the exoplanet story, I had already noticed some buzz about this result on Twitter, with several mentions of a *big* exoplanet result coming. Not surprisingly the ESO press conference was a big hit, with over 60 science writers calling in. By the end, writers had a LOT of information about the result, including ESO's press package and substantial quotes from the researchers, plus the package from Nature. Assuming that the embargo held, they now had a bit over 24 hours to finish their stories. The embargo didn't make it. As documented by Ivan Oransky at his excellent blog Embargo Watch a writer for a Croatian news site broke the embargo with a story time-stamped at 17:25. Assuming this is Central European Summer Time, that's 11:25am EDT, so not long after the press conference likely ended. According to Embargo Watch, the news then spread around Twitter until it reached astronomy blogger Daniel Fischer, who alerted ESO and the American Astronomical Society, who both alerted Nature. Nature then decided to officially lift the embargo later on Tuesday.
Nadia Drake explained some more details to me about what happened. Someone saw the story on the Croatian website and posted it to Facebook, then someone saw that and did a blog post, and then someone saw that and tweeted about it. Once it reached Twitter it bounced around and news inevitably got back to Nature about the embargo break, as mentioned above. With the embargo lifted, stories could be published straight away, and the rush was then on to get stories out quickly. Some writers, like Drake, were anticipating an embargo break, so they got their stories finished early, just in case and then monitored sites like Twitter to see if anything slipped out. You can see that most of the articles mentioned above were published on Tuesday, October 16th.
This is a good case study in observing the big effect that social media, especially Twitter, is having in communicating science. Excitement built up, while the embargo held, and then the embargo break was quickly reported, so writers could publish their stories. One might say that the social media buzz indirectly contributed to the embargo break, but that would be speculation. Afterwards, the many stories were quickly spread around, with the best ones being deservedly celebrated. All of this happened in just a few days, like a Dan Brown novel, and it was fun to watch. However, the story of Alpha Centauri Bb is not yet over and many mysteries remain about the system.
[1.] My PhD thesis at the University of Sydney involved searching for oscillations on stars like the sun, including Alpha Centauri A & B. The technique involved radial velocity measurements just like these exoplanet hunters, so there's a lot of overlap, except I was looking for wobbles *in* the star and they are looking for wobbles *of* the star. Also, I was searching for lower amplitude variations on a much shorter timescale.
Great post Peter. It is very insightful from a scientist's perspective. I have actually been considering the scenario where a scientist has evidence of an interesting physical phenomenon, but it is not high enough significance to claim a "detection" (say >3 sigma). It is still an interesting result because the evidence tells scientists that something astonishing might be happening, and with further research it could be confirmed at a significant level.
ReplyDeleteIf done properly, the public coverage of this evidence could shed light on the scientific process. That it is an iterative process. Rarely is the first measurement significant and definitive, but the first measurement typically provides some evidence and more measurements are needed to reach a high level of significance/confidence.
Do you think that such intermediate results have a place in public coverage (assuming they have been published in a journal)? This case is a bit more extreme that the planet finding case because the results would not be claimed as a high significance detection. As you noted in your post many journalist will present the proper level of caution, however it is all too easy to misrepresent such a finding.
Thanks Will, for your comments, and I agree that this is an iterative process. I think that there is a place for discussing results that offer tantalizing hints rather than statistically significant evidence, but the results have to be presented very carefully and an even stronger emphasis should be placed on the steps necessary to increase the confidence levels. Also, the evidence might not be presented in a press release, but in some other format like a blog post. However, in cases like this I think it's important for a discussion between the scientists and publicity people about how a tantalizing result can be tested and whether it's worth waiting for these tests to be carried out *before* doing publicity, because the chances are you'll be left either with a stronger result that motivates a press release or a non-result.
DeleteEmbargos should lead to better articles, because the writers have more time to contact other experts in the field to get a second opinion. Are broken embargos really a big problem? Did the above mentioned good articles (including this one) less hits, because someone in Croatia broke the embargo?
ReplyDeleteI'm afraid that without embargos, there would be much more copied press releases, just to be quick. On the other hand, I would also read good articles about interesting stories some time after the initial press releases, but I don't know how big this 'maket' is.
Thanks Andreas, for your comment and I agree with you. I described the embargo break as a "misadventure" because it changed the timeline for reporters, but the embargo period that did remain, for about a day, still provided reporters with time to research their story by talking to experts, listening into the ESO press conference, etc. So, the embargo did its job, even if it was a limited one. I was going to discuss some of these issues, but the post was already long. I may discuss this in a follow-up post.
Delete