Guest Posted April 11, 2012 Share Posted April 11, 2012 I was just perusing Nature's "peer review" policy and was shocked to find this: We do not release reviewers' identities to authors or to other reviewers, except when reviewers specifically ask to be identified. Unless they feel strongly, however, we prefer that reviewers should remain anonymous throughout the review process and beyond. Before revealing their identities, reviewers should consider the possibility that they may be asked to comment on the criticisms of other reviewers and on further revisions of the manuscript; identified reviewers may find it more difficult to be objective in such circumstances. We ask reviewers not to identify themselves to authors without the editor's knowledge. If they wish to reveal their identities while the manuscript is under consideration, this should be done via the editor, or if this is not practicable, we ask authors to inform the editor as soon as possible after the reviewer has revealed his or her identity to the author. We deplore any attempt by authors to confront reviewers or determine their identities. Our own policy is to neither confirm nor deny any speculation about reviewers' identities, and we encourage reviewers to adopt a similar policy. http://www.nature.com/authors/policies/peer_review.html WTHeck? A scientist puts his professional reputation on the line by submitting an article for peer review. I'm sure it gets around if/when someone submits a bad article that gets savaged. Yet the people who do the savaging are not only NOT identified, but Nature prefers things that way! Nature claims that it's reviewers are of the highest caliber...but w/o knowing who those reviewers are, we have to take that "on faith". I note the presenting scientist is permitted nothing "on faith". No one outside the journal is supposed to know who reviewed or be able to tie specific comments to a specific reviewer. That's asking everyone from the submitting scientist to the general public to accept a "black box" process. We have no idea who those "reviewing peers" are, what their credentials are, or what biases they may have. For all we know, a "peer review" panel for a bf paper could consist of Greg Long, James Randi, Andrew Skolnick, and Robert Sheaffer. Yet another reason to distrust "peer review". Link to comment Share on other sites More sharing options...
Guest MikeG Posted April 11, 2012 Share Posted April 11, 2012 You could see it another way entirely. If there were no protection for the reviewer, it is possible that they might modify their criticisms if, say, the author were in a strong position in the field, for instance, or might be involved in reviewing some of the reviewer's own work at a later date (ie positions reversed), or might be a potential employer of the reviewer, or maybe was a student or teacher of the author. No, I wouldn't see this as a problem. I am curious to know whether the authorship of the papers are known to the reviewers. If so, I wonder if it might be a better idea to have the authorship revealed only after the reviewer has submitted their comments. That way any personal bias, or accusations of personal bias, could be eliminated. Rather than look up Churchill's quote on democracy, I'll paraphrase it: It may not be a great system, but it's the least bad of them all. Actually, I believe it to be rather better than that, and whilst not perfect, it has clearly served us pretty well for some time. I'm sure it will continue to evolve and improve. Mike Link to comment Share on other sites More sharing options...
Guest Transformer Posted April 11, 2012 Share Posted April 11, 2012 I think the problem may be that Mulder thinks that the "reviews" are like book reviews. The reviews are not just opinion based on the author's personal opinion of what they like or do not like, they must be scientific rebuttals of the either the data or conclusions drawn from that data in the paper. In other words the paper submitted must be able to satisfy the proper scientific protocols that all papers submitted with the same sort of data (in this case DNA). If the data supports the conclusions then "opinions" do not matter. Link to comment Share on other sites More sharing options...
Drew Posted April 11, 2012 Share Posted April 11, 2012 Mulder, do you think Grand Jury members should be Identified as well? Link to comment Share on other sites More sharing options...
southernyahoo Posted April 11, 2012 Share Posted April 11, 2012 (edited) I am curious to know whether the authorship of the papers are known to the reviewers. If so, I wonder if it might be a better idea to have the authorship revealed only after the reviewer has submitted their comments. That way any personal bias, or accusations of personal bias, could be eliminated. ^ Exactly, You can't know what the reviewers intentions are, and in this case, there is no keeping one of the authors' identity secret. It's not a perfect scenario where only the Journal editors know who everyone is. Edited April 11, 2012 by MikeG quote tags added Link to comment Share on other sites More sharing options...
Guest Transformer Posted April 11, 2012 Share Posted April 11, 2012 Once again I feel the need to point out what should be obvious. The reviewers must use arguments based on science not their own personal opinion. Either the paper's science is wrong - the reviewer's science is wrong. Link to comment Share on other sites More sharing options...
Guest BFSleuth Posted April 11, 2012 Share Posted April 11, 2012 Keeping anonymity is a way to prevent issues such as bribery or intimidation at the far end of the scale. Anonymous review is common practice and is done to protect the process. The editors are responsible for managing the reviewers and making sure everything is on the up and up. This is also a way to prevent a buddy-buddy system whereby a reviewer can give a favorable review in return for a favorable review. Remember, reviewers also submit papers for consideration. Saskeptics and Parnesus's numbers are a good case in point, they have submitted papers and also do reviews. It would be unseemly at the least to be put in a position where they would want to be lenient with someone that will later be doing a review of their own work. I understand your concern Mulder. However, the policy of anonymity is a good one and is to the great benefit of the review process. Link to comment Share on other sites More sharing options...
Guest Posted April 11, 2012 Share Posted April 11, 2012 The biggest problem with ''anonymity'' in the past has been Reviewers who also work for profit reviewing a competitors work and the ''fraud'' that also came to light later as the competition's reviewer found fault with a peer-reviewed paper as a means to stall the competition...hope that was clear. It happens, it's been documented and I have no reason to think it doesn't continue, at least till it comes to light. Science is a highly competitive field, and the players have as much ''game'' as the NFL. Link to comment Share on other sites More sharing options...
southernyahoo Posted April 11, 2012 Share Posted April 11, 2012 Once again I feel the need to point out what should be obvious. The reviewers must use arguments based on science not their own personal opinion. Either the paper's science is wrong - the reviewer's science is wrong. Who's reviewing the reviewers science? Do you think they could blow smoke at the editors? If we accept that your argument works both ways, then let us remember that if and when this paper publishes, the quality of the science would then be verified. Link to comment Share on other sites More sharing options...
Guest BFSleuth Posted April 11, 2012 Share Posted April 11, 2012 Reviewing the reviewers would be the purview of the editors. The editors are ultimately responsible for what gets published and they do take their responsibility seriously because they have the reputation of the journal itself to uphold. Link to comment Share on other sites More sharing options...
bipedalist Posted April 11, 2012 BFF Patron Share Posted April 11, 2012 (edited) I see nothing wrong with a "blind" review process of coding author and reviewer identity either one..... of course, it's not like an NIH clinical trial process or something where there is a double-blind control but it is a process that is explained up front by Nature and a rationale is provided for the reason they prefer to operate that way. That reviewer's do not know one another is not that unusual if the journal is truly going after quality control and expertise in individual sub-specialities and such. Not everyone would probably recognize each other by name anyhow under some circumstances (with the probable exception of primatology I would think). Then it is up to the author's/co-author's and associate editors/reviewers to decide whether they want to go with it or not. It is a choice to submit to certain journals and as long as author's do their "homework" they should know what to expect. Of course, if Bigfoot/Sasquatch or North American Primate were a title or subtitle or tag I'm sure cover would be blown pretty fast for the author anonymity as was mentioned. Edited April 11, 2012 by bipedalist Link to comment Share on other sites More sharing options...
southernyahoo Posted April 11, 2012 Share Posted April 11, 2012 Yeah, I'm sure they would see a disparity in comments from the reviewers if one went on a wild tear, while the others were more approving with only minor recomendations. It would be interesting to read their comments and see if they had the same cynical tone we hear around here. Link to comment Share on other sites More sharing options...
Martin Posted April 11, 2012 Share Posted April 11, 2012 The road to anonymous peer review began with a Hockey Stick. Anonymous review is ok in it's purest form. In it's worse form it can be used to freeze out contrary papers. 1 Link to comment Share on other sites More sharing options...
Guest Transformer Posted April 11, 2012 Share Posted April 11, 2012 Who's reviewing the reviewers science? Do you think they could blow smoke at the editors? If we accept that your argument works both ways, then let us remember that if and when this paper publishes, the quality of the science would then be verified. I agree with your last point for sure. If the paper is published in a reputable journal then the science and conclusions have been accepted by her peers and I will be be happy to point out to my friends that my interest in sasquatch has not been a waste of time. Link to comment Share on other sites More sharing options...
Guest Posted April 11, 2012 Share Posted April 11, 2012 If there were no protection for the reviewer, it is possible that they might modify their criticisms if, say, the author were in a strong position in the field, for instance, or might be involved in reviewing some of the reviewer's own work at a later date (ie positions reversed), or might be a potential employer of the reviewer, or maybe was a student or teacher of the author. Such conflicts of interest in any other field must be disclosed...why should they not be in this case? I understand your argument (the "Emperor's clothes" effect), but only with a transparent process can the conflicts you mention above be avoided. Put everyone "on record" and fully identified makes shennanigans of all sorts far less likely as there are no shadows to hide them in. No, I wouldn't see this as a problem. I am curious to know whether the authorship of the papers are known to the reviewers. If so, I wonder if it might be a better idea to have the authorship revealed only after the reviewer has submitted their comments. That way any personal bias, or accusations of personal bias, could be eliminated. How would that prevent topical bias, which is the more important thing.. Would you, for example, want James Randi anywhere near an ESP paper, given his documented history of not just Skepticism, but outright debunkery of the topic. I wouldn't, especially after reading about how rigged his "million dollar challenge" is/was. ( http://www.dailygrail.com/features/the-myth-of-james-randis-million-dollar-challenge ). Rather than look up Churchill's quote on democracy, I'll paraphrase it: It may not be a great system, but it's the least bad of them all. Which doesn't mean it can't be improved. Originally, democracy was only for wealthy, land-owning men... Mulder, do you think Grand Jury members should be Identified as well? They already are via the court documents when they are empaneled. Also, they must physically face the accused and/or his represenatives during hearings. We don't believe in "star chamber justice" in the US, nor should we believe in "star chamber science". Once again I feel the need to point out what should be obvious. The reviewers must use arguments based on science not their own personal opinion. Either the paper's science is wrong - the reviewer's science is wrong. Then the reviewers should have no problem putting their name on their reviews, just as the submitter must do. Link to comment Share on other sites More sharing options...
Recommended Posts