Anonymity And "peer Review"

[Guest BFSleuth]

Once a report is published then it gives a chance for the entire science community to review the evidence or recreate the experiment. One of the tenets of science is repeatability. If one scientist does an experiment, then another scientist should be able to repeat the experiment following the procedure laid out in the published report. If other scientists are unable to repeat an experiment, or upon examining the evidence come up with a different conclusion, then it calls the report or results into question then other reports become published to debunk the first report and science progresses.

Peer review to publication is simply a way to have some kind of control in place to catch mistakes, like quality control. This doesn't guarantee accuracy, but it greatly reduces the chance of errors. Does that mean that reports still get through the process? Sure it does, it is a human system. Even manufacturing companies with excellent QA departments still ship defective products. Stuff happens.

The point is that peer review is the author's best friend, making sure the paper is presented in the best possible way and double and triple checking the science. Having double blind review to me is the best way to make sure politics is removed from the process, but I can also see why some reviewers may want their name on their review as well.

[Guest]

If anyone wants ''tenure'' they toe the party line. Better science be dammed since rocking the boat is a career killer.

I'm sorry, that's simply not true. Good journals, as a matter of policy, do not publish things unless they offer something new and different. If you want to make a real name for yourself in science, you do so by publishing something that expressly does rock that boat.

Some boats are harder to rock than others, however, so you need really solid evidence to get it moving. There is no party line to toe: there's just the weight of evidence supporting an idea and the weight of evidence challenging it. That's the issue with Clovis - it's a big boat and it's needed a lot of evidence to get it rockin'. If you've found something that will necessitate a re-writing of the textbooks, then you've gotta have some rock-solid evidence - and analysis - to justify that action. The evaluation of that evidence and analysis takes time.

This notion of science as some kind of gentleman's club from which one can be blackballed for saying or writing the wrong thing is Hollywood fantasy.

[bipedalist]

Important to note that experiments and clinical trial-like studies "in the field" can be done, but if they are retrospective and/or a posteriori..... they are not going to be seen as being tidily bound into a nice neat, bow. But basically if the Ketchum study in particular is (as described or alleged) to have put into place it's own system of repeated analyses then as long as the methodologies and standard protocol with novel or recognized primers can be defended in the case of the DNA work and the results in fact are found to derive from differing scientists in differing labs on the same samples with the same profoundly, statistically or clinically significant results indicating an unknown primate, then I think they have built into their own study a type of self-check, replication that should make the study publishable in a peer review journal (and hopefully repeatable by others on any remaining unused samples; or patially consumed ones anyway).

Does anyone know if the Ketchum study has made a determined effort to storehouse additional material which was partially consumed in the initial round of studies, for later use as "checks"?

Edited April 12, 2012 by bipedalist

[Guest Cervelo]

BFS,

Very well put and very true IMO!

[Guest FuriousGeorge]

There probably won't be bias. Science won't be the cause of failure (directly). I see this way;

If it is bf and the proof is solid, the people on peer reviewer would be ecstatic to be a part of this monumental process.

If it isn't bf........ it isn't bf..... and the blame can and will only lie in one place.

This topic reads to me as.... what if this paper sort of has a smidge of potential to be proof of bf one day and the peer reviewers squash it. Either it is or it isn't. If it isn't, I hope they squash it, then light it on fire. The "man" isn't holding us down. Lack of proof is. I won't blame the reviewer for rejecting a blobsquatch or any "evidence" that is open to interpretation. If the proof is solid and it gets rejected, just send it to the next guy and then proceed to make the first guy(s) look silly. This isn't a one shot deal. If it is solid, someone will notice. If it isn't, they won't.

Edited April 12, 2012 by FuriousGeorge

[Guest]

Some boats are harder to rock than others, however, so you need really solid evidence to get it moving. There is no party line to toe: there's just the weight of evidence supporting an idea and the weight of evidence challenging it. That's the issue with Clovis - it's a big boat and it's needed a lot of evidence to get it rockin'. If you've found something that will necessitate a re-writing of the textbooks, then you've gotta have some rock-solid evidence - and analysis - to justify that action. The evaluation of that evidence and analysis takes time.

I've always been interested in the Clovis-Pre Clovis debate, both for the actual ramifications of an established Pre-Clovis culture to the archaeological record, and for the way archaeology in general treats any 'established' Pre-Clovis evidence. Of course the timeline of NA occupation may get pushed back a little, but I find the more drastic Pre-Clovis dates more interesting...I guess for the same reasons I find the BF mystery worth following.

I'm not as familiar with the NA archaeological record as I once was (was my first degree) but I think it is still an active debate?!

Cheers

Edited April 12, 2012 by summitwalker

[Guest]

In the case of the Clovis VS. Solutrean it does on the surface appear as tho plenty of folks stuck with the ''status'' quo instead of following the evidence and it's soon to ALL be overturned. Tons of ''out of place'' artifacts were found and dismissed as irrelevant. So Mulder's statement is a fair one given that it happens more than the layperson is aware across many fields of study.

If anyone wants ''tenure'' they toe the party line. Better science be dammed since rocking the boat is a career killer. Altho I'll qualify that by saying it's the same in almost every career field.

Ask Virginia Steen-McIntyre (documenter of the 250,000 year old tool using human habitation at the Hueyatlaco site) in Mexico about what can happen to scientists who don't toe the party line.

The point is that peer review is the author's best friend, making sure the paper is presented in the best possible way and double and triple checking the science. Having double blind review to me is the best way to make sure politics is removed from the process, but I can also see why some reviewers may want their name on their review as well.

"Peer review" =/= "double blind peer review".

Why are you so supportive of "star chamber science"? If the reviewers' position is sound and science-based, then there is no harm in putting their names to it.

Only people with something to hide try to hide things.

Let the sunlight in!

This notion of science as some kind of gentleman's club from which one can be blackballed for saying or writing the wrong thing is Hollywood fantasy.

Dr Meldrum, anyone? The aforementioned Steen-McIntyre?

[Guest BFSleuth]

The Hueyatlaco site is an excellent reference point Mulder. VSM did manage to publish her work, and the debate after publication was "vigorous" and often heated. Any time a revolutionary idea is presented to the body of science it is natural that it would be attacked.

The Structure of Scientific Revolutions, by Thomas Kuhn, 1962 is an excellent work that everyone should read. It discusses the nature of the "body of science" and how it does want to protect the current paradigm. Only in the face of overwhelming evidence to the contrary will the community of scientists change to the new paradigm. This process takes place by way of scientists that try to validate an experiment or study by trying to replicate the findings of a new phenomena. Paradigm shifts are often protracted and nasty.

IMHO the world of anthropology and hominology are approaching a similar paradigm shift as DNA analysis has presented new and exciting developments especially in the last 10 years. Neanderthals mating with humans, the discovery the new species of Denisovans, and the hobbit people of Indonesia to name a few. More scientists are discussing the idea that the line of homo is going to be much more complex as we uncover the DNA trail of evolution, and I think this in itself is creating a much more open climate for the introduction of a paper such as Dr. Ketchum has submitted.

We live in very interesting times. It is exciting.

[Guest]

^OK, so let's say a reviewer ("Diane") goes public about a manuscript (written by "Carlos") she reviewed and found to be really great. Because she had nothing but praise for it, she was happy to reveal her identity to Carlos. There are at least two ways this could ultimately be damaging.

First, Carlos may have enemies. Maybe he occupies one side of a protracted scientific debate or he works in a lab that is respected by many by reviled by some too. If one of his opponents ("Erica") submits a contrasting paper to Carlos', she might already know that she doesn't want Carlos to be a reviewer. But with Diane going public, now she is another person whom Erica might name and not want to review her paper. That's a shame, because Diane might not care one wit about the feud between Erica's and Carlos' labs and she does have the expertise to give Erica's paper a great review, but once there's even the spectre of favoritism, Diane will have effectively removed herself from the potential pool of referees. That's something that could end up hampering the progression of the science.

Next, Diane's outing herself could ultimately hurt Carlos if he is up for tenure. A key element of that process is for the committee considering your body of work to query experts in your field - but with whom you have no formal association - to compose a detailed letter assessing the impact of your career. If Diane stays anonymous on her review, then she's someone who could be tapped to write a letter for Carlos. She has the right expertise, and she'd probably write a very positive letter, considering how much she liked that paper she reviewed. But if Diane outs herself as a reviewer on that seminal paper, chances are she will not be asked to review Carlos' tenure and promotion materials.

While these sorts of things clearly don't apply in every case, they do apply in many cases. Let's call them "fetched" because they're certainly not "far-fetched." As mentioned, I have colleagues who do sign their reviews and I occasionally do too, but on balance, I find it better for authors and reviewers to keep that veil of anonymity between them.

[Guest FuriousGeorge]

There was the old way to do science and there is the new and improved method.

The old way; if you were a king, philosopher, elder of the non-crazy variety and such, or any respected member of society and you made a claim, that claim was true.

The new way (in it's simplest form); basically you take the first part, only now it can be anyone, and then you have someone else verify it using proof that can be measured.

Most of the arguments (sometimes including Dr. Ketchum's paper) and especially "so and so" has hair, blood, a dehydrated superfluous nipple in a jar of formaldehyde, have to do with the old way we did science and they lack the key parts to the new way, like verification.

I like the new method.

I've said it a million times,.... it's not about the claim, it's about verification of said claim. If it gets rejected, it's doesn't have to be over right then and there. "Science" is not one small group of people in an ivory tower. There are new ones sprouting up all over the place every day. Just have one them look things over if you feel that James Randi is blindly saying "no way Jose". Someone will want in on this huge discovery. And if they can verify the proof, then "science" can't squash it no matter how bias you feel they may be.

[southernyahoo]

Does anyone know if the Ketchum study has made a determined effort to storehouse additional material which was partially consumed in the initial round of studies, for later use as "checks"?

I have enough for another round, I doubt other samples are fully consumed either.

[Guest]

The Hueyatlaco site is an excellent reference point Mulder. VSM did manage to publish her work, and the debate after publication was "vigorous" and often heated. Any time a revolutionary idea is presented to the body of science it is natural that it would be attacked. The Structure of Scientific Revolutions, by Thomas Kuhn, 1962 is an excellent work that everyone should read. It discusses the nature of the "body of science" and how it does want to protect the current paradigm. Only in the face of overwhelming evidence to the contrary will the community of scientists change to the new paradigm. This process takes place by way of scientists that try to validate an experiment or study by trying to replicate the findings of a new phenomena. Paradigm shifts are often protracted and nasty. IMHO the world of anthropology and hominology are approaching a similar paradigm shift as DNA analysis has presented new and exciting developments especially in the last 10 years. Neanderthals mating with humans, the discovery the new species of Denisovans, and the hobbit people of Indonesia to name a few. More scientists are discussing the idea that the line of homo is going to be much more complex as we uncover the DNA trail of evolution, and I think this in itself is creating a much more open climate for the introduction of a paper such as Dr. Ketchum has submitted. We live in very interesting times. It is exciting.

Your points are well taken, if off topic.

What does any of that have to do with requiring reviewers to put their necks on the block by being identified in their reviews?

You're defending the concept of "peer review"...which I am not attacking. I'm attacking "peer review" as practiced (in anonymity).

I have enough for another round, I doubt other samples are fully consumed either.

Theoretically, southern, would another round of analysis be required as part of the review process, esp if it were requested by a reviewer (To wit: "These results look good, but I won't sign off on them without a 'confirming study' "?

[Guest BFSleuth]

I was pointing out that VSM did get published. While she certainly endured "difficulties" to say the least, many scientists through history have also faced similar circumstances.

Saskeptic has fleshed out a very nice scenario in regards to anonymity and its benefits to the peer review process. I fully agree that anonymity is the best method for peer review.

Regarding whether there is enough sample material for another round of DNA analysis, I'm not as familiar with DNA research as some on this forum, but I think that when working with the DNA to prepare for sequencing they increase the quantity in order to have enough material for analysis. As such, would Dr. Ketchum then have that material stored for sharing with other scientists?