Anyone who has read some of my postings, especially those smacking the quacks, would have noticed that I emphasized frequently the call for peer-reviewed journals. It is not that I'm in love with them, it is just that from history (at least since the beginning of the 20th century), there have been NO discoveries or ideas that have made significant advancement in the body of knowledge of physics that have not appeared in peer-reviwed journals. This means that for something to make any impact on physics at all, the necessary, but not sufficient criteria is that it must first appear in a peer-reviewed journal, not on someone's webpage, or some e-print repository, or some discussion site. No matter what the claim is, if it hasn't appeared in a peer-reviewed journal, it has ZERO possibility of making ANY dent in the physics body of knowledge. There has been no exception.
The most common misconception about papers in peer-reviewed journals is that, since it has appeared in print, it must be valid. This isn't true, nor is it the aim of peer-reviewed journals. In science, and physics especially, the validity of something cannot be established immediately. Experimental discovery must be repeated, especially by independent parties, for there to be confirmation that such a discovery is real. Theoretical ideas require a slew of experimental verifications and tests for it to be confirmed as a valid description of a phenomenon. So all of these take time, and certainly several more publications. Hence, the "validity" of something isn't guaranteed just because it appeared in print.
What is more relevant though is that a paper in a peer-reviewed journal is (i) not written out of ignorance (ii) not obviously wrong (iii) full of enough information for someone else to reproduce either the experiment or the calculation (iv) written in a clear and understandable manner. This is where the referee comes in. The referee's task is to make sure at least those criteria are met (other more prestigious journals such as Nature, Science, and PRL have additional requirements).
Refereeing is done voluntarily. It is part of one's duty to the community, and referees get no compensation of any kind from the journals other than a thank you. You referee in your area of expertise, so the journals will send you papers in your area, and often, written by your competitors. I think journals like the Physical Reviews keep track of a referee's activities, how prompt he/she is at responding, how much "complaints" that referee has received, etc. etc... (these are just my guess upon my casual conversations with a few of the editors). While the editors know who is refereeing what, the referees themselves are anonymous to the authors of the paper. So for better or for worse, the referees are free to criticize the paper being considered without revealing their identities (the editors do sometime intervene if referees get out of hand with their reviews, but I do not know how often this occurs. It certainly hasn't occured to me).
To be considered as a referee for a journal, one has to demonstrate one's expertise in a specific area of physics. Most often, it is by publishing in that particular journal. That's the most direct way of establishing one's reputation and expertise. I think that some journals such as the Phys. Review do keep track of a referee's publication track record (again, another guess).
My involvement in refereeing physics papers began when I was a postdoc. I already had published several papers in PRL, PRB, Nature, and Physica C at that time. One day, my postdoc supervisor contacted me and told me that he was going to refer the paper that he had been asked to review to me because, in his words, "you know more about this than I do". So he formally informed the editors of PRL to send the manuscripts to me to review. That's how I got onto the Physical Review referee database and I've been reviewing papers for them ever since. I also am the referee for J. Elect. Spect. because I attended a conference on Electron spectroscopy and refereed a proceeding paper that appeared in that journal.
My approach to refereeing a paper involves several steps. The first and foremost is, is the paper clear in what it is trying to convey? At this stage, I really don't care yet if it's accurate, valid, or consistent, or even possibily of any importance. I want to know if they have been clear on what they're trying to say. A confusing paper does more harm towards the advancement of knowledge than a wrong paper. I want to make sure the authors has made as clear as possible what they are trying to convey.
The second step is to see if the result is obviously wrong or impossible. I tend to referee experimental work, so the capabilities of the experimental setup is always something I pay attention to. To claim to be able to detect something beyond the resolution of a particular instrument would be a warning bell, for example. So trying to figure out if there are anything obvious would be the most logical step.
Thirdly, have the authors described in full their work to allow for someone else to reproduce? At the very least, they should include a citation of their experimental details if they have appeared elsewhere.
Fourth, are the results being presented clear and "complete". Often, a referee can make suggestions to further bolster the authors result by including either additional data, or even a new experimental measurement (this is the sign of a good referee).
Fifth, is the result consistent and/or contradictory to already-published theories and/or experiments? This is where a referee really comes in, because the referee is expected to know the state of knowledge of that particular field. This means being aware of all the possible papers in that field of study so that something that is claiming contradictory results must address all that have been published. Thus, if a paper is claiming something contradictory, it MUST address this contradiction. Why is it different? Are the previous papers wrong? Is this a different angle of attack? Is it consistent with established ideas? Is this new? etc.. etc. This usually takes the most effort out of a referee because he/she sometime may have to do some "homework" to double check facts and what have already been published or reported.
Finally, for journals such as PRL, Nature, and Science, there is an additional requirement that the paper just have wide-ranging impact beyond just that narrow field of study. While most papers in peer-reviewed journals reports on new ideas and results, these 3 journals require that the paper contains significantly important information that scientists from other areas too may find informative. So this is a tougher criteria to fulfill.
Note that for many journals, a manuscript being considered is often sent to more than 1 referee. It is not uncommon for PRL, Nature, and Science to send a manuscript to 3, even 4 referees. All of them must approve that paper for it to be published. So one can imagine how difficult it is to publish in these journals.
While science is still a human endeavor, peer-reviewed journals are still the best means of communication we have so far in advancing legitimate work.
Zz.
No comments:
Post a Comment