A welcome development.
The traditional method, in which independent experts evaluate a submission, often under a veil of anonymity, can take months, even years. Clubby exclusiveness, sloppy editing and fraud have all marred peer review on occasion. Anonymity can help prevent personal bias, but it can also make reviewers less accountable; exclusiveness can help ensure quality control but can also narrow the range of feedback and participants.
Just as “consensus” science is meaningless so is expecting good science to be subject to a “democratic process”. But when reviewers show bias (in acceptance or in rejection) or misuse and hide behind the cloak of anonymity and are not required to be accountable then Hausergate and Climategate become inevitable.
From Wikipedia: http://en.wikipedia.org/wiki/Peer_review
Richard Horton, editor of the British medical journal The Lancet, has said that
The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.
The New York Times reports (http://www.nytimes.com/2010/08/24/arts/24peer.html?_r=1):
Now some humanities scholars have begun to challenge the monopoly that peer review has on admission to career-making journals and, as a consequence, to the charmed circle of tenured academe. They argue that in an era of digital media there is a better way to assess the quality of work. Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience.
“What we’re experiencing now is the most important transformation in our reading and writing tools since the invention of movable type,” said Katherine Rowe, a Renaissance specialist and media historian at Bryn Mawr College. “The way scholarly exchange is moving is radical, and we need to think about what it means for our fields.”
That transformation was behind the recent decision by the prestigious 60-year-old Shakespeare Quarterly to embark on an uncharacteristic experiment in the forthcoming fall issue — one that will make it, Ms. Rowe says, the first traditional humanities journal to open its reviewing to the World Wide Web. Mixing traditional and new methods, the journal posted online four essays not yet accepted for publication, and a core group of experts — what Ms. Rowe called “our crowd sourcing” — were invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly’s editors, who made the final decision to include them in the printed journal, due out Sept. 17.
“Knowledge is not democratic,” said Michèle Lamont, a Harvard sociologist who analyzes peer review in her 2009 book, “How Professors Think: Inside the Curious World of Academic Judgment.” Evaluating originality and intellectual significance, she said, can be done only by those who are expert in a field.
At the same time she noted that the Web is already having an incalculable effect on academia, especially among younger professors. In her own discipline, for instance, the debates happening on the site Sociologica.mulino.it “are defined as being frontier knowledge even though they are not peer reviewed.”
The most daunting obstacle to opening up the process is that peer-review publishing is the path to a job and tenure, and no would-be professor wants to be the academic canary in the coal mine.
The k2p blog 
