Posts Tagged ‘peer review’

Remote Sensing: A case of editorial cowardice in the face of bullying from the orthodoxy

September 3, 2011

If this had been the middle ages there would have been witch-hunts and burnings at the stake (but of course it would then have been during the Medieval Warm Period).

Spencer and Bracewell had their paper On the Misdiagnosis of Climate Feedbacks from Variations in Earth’s Radiant Energy Balance published in the Journal Remote Sensing. The paper was refereed in the normal way and gained a lot of attention because it went against global warming orthodoxy. The global warming cabal were not amused and their blogs were full of objections to the paper. (This was all before the CERN CLOUD experiments).

The Editor-in-Chief of the Journal – a certain Wolfgang Wagner – obviously faced a lot of heat from the members of the global warming orthodoxy for publishing such heresy. He was clearly threatened by having the flow of scientific articles to his new journal throttled. But he could not retract the paper – not having any basis for doing so. Instead he has resigned in a blaze of publicity saying that the paper should not have been published!! Where peer review failed to find any fault with the paper, the editor has resorted to grandstanding to attack the paper.

It seems a simple case of the high priests of a religion threatening to excommunicate the poor little editor-in-chief of this new journal who has caved in on the basis of blog comments (and no doubt some irate telephone calls). A simple case of editorial cowardice.

The full story is detailed in these posts:

Editor-in-chief of Remote Sensing resigns over Spencer & Braswell paper

Critiques and responses 

Comment On The Resignation of Wolfgang Wagner

Polargate: When peer review is degraded to spouse-review and friend-review

August 13, 2011

An earlier post carried the story of Charles Monnett who apparently when flying over the Arctic to survey whales thought he saw 3 or 4 dead polar bears in the water. He did not get any pictures and did not retrieve any carcases but instead wrote a paper  published in Polar Biology and which was supposedly peer-reviewed. He baldly presented his observations and then speculated that the bears had probably drowned in a storm and that many more of them would drown if global warming led to the melting of Arctic ice in the summers and forced  the poor polar bears to spend more time in open water.

It was all speculation even if one supposes that he actually saw some dead polar bears. His speculations were taken as established fact and blown up by the Global Warming orthodoxy. Al Gore, the almost -President of the US and the self-proclaimed inventor of the internet, picked up the story with gusto which then  played a major part in his science fantasy movie, “An Inconvenient Truth”, which helped to win him a Nobel prize.

Panic over the dead bears and Monnett’s wild hypotheses about them helped fuel calls for declaring the bears endangered, despite all evidence that their populations have actually been increasing over the last few years.  Monnett did quite well from the work, parlaying his fame into management of a $50 million study budget, the dream of all academics. – Coyote Blog

Monnet is now being investigated by the Interior Department’s Inspector General’s office for some kind of wrong-doing associated with his award of research contracts which has also led to interrogations about his sightings and his paper and subsequent research grants. Investigators are apparently  examining Monnet’s procurement of one of those research studies on polar bears conducted by Canada’s University of Alberta, as well as the “disclosure of personal relationships and preparation of the scope of work,” according to a July 29 memo from the Interior Department’s inspector general’s office.

In particular, investigators are asking questions about the peer review on Monnett’s drowned polar bear paper, which was done by his wife, Lisa Rotterman, as well as by Andrew Derocher, the lead researcher on the  million dollar Canadian study funded by Monnet’s generosity.

Monnets co-author Jeffrey Gleason is back-pedalling and is in damage-control mode.

Although the four dead bears cited in the paper were observed from 1,500 feet during flights over the Beaufort Sea, and the carcasses were never recovered or examined, Gleason told investigators it is likely the creatures drowned in a sudden windstorm that produced 30-knot winds, not for lack of an ice pack.  
“We never mentioned global warming in the paper,” Gleason told the investigators.
 Gleason told investigators that reaction to his and Monnett’s paper was overblown and spun out of context.

Monnett is being legally defended by Public Employees for Environmental Responsibility who have also demanded an investigation of the investigation. They should perhaps pick their causes a little more carefully. Even the New York Times  weighs in but tries to trivialise the impact of the wrong-doing. Though just how they take computer model results to be a  “broad array of evidence” that “polar bear populations — and the health of the planet — will be threatened by climate change in future decades” is just a bit mysterious if not plain gullible.

A modest scientific observation about a few drowning polar bears has enmeshed a government wildlife biologist in an investigation into whether he is guilty of scientific misconduct. The investigation has taken on symbolic importance in the debate over global warming. …… Whatever the ultimate verdict on Dr. Monnett, the controversy over his observations is a minor sideshow in the global warming debate. A broad array of evidence suggests that polar bear populations — and the health of the planet — will be threatened by climate change in future decades even if not a single additional polar bear drowns while swimming far from shore.

That peer-review is often corrupted is not new but Monnet must be congratulated on getting his wife and a “friend” to be the reviewers.

But the Journal Polar Biology has been silent. How were the reviewers chosen?

Monnets original paper is here : Observations of mortality associated with extended open-water swimming  by polar bears in the Arctic Beaufort Sea 


Peer review becomes incestuous when it comes to Global Warming

February 17, 2011

The Climategate emails revealed the extent to which the global warming establishment were prepared to go to pervert and doctor the peer review process to inhibit the publication of any papers challenging global warming orthodoxy.

Fraser Nelsonthe editor of the Spectator relates the story in his blog of how a critique of faults in a paper written by one of the establishment (Eric Steig) was sent to the criticised Steig himself for peer review!! He writes:

Debunking the Antarctica myths

In January 2009, Nature magazine ran the a cover story (pictured) conveying dramatic news about Antarctica: that most of it had warmed significantly over the last half-century. For years, the data from this frozen continent – with 90 percent of the world’s ice mass – had stubbornly refused to corroborate the global warming narrative. So the study, led by Eric Steig of the University of Washington, was treated as a bit of a scoop. It reverberated around the world. Gavin Schmidt, from the RealClimate blog, declared that Antarctica had silenced the sceptics. Mission, it seemed, was accomplished: Antarctica was no longer an embarrassment to the global warming narrative.

He spoke too soon. The indefatigable Steve McIntyre started to scrutinise his followings along with Nicholas Lewis. They found several flaws: Steig et al had used too few data sequences to speak for an entire continent, and had processed the data in a very questionable way. But when they wanted to correct him, in another journal, they quickly ran into an inconvenient truth about global warming: the high priests do not like refutation. To have their critique (pdf) of Steig’s work published, they needed to assuage the many demands of an anonymous ‘Reviewer A’ – whom they later found out to be Steig himself.

Lewis and Matt Ridley have joined forces to tell the story in the cover issue of this week’s Spectator.

It’s another powerful, and depressing tale of the woeful state of climate science. Real science welcomes refutation: with global warming, it is treated as a religion.

As they say in their cover story:

“Nature’s original peer-review process had let through an obviously flawed paper, and no professional climate scientist then disputed  it – perhaps because of fear that doing so might harm their careers. As the title of Richard Bean’s new play – The Heretic – at the Royal Court hints, young scientists going into climate studies these days are a bit like young theologians in Elizabethan England. They quickly learn that funding and promotion dries up if you express heterodox views, or doubt the scripture. The scripture, in this case, being the assembled reports of the Intergovernmental Panel on Climate Change. …… So has Antarctica been warming? Mostly not – at least not measurably. Retreat of the floating Antarctic ice shelves is a favourite story for the media. But, except in a very few peripheral parts, Antarctica is far too cold to lose ice by surface melting.”

 

Drawing the line between science and faith

January 9, 2011

Steve McIntyre takes up another case of  somebody publishing a paper but refusing access to the data the paper is said to be based on at http://climateaudit.org/2011/01/06/more-data-refusal-nothing-changes/.

I have always found my simple and absolutely reliable demarcator between science and faith as being the words ” I believe that….”. The moment any statement is a matter of  “belief” rather than ” a conclusion drawn from the evidence” it becomes a matter of faith rather than of  “science”.

The moment an author cannot – for whatever reason – provide the data he has used then he is asking the reader to rely on “faith” or “trust” that the data does exist and is not faked or imagined or invented. For the reader the matter immediately descends to becoming a question of “belief” in the author if nothing else. And the author is surely not God to command an unquestioning belief.

  • Whenever an author refuses access to his data he reduces his own conclusions from being matters of science to becoming matters of “faith”.
  • When such a paper is said to be peer-reviewed then it reduces the group of peers to be little more than the acolytes to a faith.
  • When a journal publishes a paper without insisting that the data be archived and accessible then it reduces the journal in which the paper is being published to being no more than the parish magazine for a cult.

Researchers show that peer review is easily corrupted

September 18, 2010
PhysicsWorld reports on a new paper:
Peer-review in a world with rational scientists: Toward selection of the average
by Stefan Thurner and Rudolf Hanel
1Section of Science of Complex Systems, Medical University of Vienna, Spitalgasse 23, A-1090, Austria

Just a small number of bad referees can significantly undermine the ability of the peer-review system to select the best scientific papers… Scholarly peer review is the commonly accepted procedure for assessing the quality of research before it is published in academic journals. It relies on a community of experts within a narrow field of expertise to have both the knowledge and the time to provide comprehensive reviews of academic manuscripts.Stefan Thurner and Rudolf Hanel at the Medical University of Vienna created a model of a generic specialist field where referees, selected at random, can fall into one of five categories. There are the “correct” who accept the good papers and reject the bad. There are the “altruists” and the “misanthropists”, who accept or reject all papers respectively. Then there are the “rational”, who reject papers that might draw attention away from their own work. And finally, there are the “random” who are not qualified to judge the quality of a paper because of incompetence or lack of time.Within this model community, the quality of scientists is assumed to follow a Gaussian distribution where each scientist produces one new paper every two time-units, the quality reflecting an author’s ability. At every step in the model, each new paper is passed to two referees chosen at random from the community, with self-review excluded, with a reviewer being allowed to either accept or reject the paper. The paper is published if both reviewers approve the paper, and rejected if they both do not like it. If the reviewers are divided, the paper gets accepted with a probability of 0.5.

Peer review gauntlet

Thurner and Hanel find that even a small presence of rational or random referees can significantly reduce the quality of published papers. Daniel Kennefick, a cosmologist at the University of Arkansas with a special interest in sociology, believes that the study exposes the vulnerability of peer review when referees are not accountable for their decisions.

Kennefick feels that the current system also encourages scientists to publish findings that may not offer much of an advance. “Many authors are nowadays determined to achieve publication for publication’s sake, in an effort to secure an academic position and are not particularly swayed by the argument that it is in their own interests not to publish an incorrect article.”

(This could have been written about Marc Hauser — https://ktwop.wordpress.com/2010/09/17/harvard-reviews-hausers-work-but-is-the-purpose-investigation-or-vindication/)

But Tim Smith, senior publisher for New Journal of Physics feels that the study overlooks the role of journal editors. “Peer-review is certainly not flawless and alternatives to the current process will continue to be proposed. In relation to this study however, one shouldn’t ignore the role played by journal editors and Boards in accounting for potential conflicts of interest, and preserving the integrity of the referee selection and decision-making processes,” he says.

In fact Journal Editors have much to answer for in the perversion of the peer review process which was revealed by Climategate. (The Hockey Stick Illusion by Andrew Montfordreviewed here)

Thurner argues that science would benefit from the creation of a “market for scientific work”. He envisages a situation where journal editors and their “scouts” search preprint servers for the most innovative papers before approaching authors with an offer of publication. The best papers, he believes, would naturally be picked up by a number of editors leaving it up to authors to choose their journal. “Papers that no-one wants to publish remain on the server and are open to everyone – but without the ‘prestigious’ quality stamp of a journal,” Thurner explains.

When reviewers show bias (in acceptance or in rejection) or misuse and hide behind the cloak of anonymity and are not required to be accountable then Hausergate and Climategate become inevitable.

Oliver Manuel comments: The most basic problem with ANONYMOUS peer-review is this: “That methodology is flawed and those flaws have been gradually undermining, corrupting, and trivializing American science for decades.” Anonymous peer review of papers and proposals has been steadily “undermining, corrupting, and trivializing American science” since I started my research career in 1960.

The evolution of peer review with the use of open servers in now overdue but is beginning.

IPCC: Self adulation or just simple plagiarism

September 7, 2010

It would appear that large sections of the IPCC 1995 Working Group 2 report has just lifted sections from a book published by its lead author.

It could be just simple plagiarism or is perhaps the self-adulation to be expected from lead authors – or the IPCC report being used for marketing the book??

http://www.rescuepost.com/.a/6a00d8357f3f2969e2013485bc0fc9970c-250wi

The Book the IPCC Plagiarized

by Donna Laframboise.

(more…)

Peer-review evolves

August 31, 2010

A welcome development.

The traditional method, in which independent experts evaluate a submission, often under a veil of anonymity, can take months, even years. Clubby exclusiveness, sloppy editing and fraud have all marred peer review on occasion. Anonymity can help prevent personal bias, but it can also make reviewers less accountable; exclusiveness can help ensure quality control but can also narrow the range of feedback and participants.

Just as “consensus” science is meaningless so is expecting good science to be subject to a “democratic process”. But when reviewers show bias (in acceptance or in rejection) or misuse and hide behind the cloak of anonymity and are not required to be accountable then Hausergate and Climategate become inevitable.

From Wikipedia: http://en.wikipedia.org/wiki/Peer_review

Richard Horton, editor of the British medical journal The Lancet, has said that

The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.

The New York Times reports (http://www.nytimes.com/2010/08/24/arts/24peer.html?_r=1):

Now some humanities scholars have begun to challenge the monopoly that peer review has on admission to career-making journals and, as a consequence, to the charmed circle of tenured academe. They argue that in an era of digital media there is a better way to assess the quality of work. Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience.

“What we’re experiencing now is the most important transformation in our reading and writing tools since the invention of movable type,” said Katherine Rowe, a Renaissance specialist and media historian at Bryn Mawr College. “The way scholarly exchange is moving is radical, and we need to think about what it means for our fields.”

That transformation was behind the recent decision by the prestigious 60-year-old Shakespeare Quarterly to embark on an uncharacteristic experiment in the forthcoming fall issue — one that will make it, Ms. Rowe says, the first traditional humanities journal to open its reviewing to the World Wide Web. Mixing traditional and new methods, the journal posted online four essays not yet accepted for publication, and a core group of experts — what Ms. Rowe called “our crowd sourcing” — were invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly’s editors, who made the final decision to include them in the printed journal, due out Sept. 17.

“Knowledge is not democratic,” said Michèle Lamont, a Harvard sociologist who analyzes peer review in her 2009 book, “How Professors Think: Inside the Curious World of Academic Judgment.” Evaluating originality and intellectual significance, she said, can be done only by those who are expert in a field.

At the same time she noted that the Web is already having an incalculable effect on academia, especially among younger professors. In her own discipline, for instance, the debates happening on the site Sociologica.mulino.it “are defined as being frontier knowledge even though they are not peer reviewed.”

The most daunting obstacle to opening up the process is that peer-review publishing is the path to a job and tenure, and no would-be professor wants to be the academic canary in the coal mine.

Climategate and Hausergate: Different routes to the faking of science

August 30, 2010

Clearly academia can only reflect surrounding society. Scientists are not saints and political motives, financial greed and fame-seeking will be just as prevalent within academia as in the surroundings. Frauds and fakers will inevitably exist. Nevertheless it is peer review – by colleagues within the organisation and within the peer-review process – which is supposed to maintain the quality of scientific work but perhaps it must now be expanded to protect and maintain the integrity of scientific work as well. Reviewers cannot continue to use the independence of the review process as an excuse to remain cocooned within their comfort zones of anonymity. They do need to stand up and be counted.

In recent months two very different scandals in the scientific world but both relying on fake science have surfaced. In one peer-review has been lax and in the other it has been perverted to a cause.

In the case of Climategate (and the IPCC), the peer-review process was perverted to falsify scientific conclusions and suppress dissent in support of a particular political (and financial) agenda.

In the case of Hausergate predetermined conclusions were supported by falsified data which were then endorsed by the peer-review process to make non-science seem to be science. The financial motive is probably only secondary to the primary motive of seeking acclaim and reputation.

http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257

In both cases the normal sequence of the scientific method of

Question > research >hypothesis > experiment /test > analysis > conclusion

has been distorted.

In the case of Climategate a small clique of academics perverted the peer-review process to control and prevent the publication of opposing views. The IPCC (where the authors were often the same academics with a few charlatans, railway engineers, thrill seekers, politicians and financiers thrown in for good measure) not only prevented the consideration of alternate views but went further by including non-peer-reviewed advocacy reports, newspaper articles and the like when they were favourable to their cause. Of course the IPCC is a political institution so perhaps it is asking too much to expect it to be a force to maintain scientific integrity.

“Since the IPCC gives Lead Authors the sole right to determine content and accept or dismiss comments, it is more like a weblog than an academic report.”

See “Fix the IPCC process” by Ross McKitrick at http://opinion.financialpost.com/2010/08/27/fix-the-ipcc-process/

An alarmist agenda was used to satisfy the greed associated with the Carbon Offset and Trading scams. Some data manipulation is also evident from the Climategate e-mails but the control of peer-review was the main tool used.

In the case of Marc Hauser he simply fabricated data to fit the conclusions he had already come to (and it is irrelevant that his theories or conclusions may or may not be correct). It is stated that he was publishing at the rate of a paper – each one peer-reviewed – every month for 4 years. Obviously not too difficult to do or too time-consuming  if data only had to be fabricated whenever needed. What were the peers and reviewers doing? Had his colleagues and reviewers no suspicions or doubts?

What is not clear is why Hauser felt it was advantageous to fake the science instead of doing the science. Clearly he could not have been as prolific if he had to actually do the science and perhaps account for data which did not support his  theories. It would seem therefore to be connected with the gaining of an academic reputation quickly and perhaps also with the financial benefits flowing from that.

But the message coming through is that peers and peer-review must be transparent and very much more rigorous. They cannot restrict themselves to quality control alone – which itself is not applied uniformly – and not take a position on the integrity of the work. Reviewers are effectively servants of the Journals they serve and the Journals too cannot escape responsibility for what they publish and what they choose not to.

“Hausergate” and the perversion of peer review

August 14, 2010

It would seem that  Marc Hauser fudged or exaggerated or imagined or just plain made up some of the results in at least 3 papers which were published after peer review.

Predictably, Harvard is being very reticent with information but as reported by The Boston Globe the university has assured the world that all necessary corrections will be made. Harvard University confirmed yesterday that it has examined concerns about scientific work by prominent psychology professor Marc Hauser and said it has “taken steps to ensure that the scientific record is corrected’’ in three journal articles for which he was a coauthor.

Also predictably others at Harvard are rationalising and taking a sympathetic view. Greg Laden says: “I know Marc Hauser, and I trust him.”

http://homelessmanspeaks.files.wordpress.com/2008/11/circling-wagons-nov-27-2008.jpg

Hauser himself is taking a year off as penance and to purge himself of his misconduct.

1. A 2002 paper published in the journal Cognition is being retracted by Hauser and two coauthors. The retraction notes that an internal Harvard examination found that the data do not support the findings.

The journal ( or is it magazine) and Elsevier need to now defend their editorial process. Who were the peers and what did they review?

2. Also called into question by the investigation is a 2007 paper in the journal Science. Ginger Pinholster, a spokeswoman for the journal, said that one of the coauthors — Justin Wood, a former graduate student at Harvard who now is an assistant professor of psychology at the University of Southern California — wrote a letter to the journal in late June. According to Pinholster, the letter stated that an internal investigation at Harvard found there were missing field notes and that the team at Harvard had recreated its research as a result. Science has yet to make a formal change to the article.

Did the missing notes ever exist? Time for Science to open up.

3. A 2007 paper in the Proceedings of the Royal Society B has already been corrected, because of missing video records and field notes. Earlier this week, Victoria Millen, publishing editor of the British journal, confirmed that the authors contacted the journal last month and informed it of the investigation. The correction notes that incomplete video records and field notes were collected by “the researcher who performed the experiments,’’ a scientist named David Glynn, who has not responded to multiple e-mail and voicemail messages.

Hausergate and Climategate and all its associated “gates” show that the peer review process is sufficiently perverted and corrupted that it needs an overhaul. It is time for the assenting and dissenting peers to stand up and be counted and not hide behind the skirts of anonymous independance.

The Harvard statement said that in cases like Hauser’s, Harvard reports its findings to federal funding agencies, which do their own reviews.

But Harvard cannot pass the buck.

Has “peer review” failed??

August 7, 2010

An interesting, provocative and thought provoking post by Nigel Calder here.

It would seem that “peer review” which was intended to improve the quality of scientific papers has actually been a hindrance to new discoveries rather than a help. As Climategate has shown very clearly the “peer review” process is easily perverted by the ruling clique preventing anything opposing their views to be published and the prevailing group-think ensuring that anything “heretical” is suppressed.

“As there is not the slightest sign of any end to science, as a process of discovery, a moment’s reflection tells you that this means that the top experts are usually wrong. One of these days, what each of them now teaches to students and tells the public will be faulted, or be proved grossly inadequate, by a major discovery. If not, the subject must be moribund.”

Back in 1989, James Lovelock had this to say about “peer review”: ‘Before a scientist can be funded to do a research, and before he can publish the results of his work, it must be examined and approved by an anonymous group of so-called peers. This inquisition can’t hang or burn heretics yet, but it can deny them the ability to publish their research, or to receive grants to pay for it. It has the full power to destroy the career of any scientist who rebels.’

Galileo facing the inquisition

Calder continues:This month the life sciences magazine The Scientist has interesting articles on peer review.

One, entitled “Breakthroughs from the Second Tier”, describes five “high-impact” papers that should have been published in more prestigious journals than they were. You can see it here http://www.the-scientist.com/2010/8/1/30/1/.

Also in The Scientist is “I Hate Your Article” by Jef Akst, who quotes David Kaplan, professor of pathology at Case Western Reserve University:

Theoretically, peer review should “help [authors] make their manuscript better,” [Kaplan] says, but in reality, the cut-throat attitude that pervades the system results in ludicrous rejections for personal reasons—if the reviewer feels that the paper threatens his or her own research or contradicts his or her beliefs, for example—or simply for convenience, since top journals get too many submissions and it’s easier to just reject a paper than spend the time to improve it. Regardless of the motivation, the result is the same, and it’s a “problem,” Kaplan says, “that can very quickly become censorship.”

Akst’s full article is here: http://www.the-scientist.com/article/display/57601/ . It goes on to discuss some of the ideas on offer for easing the peer review problem. That’s the basis for this brief update to be added to Magic Universe.

Amid growing recognition of problems with peer review, a few scientific journals tested various remedies. As reported by The Scientist magazine, by 2010 they included ending the anonymity of reviewers, so that they could both be held responsible for their comments and be acknowledged for their work, which was time-consuming. Another policy was to insist that reviewers should concern themselves only with the rigour and proper reporting of the work, not with its impact or scope. And to speed up publication, reviewers’ comments made for one journal might be passed on to others. Some journals went so far as to publish preliminary versions of papers before the peer-review process was complete.

Peer review has to get back to being strictly a review of the quality of the work done and not a commentary on or a review of the conclusions to be drawn.