In time, incorrect results get corrected. In time, bad science cannot prevail – or so the belief is. But if all the articles about fraud in funding applications, dodgy peer review, predatory journals, confirmation bias and plain fraud in science are only half true, then most of what is reported as current science is not worth the paper it (isn’t) written on. Results reported are not concerned about being correct but about getting the next tranche of funding. “Politically” correct beliefs are not challenged by younger researchers because research funding will be jeopardised if “authority” is challenged. “Peer reviews” become “pal reviews” and even “self reviews”. Journals manipulate impact factors by “pal citations”.
It should all get corrected in time, except that publication of corroborating results is discouraged as not being original while “negative results” are not considered worthy of publication. A generally accepted but incorrect hypothesis then never gets corrected until an opposing theory is “proven” with positive results.
“In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world”.
These two articles, one in The Lancet and one in SMH illustrate the point:
1. The Lancet: What is medicine’s 5 sigma?
“A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted, since the forthcoming UK election meant they were living in “purdah”—a chilling state where severe restrictions on freedom of speech are placed on anyone on the government’s payroll. Why the paranoid concern for secrecy and non-attribution? Because this symposium—on the reproducibility and reliability of biomedical research, held at the Wellcome Trust in London last week—touched on one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations.
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”. The Academy of Medical Sciences, Medical Research Council, and Biotechnology and Biological Sciences Research Council have now put their reputational weight behind an investigation into these questionable research practices. The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale. We reject important confirmations. Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication. National assessment procedures, such as the Research Excellence Framework, incentivise bad practices. And individual scientists, including their most senior leaders, do little to alter a research culture that occasionally veers close to misconduct.
Can bad scientific practices be fixed? Part of the problem is that no-one is incentivised to be right. ………
2. SMH: How Australian scientists are bending the rules to get research funding
“Science has become really opaque, especially when it comes to grant funding”, says UNSW climate researcher Ben McNeil. As a result innovation suffers, he says.
The offences in question range from junior scientists ghost-writing grant applications for senior colleagues to researchers conspiring with others to influence who might review their work.
In one extreme case a cancer scientist discovered his unfunded project idea had been stolen and used by another research group a year later.
The two major schemes that fund research in Australia – the National Health and Medical Research Council (NHMRC) and the Australian Research Council (ARC) – hand out about 1.5 billion dollars a year. The impact of these grants is almost impossible to quantify, but some have resulted in big medical discoveries such as the cervical cancer vaccine and new cancer treatments. They also generate new knowledge, jobs and industries.
While their $1.5 billion budget seems hefty, together the ARC and NHMRC reject about four out of every five ideas each year. In 2013, only 1883 ideas out 9004 received funding. The consequence of researchers’ attempts to “game” the system is that, if undetected, precious money may be allocated to unworthy research projects, potentially at the expense of the next lifesaving vaccine. ……
While the NHMRC and the ARC say they have no evidence that “gaming” is widespread, a recent survey of 200 health and medical researchers suggests this may not be the case.
Before handing out money, both bodies ask panels of anonymous experts to assess project ideas, as well as the calibre of the people who propose the idea.
When public health researcher Adrian Barnett and two colleagues surveyed researchers about whether they form alliances with others to boost their chances of a better review, they were shocked to see one in five admitted to the practice.
“I knew it was going on, but I didn’t think it’d be as high,” says Barnett, from the Queensland University of Technology.
Tags: peer review, research funding
The k2p blog 