cherry picking science

http://www.smh.com.au/articles/2004/05/31/1085855500131.html

Scientists accused of cherry-picking
By Robert Matthews, in Oxford
June 1, 2004

Page Tools
Email to a friend Printer format
Scientists are routinely singling out the results of clinical trials
so they can present the findings they want, a study by academics at
Oxford University shows.

The research, which assessed the published results of more than 100
scientific trials, also found that inconvenient findings were often
not disclosed to the public. In several cases, the stated purpose of
the trial was altered as it progressed so that acceptable findings,
rather than inconvenient results, could be published.

The manipulation, which contravenes official guidelines on reporting
medical research, was uncovered by academics at Oxford University, led
by Dr An-Wen Chan, a researcher on clinical medicine. Dr Chan warned
that the findings called into question the evidence-based approach to
developing medicine, in which clinical trials are used to determine
whether to introduce new treatments.

"The reporting of trial outcomes is not only frequently incomplete but
also biased and inconsistent with protocols," the team said.
"Published articles, as well as reviews that incorporate them, may
therefore be unreliable and overestimate the benefits of an
intervention."

Advertisement
Advertisement
Suspicion about the reliability of published medical research, which
has been increasing for some time, has been prompted by concern over
the influence of drug company funding.

A recent study at the Yale School of Medicine in the US showed 80 per
cent of clinical trials backed by drug makers reported positive
findings - compared with 50 per cent of those carried out by
independent academics. Other studies have shown evidence of a bias
against unclear trial results being published in academic journals,
and of positive results being repeatedly published - giving the
impression that a drug is far more effective than it really is.

The Oxford team's findings, which are published in the latest edition
of the Journal of the American Medical Association, are based on an
assessment of the original paperwork from more than 100 trials of
medical techniques ranging from drugs to surgical methods.

In almost two-thirds of these cases, the results omitted concerns over
potential harmful effects. Crucial information was either downgraded
in importance or omitted from the published report.

When contacted by the Oxford team, almost 90 per cent of the research
teams denied they had failed to report everything, despite evidence to
the contrary.

Dr Doug Altman, a professor of statistics in medicine at the Institute
of Health Sciences, Oxford, and a member of the research team, said:
"All trials should be published honestly and transparently, and this
study shows neither is happening.

"The most worrying aspect is that over 50 per cent of the outcomes
found by the trials weren't reported, and so can't be included in the
reviews used to assess different treatments. This has serious
implications for the reliability of the recommendations made to the
National Health Service."

Leading British authorities on medical research expressed dismay at
the findings of the study.

The Association of the British Pharmaceutical Industry said it agreed
that researchers had a duty to reveal any changes made to the original
trial design, and supported full disclosure - but only when the trials
had been completed.

***

TC

.