Study highlights 'lurking question' of measuring EHR effectiveness: The science in Medical Informatics is dead

The science in Medical Informatics is dead.

I'm not going to even use academic fabric softener in my assertion, e.g., "may be", "appears to be", or "is it?" (as a question) dead.

It's dead.

When HIT experts recommend changing the study goalposts when existing studies don't give results they'd like to see, rather than first and foremost critically and rigorously examining why we're seeing unexpected results, science is dead.

http://www.healthcareitnews.com/news/study-highlights-lurking-question-measuring-ehr-effectiveness

Study highlights 'lurking question' of measuring EHR effectiveness

December 22, 2010 | Molly Merrill, Associate Editor

WASHINGTON – Hospitals' use of electronic health records has had just a limited effect on improving the quality of medical care nationwide, according to a study by the nonprofit RAND Corporation.

The study, published online by the American Journal of Managed Care, is part of a growing body of evidence suggesting that new methods should be developed to measure the impact of health information technology on the quality of hospital care.

[In other words, we're not getting the results we thought and hoped we'd get with "Clinical IT 1.0", so let's alter the study methodologies and endpoints --- rather than using the results we have to identify the causes and improve the technology to see if we can do better with "Clinical IT 2.0."

Further, it's not as if there's no other data on why health IT might not work as hoped - ed.]

Most of the current knowledge about the relationship between health IT and quality comes from a few hospitals that may not be representative, such as large teaching hospitals or medical centers that were among the first to adopt electronic health records.

[This implies "other" "representative" hospitals are either not doing it right, or the technology is ill suited for them and may never work. Which is it? We really need to know before we proceed with hundreds of billions more in this "Grand Experiment" - ed.]

The RAND study is one of the first to look at a broad set of hospitals to examine the impact that adopting electronic health records has had on the quality of care.

The research included 2,021 hospitals – about half the non-federal acute care hospitals nationally. Researchers determined whether each hospital had EHRs and then examined performance across 17 measures of quality for three common illnesses – heart failure, heart attack and pneumonia. The period studied spanned from 2003 to 2007.

The number of hospitals using either a basic or advanced electronic health records rose sharply during the period, from 24 percent in 2003 to nearly 38 percent in 2006.

[How many billions of dollars diverted from patient care needs does that represent? - ed.]

Researchers found that the quality of care provided for the three illnesses generally improved among all types of hospitals studied from 2004 to 2007. The largest increase in quality was seen among patients treated for heart failure at hospitals that maintained basic electronic health records throughout the study period.

However, quality scores improved no faster at hospitals that had newly adopted a basic electronic health record than in hospitals that did not adopt the technology.

[In other words, the improvements or lack thereof had little to do with electronic vs. paper record keeping - ed.]

In addition, at hospitals with newly adopted advanced electronic health records, quality scores for heart attack and heart failure improved significantly less than at hospitals that did not have electronic health records.

[In other words, the clinical IT was probably impairing doctors compared to simpler paper methods and good HIM personnel - ed.]

EHRs had no impact on the quality of care for patients treated for pneumonia.

Researchers say the mixed results may be attributable to the complex nature of healthcare.

[That is likely true, but maybe the mixed results are also -- and even more likely in major part -- due to poorly designed and/or poorly implemented IT - ed.]

Focusing attention on adopting EHRs may divert staff from focusing on other quality improvement efforts.

[That speaks to poor EHR overkill, poor usability, unfitness for purpose, and other issues that may or may not be remediable in short or even long term - ed.]

In addition, performance on existing hospital quality measures may be reaching a ceiling where further improvements in quality are unlikely.

[That speaks to a low ROI or even negative for the hundreds of billions of dollars being diverted to the IT sector - ed.]

"The lurking question has been whether we are examining the right measures to truly test the effectiveness of health information technology," said Spencer S. Jones, the study's lead author and an information scientist at RAND. "Our existing tools are probably not the ones we need going forward to adequately track the nation's investment in health information technology."

["Probably" not the ones we need? How can the authors know this? This is not science, it is speculation. Further, I'd say the scientific imperative before we design "the right measures" to "truly" test the effectiveness of HIT is to understand why the measures we're using now are not showing the desired results, because perhaps they are perfectly adequate and are revealing crucial flaws, overestimations and false assumptions that need to be dealt with, now, not after another round of billions is spent - ed.]

New performance measures that focus on areas where EHRs are expected to improve care should be developed and tested, according to researchers.

[In pharma clinical trials, this is akin to what is known as "changing the study methodologies and endpoints" - a form of manipulating clinical research, usually with the true ultimate endpoint of money - ed.]

For example, EHRs are expected to lower the risk of adverse drug interactions, but existing quality measures do not examine the issue.

[I believe the studies that have been done of CPOE have not been consistently supportive, and in fact show CPOE might create new med errors - ed.]

"With the federal government making such a large investment in this technology, we need to develop a new set of quality measures that can be used to establish the impact of electronic health records on quality," Jones said.

[This is truly putting the cart before the horse as I wrote here. The studies showing the benefit should have long preceded the "large investments" that were decided upon - ed.]

Support for the study was provided by RAND COMPARE (Comprehensive Assessment of Reform Efforts). RAND developed COMPARE to provide objective facts and analysis to inform the dialogue about health policy options. COMPARE is funded by a consortium of individuals, corporations, corporate foundations, private foundations and health system stakeholders.

Other authors of the study are John L. Adams, Eric C. Schneider, Jeanne S. Ringel and Elizabeth A. McGlynn.

The overarching assumption is that the metrics are wrong, not the quality and fitness for purpose of the technology, the 'wrongness' of which is painfully obvious from the aforementioned other literature, e.g., link, link, link, and the many posts at this blog referring to other literature. Are the authors unaware, one might ask? I know they are not. (Or - blinded? That is, could there be external pressures affecting the thought processes? The arguments might not unreasonably be construed to be skewed from that perspective.)

Recognizing the atrocious user experience and mission hostile nature of the technology (link), how disruptive it is (link, link), how poorly implemented it is often by domain amateurs to support financial battles of the payers, not cognitive processes of clinicians, I am amazed there's any signs of improvement at all, not outright deterioration. (That there is not outright deterioration of care displays if anything the results of the hard mental labor and ingenuity of clinicians to work around the technology's deficits.)

I note that the last time RAND looked at such matters there were problems with pro-health IT bias, among other issues (see my Feb. 2009 post "Heartland Institute Research & Commentary: Health Information Technology").

Carl Sagan wrote that science is a candle in the dark in a demon haunted world.

It seems the demons are winning.

-- SS

"Trouble Coming Every Day" as Discussed by our Fellow Health Care Skeptics

With apologies to the late Frank Zappa... even though we are going through the dog days of summer, the parade of health care troubles in the news is never ending, so I thought I would recap some of the more interesting issues discussed by some of my fellow health care skeptic bloggers.

We have discussed the ongoing decline of primary care. On DB's Medical Rants, Dr Robert Centor takes on the topic: "The system has, without consciously meaning to, held primary care in contempt." The result is a "quiet rebellion: of primary doctors.

We have discussed whether the currently fashionable idea of "accountable care organizations" (ACOs) might turn out to be a cover for health care oligopolies. See what Paul Levy, CEO of the Beth Israel/ Deaconess Medical Center said about them in his blog, Running a Hospital.

We frequently talk about the manipulation or suppression of clinical research studies by those with vested interests in the results pointing in a particular direction. On the Hooked: Ethics, Medicine and Pharma blog, Dr Howard Brody wondered why the FDA will not even reveal the identities of the clinical researchers who did the studies on which it based its decision to approve the Infuse bone growth enhancement device.

Prof Carl Elliott has done an outstanding job investigating commercially funded and implemented clinical research. Dr Brody reviewed his new book, White Coat, Black Hat.  In the Carlat Psychiatry Blog, Dr Daniel Carlat reviewed Prof Elliott's new article in Mother Jones on how a pharmaceutical company sponsored drug trial went badly awry.

We asked whether the former CEO of a for-profit hospital chain who resigned after the company paid a > $1.7 billion penalty to settle charges of fraud would be a proper candidate to be governor of Florida. After Rick Scott won the Florida Republican primary, Maggie Maher, writing in the Health Beat blog, took on Scott's dubious past as prologue to a worrisome future.

Well I'm about to get sick
From watchin' my TV
Been checkin' out the news
Until my eyeballs fail to see
I mean to say that every day
Is just another rotten mess
And when it's gonna change, my friend
Is anybody's guess

So I'm watchin' and I'm waitin'
Hopin' for the best
Even think I'll go to prayin'
Every time I hear 'em sayin'
That there's no way to delay
That trouble comin' every day
No way to delay
That trouble comin' every day
- Frank Zappa

The Avandia Spin Cycle Continues Even After the FDA Safety Hearings

We have posted multiple times about Avandia (rosiglitazone), GlaxoSmithKline's star-crossed glucose-lowering drug.  While Avandia has received considerable media coverage, we focused on two questions: 1 - what are the benefits and harms of rosiglitazone as a treatment of type 2 diabetes, and therefore for which patients under what circumstances should this drug be used? 2 - what barriers have prevented physicians and patients from getting the best possible answer to the first question, and what can be done about them?  (See recent post here.) 

In particular, the Avandia case has illustrated how those with vested interests in the success of a health care product have done their best to obscure information that might threaten its success, even when doing so obscures the information that physicians and patients need to make the best possible decisions.  At one point, (in 2007, no less) we called this the "Avandia spin cycle."

Avandia once again has been in the news after the US Food and Drug Administration's hearings on the safety of the drug.  These hearings were so well covered in the media that a lengthy summary would be superfluous.  However, their main points demonstrated the persistence of the Avandia spin cycle:
- An FDA reviewer felt that "the company's misreadings of the ... Record trial ... were so profound, he concluded, that they 'suggest serious flaws with trial conduct.'" (per Gardiner Harris writing in the New York Times)
-A former FDA reviewer "withheld from regulators a study showing its Avandia diabetes drug may cause heart attacks." (per Bloomberg News)
- GlaxoSmithKline's forerunner SmithKline Beecham "secretly began a study to find out if its diabetes medicine, Avandia, was safer for the heart than a competing pill, Actos, made by Takeda."  However, "the study also provided clear signs that it [Avandia] was riskier to the heart.  But instead of publishing the results, the company spent the next 11 years trying to cover them up." (per Gardiner Harris writing in the New York Times.
- "Government experts and a panel of medical advisers repeatedly voiced skepticism on Tuesday about the trustworthiness of GlaxoSmithKline, which makes the controversial diabetes drug Avandia." (per Gardiner Harris again writing in the New York Times.)

So the Avandia saga has brought to the front pages the concerns we have had with suppression and manipulation of clinical research, especially when pursued by health care organizations with vested interests in the results of specific research projects coming out a certain way, and how they have been enabled by those with conflicts of interest.  Doctors thus should be worried whether those of us who try to practice evidence-based medicine have been fooled into practicing pseudo-evidence-based-medicine.

Those commenting on the story focused on the need for transparency when clinical research is funded and run by the corporations whose products are being evaluated.
-  "GlaxoSmithKline, the maker of Avandia, can't be trusted to report adverse clinical results fairly.  The company must be watched like a hawk as additional trials that it sponsors go forward."  (NY Times editorial)
-  "What America should demand in return for ... [generous patent] protection is that the FDA be able to make an honest evaluation of the efficacy of drugs.  When drug companies make this impossible by suppressing test results, not only do they violate their fundamental obligation of honesty with the public, their customers and their regulator, but they also break the bargain they have struck in return for the protection of their intellectual capital."  (Former NY Attorney General Eliot Spitzer in Slate.)

Instead, as I have written before, maybe we ought to consider whether those with vested interests in drugs or devices ought to be running clinical research meant to evaluate their own products.

 Ironically, while this discussion of how the Avandia spin cycle first began to revolve were going on, others were still trying to add revolutions (per minute).  In particular, a Reuters story noted:

Three influential groups of doctors who treat diabetes urged patients not to stop taking Avandia, saying on Thursday that while news about the controversial drug may be frightening, it would be worse to suddenly stop taking it.

That is odd, given that Avandia has never been shown to improve clinical outcomes for patients with diabetes, and that there are many other drugs that control blood sugar which appear to be safer. But wait, there is more,

The Endocrine Society, American Diabetes Association and the American Association of Clinical Endocrinologists worried that patients may be afraid to take Avandia.

'Patients should continue taking all currently prescribed medications unless instructed otherwise by their health care provider,' Dr. Robert Vigersky of the Endocrine Society said in a statement.

'Stopping diabetes medications can cause significant harm and result in higher levels of blood glucose that may cause severe short term health problems and could increase the risk of diabetes-related complications in the long term.'

Would not it make more sense to advise patients still on Avandia to consult with their doctors urgently about possible alternatives?  Meanwhile, it does not seem irrational to be afraid of taking Avandia, given the increasing evidence about its harms, and increasing evidence that what we know about its harms may be an under-estimate.

So I wondered why these august medical societies seemed so unaffected about the doubts about Avandia's safety, and about the evidence offered to support its use that the latest news ought to generate. It turns out that all three of the medical societies get financial support from, -- wait for it --, GlaxoSmithKline.

The Endocrine Society lists GSK as one of its Corporate Liaison Board Members. The American Diabetes Association lists GSK as one of its Banting Circle Supporters, that is, those that give at least $1,000,000 a year. The American Association of Clinical Endocrinologists lists GSK as a member of its Corporate AACE Partnership. (I was not able to find out the total amount contributed by GSK to either the Endocrine Society or the AACE.)

So once again, the loudest voices in support of the product come from those used to, and perhaps dependent on financial support from its manufacturer. As a physician, I have been particularly disappointed that our medical societies, whose missions are ostensibly to support our professional values, seem to act more and more like marketers for the companies whose contributions, rather than members' dues increasingly support them.

The cycle keeps spinning.

For further thoughts on the latest in the Avandia case, see this post by Howard Brody on the Hooked: Ethics, Medicine and Pharma blog, and this by Alison Bass on the Alison Bass Blog.

Bring Back the DSI*? - the Avandia Case as Spy Novel

Starting in 2007, we posted quite a bit about the "Avandia case," which centered on whether Avandia (rosiglitazone, by GlaxoSmithKline), a glucose lowering drug for type 2 diabetes, presented excess cardiovascular risks, and how evidence about these risks was handled. 

Summary

The Nissen and Wolski meta-analysis [Nissen SE, Wolski K. Effects of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 2007; 356, online here] was to be the first published article to combine data from all relevant clinical trials of rosiglitazone then available.  Although two major trials of Avandia had been published, its manufacturer, GlaxoSmithKline, had performed many other smaller trials of the drug which remained unpublished.  These results eventually appeared on a web-site run by GSK. However, this web-site was relatively obscure.  It had not been created voluntarily, but in response to a settlement of legal action that alleged GSK had suppressed clinical research about its antidepresant paroxetine (Paxil). (See Steinbrook R. Registration of clinical trials - voluntary of mandatory. N Engl J Med 2004; 351: 1820-1822, link here and our post here).

Nissen and Wolski found the site, compiled the results of trials on Avandia it contained, and combined their results with those of the few published trials in their meta-analysis. It is to the credit of Nissen and Wolski that they were able to figure out how to do this. It is not to the credit of GSK that they sat on the data from these trials, only put it on this web-site when compelled to do so, did not make any effort to publicize the web-site, and did not publish a meta-analysis done by company scientists that showed qualitatively similar results to that eventually done by Nissen and Wolski (see post here).
 
I originally thought the case raised two questions:  1 - what are the benefits and harms of rosiglitazone as a treatment of Type 2 diabetes, and therefore for which patients under what circumstances should this drug be used?  2 - what barriers have prevented physicians and patients from getting the best possible answer to the first question, and what can be done about them?
 
But the immediate response to the Nissen and Wolski meta-analysis was a spin cycle that seemed to obfuscate these questions, and the answers to him (see posts here, here, here, and here)
 
A Reconsideration
 
In the last few weeks, several articles about Avandia have appeared in the media, and in medical journals, that reconsider the issues, and in particular, the "Avandia Spin Cycle."  First was a commentary in the European Heart Journal by Dr Steven E Nissen, the first author of the meta-analysis [Nissen SE. The rise and fall of rosiglitazone. Eur Heart J 2010: published online here.]  Dr Nissen provided a narrative history of the Avandia case from his viewpoint, and summarized it thus:

What were the key mistakes and lessons learned from the rosiglitazone affair?

1. The FDA rushed to approve rosiglitazone because of hepatotoxicity concerns about troglitazone, resulting in failure to consider the ‘signals’ suggesting cardiovascular toxicity.
2. An early critic of rosiglitazone was intimidated by company representatives and effectively silenced, a process antithetical to the principals of open scientific discourse.
3. Although early warnings were issued for the risk of heart failure, these warnings went largely unheeded in the face of aggressive marketing and promotion suggesting cardiovascular benefits.
4. No well-designed cardiovascular outcome trials were ever conducted for rosiglitazone, despite evidence suggesting increased cardiovascular risks. The only cardiovascular outcome trial was an open label study driven by a soft endpoint (hospitalization) with low adherence to randomized medications, seriously underpowered, and not completed until 10 years following launch.
5. Although both the FDA and the company were aware of evidence of an increased risk of adverse cardiovascular outcomes, certainly by 2005, neither warned physicians nor the public.
6. When a meta-analysis of rosiglitazone was eventually submitted for publication, the company subverted the editorial review process by stealing a copy of the manuscript and used this advance knowledge inappropriately to unblind an ongoing randomized trial.
7. Approval of diabetes drugs based exclusively upon their glycaemic effects has been short-sighted and scientifically unwise. Drugs that lower blood sugar may have other adverse effects that overcome any inherent benefits.
8. The failure of  50 other PPARs, many for adverse cardiovascular effects, went largely unreported because of negative publication bias. Such knowledge might have warned the medical community about the potential risk of these agents.

Meanwhile, the New York Times published conclusions of some internal US Food and Drug Administration (FDA) reports, and of an investigation by the US Senate Finance Committee. FDA safety officials suggested that Avandia should be taken off the market. Some key points from the Senate investigation were:

The bipartisan multiyear Senate investigation — whose results are expected to be released publicly on Monday but which were also obtained by The Times — sharply criticizes GlaxoSmithKline, saying it failed to warn patients years earlier that Avandia was potentially deadly.

'Instead, G.S.K. executives attempted to intimidate independent physicians, focused on strategies to minimize or misrepresent findings that Avandia may increase cardiovascular risk, and sought ways to downplay findings that a competing drug might reduce cardiovascular risk,' ....

In combination, the Nissen commentary, and the Senate report nicely summarized what I called the "Avandia spin cycle" above.  It appears that research was suppressed, marketing was deceptive, and critics and whistleblowers were intimidated. 

Finally, Dr Harlan Krumholz published a commentary about the thiazolidinedione drugs (a group that includes rosiglirazone) in Circulation Cardiovascular Quality and Outcomes [Krumholz HM. A perspective on the American Heart Association Presidential Commission Advisory on thiazolidenedione drugs. Circ Cardiovasc Qual Outcomes 2010; 3 available online, link here], and an op-ed in Forbes. The latter sums it all up nicely:

I want to believe in America's pharmaceutical companies. I want to believe that people in these companies believe that the best strategy for success is to do what is best for patients. I want to believe that they are interested in scientific truth and eager to know of any safety issues and ready to share that information with the public.

This week I was disappointed again.

Over the years GlaxoSmithKline ( GSK - news - people ) has repeatedly reassured the public about the safety of its blockbuster diabetes drug Avandia. But this weekend the Senate Finance Committee released a report revealing that inside the company Glaxo's own experts and advisors were raising concerns about whether the drug could cause heart problems all along.

The report, based on more than 250,000 internal documents, provides a rare and unsettling glimpse into the decision by company executives to deflect safety issues--even as their own experts agreed with conclusions of outside researchers who were warning the public about possible harms.

The documents reveal that company researchers were deeply concerned about the cardiovascular safety of the drug as far back as 2003. The pages of the Senate report read like a spy novel: Glaxo receiving confidential documents leaked by a sympathetic academic who consulted for the company; the company embarking on a campaign to intimidate critics who warned about potential safety issues with the drug; and executives pulling strings to release data early from a scientific study that was supposedly controlled by an 'independent' committee of researchers.

So,

The story here is less about the drug--the Senate report breaks no new ground about Avandia's safety issues (even among experts there remains some controversy)--and more about the ethical behavior of a company. What is clear: Glaxo failed to disclose its own concerns even as it sought to discredit outside researchers who were raising questions about the drug.


This type of behavior is eroding the public trust in the pharmaceutical industry.

Policy Implications

Finally, Dr Krumholz concludes with some solutions with which I heartily agree:

The fix is simple: Once a drug is approved, all data relevant to drug safety should be placed in the public domain and independent investigators across the country should be able to use it. There should be big financial penalties for withholding relevant information. Drug studies sponsored by industry must be truly independent--outside of company control. Companies should give outside investigators independence over every aspect of the study. There are too many examples of companies wresting control of clinical studies from their consultant investigators for reasons that seem more related to product promotion than clinical science.

And on all sides there should be a commitment to protect against the intimidation of academics who are willing to raise questions about the safety and effectiveness of company products. The free flow of information about the effects of drugs and medical devices will best serve the public's interest.

Here in the US, the debate over health care reform has reached a new phase. Today the President is hosting an attempt to get bipartisan discussion of reform going again. Now that there is a new opportunity for discussion, I submit that would be health care reformers ought to think about the underlying causes of our continuing problems with rising costs, declining access, stagnant quality, and demoralized health care professionals.

One set of causes that is rarely discussed has to do with the distortion of clinical research by commercial research sponsors with vested interests in the results turning out in favor of their products, and the further distortion of clinical discourse and decision making by the deceptive marketing practices of these same organizations. If we started introducing steps like those suggested by Dr Krumholz into health care reform, maybe we really could decrease costs, increase access, improve quality, and renew professionalism.

*Bring Back the DSI?
To lighten things up a bit, and to explain the title...

When I was in school in the 1960s, I first became a fan of the Hardy Boys mysteries, then the slightly more mature Tod Moran series.  While on the Neiuw Amsterdam (the 1938 version) one day out of Kingston, Jamaica, I viewed Dr No, the first James Bond movie, a sophisticated spy thriller, not a spoof like some later entries in the series.  I later read all the classics, like those by Agatha Christie, Earl Stanely Gardner, Rex Stout, Ellery Queen, etc.  So it should be no surprise that I, like many other schoolboys and girls, collaborated to form an amateur detective and intelligence agency.  But this was the sophisticated 1960s, so we called it the Department of Scientific Investigation (DSI).  We skulked around New York City and its suburbs looking for spies, saboteurs, and common criminals.  Luckily, we never really found any.  As more realities intruded, I gave up any ideas about becoming an investigator or intelligence agent (although I still read mysteries, spy novels, and thrillers to this day, but mainly on airplanes.)  Once I became a physicians, those days in the DSI seemed very remote (although the title does now seem like it ought to be part of the NIH ;-) ).

However, after reading Dr Krumholz's article above, maybe we need to resurrect the DSI.  At least, maybe we health professionals ought to consider how we can become better and more organized watchdogs for the kind of problems revealed in the Avandia case, and in many other cases discussed on Health Care Renewal.

But if any of my old DSI colleagues want to set up a reunion, it would be fine with me. 

ADDENDUM (25 February, 2010) - see also comments by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma blog.

Deja Vu All Over Again - Sheffield Researcher Under Threat for Trying to Present Data that Offends Research Sponsor

It's deja vu all over again.  A case reported (so far only) in the UK Times Higher Education Supplement of a biomedical researcher apparently threatened because she tried to present data that did favor a particular commercial health care product. Here is the summary:

An academic has risked the wrath of her university by submitting results to a forthcoming conference without permission.

The University of Sheffield has claimed that the submission has been made in breach of a contract it has with a pharmaceutical company, which funds work in the scholar's field.

Guirong Jiang, a research radiologist who has worked at Sheffield for 13 years, is due to face a disciplinary hearing over her actions this week.

Her findings - submitted to a symposium of the European Calcified Tissue Society (ECTS), to be held in Glasgow in June - add to the debate over what some have claimed is a distortion in the field of osteoporosis caused by the over-diagnosis of vertebral fractures.

Here are more particulars:

Sheffield has censured her for making the submission without the consent of her supervisor, Richard Eastell, head of Sheffield's Academic Unit of Bone Metabolism.

It said her actions breached the terms of a 2007 contract the unit has with pharmaceutical manufacturer Sanofi-Aventis to conduct studies relating to the osteoporosis treatment risedronate, which is sold as the drug Actonel.

It also said Dr Jiang failed to follow 'reasonable requests' to withdraw the submission.

Dr Jiang said she believed her results should be published as they had not been reflected in the unit's previous output.

She added that last December she was informed that her contract would not be renewed when it came to an end this March, which she said had prompted her to throw caution to the wind and publish without permission.

She has queried whether her work is bound by Sheffield's Sanofi-Aventis contract, which stipulates that the company must be allowed to review manuscripts and abstracts prior to publication.

She pointed out that the work was carried out in 2002 when the unit's risedronate work was funded by Procter & Gamble in partnership with Aventis. Dr Jiang added that she had not seen or signed the full Sanofi-Aventis contract.

Dr Jiang is still under threat:

The disciplinary hearing, scheduled to take place on 18 February, will consider the allegation that Dr Jiang 'acted inappropriately in making a direct submission of an abstract to a journal outside unit protocol and in contravention of the terms of the research contract'.

It will also consider the charge that she 'failed to follow a request by her head of unit and head of department to rectify her actions', which 'aggravated a situation which otherwise could have been quickly resolved'.

A long time ago, in a galaxy far, far away, academics had the expectation that they could publish or present their research without first acquiring the express approval of their academic superiors (at least as long as their work was original, the research project was under their control, and that they had properly protected the rights of any human subjects). Furthermore, in at least the US, academics are citizens who expect to have free speech.

But as academic institutions became more enamored of and dependent on "external funding," faculty were increasingly pressured to do only sponsored research. The growing dependence on sponsored research allowed the sponsors to try to get more control over how research was done.

In the 1990s, there were several important cases in North America in which academic researchers tried to present or publish results that clashed with their research sponsors' vested interests. Doing so resulted in lesser or greater threats to the faculty. Two of the classic cases of research suppression in the 1990's demonstrated this issue.


* In the "David Kern case," a textile manufacturer, Microfibres Inc., tried to prevent Dr. David Kern, a general internist and occupational medicine physicians at Memorial Hospital of Rhode Island and faculty member at Brown University, from presenting an abstract that described a case-series of a new pulmonary disease, flock-workers lung, that affected workers at Microfibres Inc. factories. Kern did present the abstract, but under a threat, never carried out, of law-suit, Memorial Hospital of Rhode Island removed Kern as head of the Occupational Medicine program, and refused to renew his contract, even though he was an Associate Professor. Brown University was unable to reverse these moves by the hospital, and Brown officials blamed Kern for signing an agreement to protect trade secrets, even though the agreement was unrelated to the research Kern did on the disease, and his research did not obviously reveal any trade secrets. [1-4]

* In the "Nancy Olivieri case," Apotex, a pharmaceutical company, acted against Dr. Nancy Olivieri for revealing preliminary data from a trial of deferiprone, a chelating agent for the treatment of iron overload in thalassemia, suggesting that the drug was often ineffective in treating iron overload, and appeared to be associated with hepatic fibrosis. Ultimately, a report by the Canadian Association of University Teachers also held that her academic freedom was abridged, in the context of a negotiation between the University of Toronto and Apotex over a large donation, and that the hospital harassed Dr. Olivieri during her dispute with Apotex (link here for report)
 
In the early 21st century, there was a case in the UK with spooky similarites to the present case: at the same university involved in the present case, and with at least one participant in common with that case.  About a year after we started Health Care Renewal, in late 2005, we wrote multiple posts about the complex and unfortunate case of Dr Aubrey Blumsohn's attempts to keep a research project honest. Our most recent summary of the case was here. Dr Blumsohn was also doing clinical research on the drug Actonel, again at the University of Sheffield.  He was denied access to the very data he had collected, and to analyses of the data by Procter and Gamble, the then manufacturer of the drug and sponsor of the research.  Dr Blumsohn found that Procter and Gamble seemed to be arranging for ghost-writers to author abstracts about the research based on these hidden analyses.  After he complained to numerous officials at Sheffield, including Dr Richard Eastell, to no avail, he spoke to the press, and thereafter lost his academic position.  The case was just recapped in an interview of Dr Blumsohn published in the British Medical Journal(5).
 
So we seem to have made little progress.  Most clinical research at academic medical institutions is sponsored by firms that make drugs and devices.  The firms try to secure contracts that give them control over most aspects of the research.  Thus, they can determine how the research is designed, implemented, ana analyzed.  If the resulting manipulation still does not yield results that make the firms' products look good, the sponsors can then just suppress them.  Academics who naively think it is their duty to find and disseminate the truth are in jeopardy when they attempt to present or publish results that challenge their research sponsors' vested interests.
 
Suppression of clinical research, however, is bad for patients and honest health care professionals, since it misleads about the benefits and harms of tests and treatments.  Suppression of clinical research dishonors research subjects who volunteer for studies, often at risk to themselves, thinking they may help to advance science and clinical care.  Suppression of clinical research undermines the fundamental mission of academia: to seek out and disseminate the truth.
 
Those who truly want better health care, here in the US and globally, should advocate for clinical research done without the influence of those with vested interests in how the research comes out. 
 
Thanks to an anonymous Health Care Renewal scout who located the Times article. 
 
References


1. DG Kern, RS Crausman, TH Durand et al. Flock worker's lung: chronic interstitial lung disease in the nylon flocking industry. Ann Intern Med 1998; 129: 261-272. (link here)
2. Shuchman M. Secrecy in science: the flock worker's lung investigation. Ann Intern Med 1998; 129: 341-344. (link here)
3. Marsh DJ. Intimidation of researchers by special interest groups. N Engl J Med 1997; 337: 1317-1318. (link here)
4. Shuchman M. Consequences of blowing the whistle in medical research. Ann Intern Med 2000; 132: 1013-1015. (link here)
5.  Cyer C. Aubrey Blumsohn: academic who took on industry. Brit Med J 2009; 339:b5293. (link here)

Physician Pleads Guilty to Fraud for Fabricating Celebrex (and Other) Study Data

As reported by the Springfield, MA, Republican:

A former chief of acute pain at Baystate Medical Center has agreed to plead guilty to falsifying medical research and must pay $420,000 in restitution to pharmaceutical companies, federal court records show.

Dr. Scott S. Reuben, of Longmeadow, was charged on Thursday with health care fraud. He signed a plea agreement with prosecutors a week earlier.

Reuben prompted a furor in the medical community in March, when he was accused of making up research results in at least 21 published studies and inventing patients in certain instances.

In one notable instance,

In 2005, Reuben received a $74,000 research grant from pharmaceutical giant Pfizer, agreeing to test Celebrex as a component of the multimodal therapy. He claimed to have treated 200 patients, 100 with Pfizer's product and 100 with a placebo.

'In fact, Reuben had not enrolled any patients into that study and the results reported both to Pfizer and to the Anesthesia and Analgesia Journal and in turn to the public were wholly made up by Reuben,' the charge states.

Albert said the fabrications were discovered by medical staff within the hospital during a routine review at the hospital's 'research week,' when clinicians design poster displays of their studies.

A report in the New London, CT, Day indicated the scope of Reuben's fake research:

Anesthesia & Analgesia later had to retract 10 papers authored by Reuben, and medical experts at the time said at least 21 journal articles by the anesthesiologist appeared to have been fabricated.

The agreement calls for Reuben to pay restitution of $296,557 to Pfizer and $16,000 to Wyeth Pharmaceuticals, two companies that merged last fall. Merck & Co. would receive $49,375 from the agreement, according to documents.

In addition to his fabricated Celebrex studies, Reuben had pretended to do research that backed the effectiveness of other pain drugs, including Pfizer's Bextra and Merck's Vioxx, according to prosecutors. Bextra and Vioxx later were pulled from the market because studies showed they increased the risk of death from heart attacks and strokes.

In addition, Reuben published data showing that Pfizer's antidepressant Effexor had exhibited painkilling properties.

Reuben's studies had been considered pioneering at the time they were published. His data had supported the use of two of Pfizer's major products - Celebrex and Lyrica - in combination to treat certain types of post-operative pain.

Reuben 'was a regular on the medical lecturing circuit,' according to court documents, and 'had become a well known figure in the anesthesia and pain management medical communities.'

Pfizer said previously that it had supported five of Reuben's research initiatives. Pfizer, which declined at the time to reveal how much it paid Reuben over the years to be part of its speakers' bureau, said the company played no part in the fraud.

Asked for a comment Friday, Pfizer was not able to provide a timely response.

Note that we usually do not post about stories of individual's clinical research misconduct, not because it is unimportant, but because individual misconduct may be related more to an individual's motives and character than to concentration and abuse of power in health care.

However, in this case it appears that Dr Reubens was enabled, at least in the psychological and logistical senses, by pharmaceutical companies happy to sponsor research which suggested the effectiveness of their products.  Since Dr Reubens apparently fabricated most of his data, notice that he apparently deliberately fabricated data that made Pfizer's Celebrex, Bextra, Lyrica, and Effexor, and Merck's Vioxx look good.  Perhaps there was careerism here.  Positive studies about "innovative" effective drugs are more likely to get published than negative studies or studies of old drugs.  However, we have noted before that pharmaceutical (and biotechnology and device) companies like to sponsor studies with results that make their products look good. 

One bit of good news in this story is that Reuben's colleagues at the hospital apparently were able to uncover the fabrications when they had an opportunity to critically look at data from several of his studies.

Certainly some Pfizer employees had easy access to the same data.  But perhaps they would not have thought to look at it critically as long as it was so unrelentingly positive about their company's products. 

As we have said before, health care professionals, policy makers and patients ought to be extremely skeptical of clinical studies sponsored (and often deeply influenced) by those with vested interests in the studies' turning out in certain ways.  We ought to reconsider national research policies that have turned over sponsorship of the majority of clinical research to those with such vested interests.

Hat tip to Prof Margaret Soltan on the University Diaries blog.

ADDENDUM (19 January, 2010) - see also comments on the Alison Bass Blog.

British Medical Journal Interviews Dr Aubrey Blumsohn

About a year after we started Health Care Renewal, in late 2005, we wrote multiple posts about the complex and unfortunate case of Dr Aubrey Blumsohn's attempts to keep a research project honest.  Our most recent summary of the case was here.  The case involved suppression and manipulation of research, ghost-writing, institutional conflicts of interest, and attempts to silence a whistle blower. It provides lessons about the downsides of letting commercial firms sponsor and hence control human research designed to evaluate the products or services they sell; and of academic medicine becoming dependent on research money from such firms for such research.

The case was just re-capped in some detail on the occaision of an interview of Dr Blumsohn published in the British Medical Journal, just available on-line yesterday.  [Cyer C. Aubrey Blumsohn: academic who took on industry.  Brit Med J 2009; 339:b5293.  Link here.]  Dr Blumsohn's final words in the interview were:

It’s hard to encourage anyone to speak out about poor practice in the current environment. This case sums up what has gone wrong with systems set in place to ensure safety and integrity in scientific medicine. It would help if regulators put as much effort into responding to serious critics and whistleblowers as they do producing glossy brochures and yet more guidance.

My hat is off to Dr Blumsohn for his determination to pursue this case, and to the British Medical Journal for getting it into the "main-stream" health care literature.

Hat-tip to PharmaGossip.

"Person No. 7," Also Known as the 83rd Richest Man in the World

This one nearly snuck by. 

A Settlement and Some Indictments

In May, 2009, we posted that international Swiss-based medical device manufacturer Synthes settled charges that it was paying surgeons who conducted clinical trials for the company with company stock, and in June, 2009, we posted that Synthes was indicted based on allegations that it had subjected patients to an experimental use of its Norian XR bone cement product on the spinal cord, a use not approved by the US Food and Drug Administration, and that its executives had lied to the FDA about these actions.  It was noteworthy that the indictment named but did not charge the then CEO of the company as "Person No. 7" who allegedly decided not to conduct clinical trials of Norian XR, but rather to have surgeons use it in a case series that was not identified as clinical research.  The company CEO at that time was a Mr Hansjorg Wyss, who is still chairman of the Synthes board, owner of 40% of Synthes stock, and the richest man in the Philadelphia area, worth $5.7 billion according to Forbes magazine, making him the 83rd richest man in the world in 2009, according to Forbes magazine.

Four Guilty Pleas

Here is what first escaped my attention.  In July, 2009, the Philadelphia Inquirer reported:

Two senior executives of a West Chester-based manufacturer of medical devices pleaded guilty yesterday in connection with illegal clinical trials of a bone cement on about 200 patients, three of whom died.

The company, Synthes USA Inc., did not tell the patients that they were participating in what amounted to human experimentation, according to a 97-count indictment filed June 16 by the U.S. Attorney's Office in Philadelphia.

Michael D. Huggins, 51, of West Chester, was president and chief operating officer of Synthes' spine division from late 1994 to January 2008.

John J. Walsh, 46, of Coatesville, has served as director of regulatory and clinical affairs in the same division since August 2003.

Before U.S. District Judge Laurence F. Stengel, each pleaded guilty to a single misdemeanor count of introducing adulterated medical devices into interstate commerce. They face $100,000 fines and a year in prison when they are sentenced Oct. 22.

Then, in August, 2009, Bloomberg reported:

Synthes Inc. official Richard Bohner pleaded guilty in Philadelphia to charges tied to illegal clinical trials of a bone cement that led to three deaths.

Bohner, Synthes’ vice president of operations, is the last of four executives to plead guilty to shipping misbranded Norian XR across state lines. He faces as long as one year in prison plus a $100,000 fine, according to documents made public today in federal court in Philadelphia.

Bohner and the other executives, Michael Huggins, John Walsh and Thomas Higgins, promoted the drug through the use of test markets aimed at persuading surgeons to publish the results of their surgeries, according to court documents.

Huggins and Walsh, who pleaded guilty July 20, will be sentenced on Oct. 22.

So that is four individual guilty pleas so far in this case. The pleas received only scant attention in the local media, and hence slipped by the technology I was using to scan the media at the time.

The Richest Man in Philadelphia

I got caught up when on 6 December, 2009, the Philadelphia Inquirer published a summary article focused on the "Person No. 7":

Wyss also is chairman of Synthes in West Chester, which faces 52 felony counts stemming from allegations that it illegally experimented on patients, three of whom died.

Synthes is fighting the charges, and a trial could be held next year. Four of its executives already have pleaded guilty to misdemeanor counts in the case and face sentencing hearings in January.

Federal prosecutors in Philadelphia did not name or charge Wyss, but their June indictment describes a "Person No. 7," who was a major shareholder and chief executive officer of the company when the alleged illegal conduct occurred, from 2001 through 2004. A Synthes representative confirmed that Wyss was CEO then.

In 2001, the U.S. attorney's indictment says, Person No. 7 decided the company should not pursue the costly and time-consuming clinical trials that the U.S. Food and Drug Administration demanded for the company's bone-cement product, Norian.

Instead, according to the indictment, Person No. 7 directed the company to 'get a few sites to perform 60 to 80 procedures and help them publish their clinical results' to help popularize Norian for a use not approved by the FDA.

The procedures involved injecting Norian into the spines of patients who had vertebral compression fractures, which are typically caused by osteoporosis. FDA officials had previously approved Norian for other uses, but they demanded trials for that type of surgery, fearing that the cement could perform differently in the spine.

Even after tests in pigs showed that Norian could leak from the spine and cause life-threatening blood clots, Synthes continued with surgeries on human patients, who were never told that the procedure was experimental.

Three of those patients died, but prosecutors do not know whether Norian played any role in their deaths, because the company and doctors did not immediately report all the fatalities to the FDA. The patients have not been identified.

Through a Synthes representative, Wyss initially agreed to be interviewed for this article, but later declined.

The Inquirer article seemed to try to balance the allegations about Wyss with his wealth and philanthropy:

Wyss grew up in Bern, Switzerland. His father sold mechanical calculators and liked to discuss world events, according to Harvard's Web site. Wyss trained as an engineer in Switzerland and soon began setting up Chrysler manufacturing plants worldwide. In 1965, he earned a master's degree at Harvard's business school.

'I didn't speak in class for the first five weeks,' he told a Harvard Web publication at the time of his gift. 'My classmates were all the crème de la crème, with button-down shirts I had never seen before.'

He worked for several large companies, but sold airplanes on the side. One of his buyers was a Swiss surgeon who founded Synthes. That connection eventually led Wyss to become president of the company's U.S. business in 1977.

Over time, he became a large shareholder and chief executive. He retired from that post in 2007, but he remains chairman.

Forbes Magazine in March estimated Wyss was worth $5.7 billion. His fortune has helped him pursue whatever he wants, including the rehabilitation of the Crooked Tree Golf Course in Tucson, Ariz., and the purchase of the 900-acre Halter Ranch & Vineyard in Paso Robles, Calif. The winery emphasizes organic growing methods and sells many wines, including one named Synthesis.

Wyss has given away large chunks of his fortune, mostly to organizations dedicated to his primary passions: science and the environment.

His $125 million donation to Harvard funded the Wyss Institute for Biologically Inspired Engineering....

Wyss also established two private foundations. The Wyss Foundation is dedicated to preserving land in the American West. Wyss went hiking there as a student and fell in love, according to Forbes.

The Wyss Foundation gave away about $13 million and had about $250 million in assets in 2007, according to tax filings.

Wyss also donates heavily from his personal funds. BusinessWeek estimated his total giving from all sources at $277 million from 2004 through 2008.

However, some of Wyss' philanthropy seems more designed to benefit his company:

Wyss' other foundation, the Hansjörg Wyss Foundation, had about $172 million in assets and gave away $4.9 million in 2007. It focuses mostly on education and training of surgeons, according to the AO Foundation, a Swiss research group with ties to Synthes and Wyss.

By funding research and training, Synthes has forged tight relationships with surgeons. The U.S. Attorney's Office alleges that the company paid for trips to educational seminars in San Diego and Charlotte, N.C., where surgeons learned to use Norian in the spine.

Marketing Over Research

This emphasis on education that actually seems to partake of company marketing is also displayed by Synthes itself.  Per the company's 2008 Annual Report, the company runs a "residency program,"

The Synthes Resident Program was originally released in the United States in 2007 by Synthes Trauma. It was first developed to train orthopaedic trauma surgical residents and consists of both online training modules and locally offered workshops.

In 2008, the Synthes Resident Program was expanded to a selection of European countries with modifications for the needs of specific areas and markets. Furthermore it is now also being offered by Synthes Spine and Synthes CM.

This program has achieved remarkable popularity, or should we say "market penetrance?"

The Synthes Resident Program has been well accepted in the U.S., with more than 90% of all U.S. orthopedic residents having utilized the program.

The company also offers an "Enhanced Surgeon Education Program," and support's the AO Foundation's educational activities.

On the other hand, the financial summary provided in the report suggests that research is not what the company emphasizes.  Its 2008 operating expenses included $943.3 million in sales and promotion, $349.4 million in general and administrative, and only $169.9 million in research and development, only 11.6% of total operating expenses.

And if you like the report's verbiage about education, you will also appreciate its full page section on "Integrity," which includes the header:

Corporate Citizenship. Acting with Integrity. Acting ethically and respectfully is a cornerstone of our business. At Synthes we are committed to responding to the challenges in our business by operating in accordance with the highest levels of professional and ethical standards in our industry. Simply put, we must always act with integrity.

It also boasts:

Our commitment to abide by the rules and regulations applies throughout Synthes, to all countries and to all employees, and to all our business partners.

Acting ethically is highly important for Synthes. Our actions create our
reputation. Our good reputation is an integral part of our Synthes brand,
and an essential element of our continued success.

And finally asserts:

Our effort to operate with the utmost integrity begins with the commitment of our Board of Directors and senior managers. [italics added for emphasis]  Our managers are role models for our employees, and lead our efforts to build and to promote a culture that encourages positive ethical conduct and that demonstrates commitment to compliance with the law.

Those four now admitted guilty top executives certainly were excellent role models. Even better was "Person No. 7," the current board chairman and former CEO.

Summary

So here we have all the elements: guilty pleas by top executives in connection with a human experiment whose nature was concealed from patients; allegations about but no charges against a fabulously wealthy company leader; a company that promotes what appears to be marketing in the guise of education; a tremendously profitable company which spends most of its money on marketing, not research; a company that boasts of its integrity and how its leaders are "role models," even as some have pleaded guilty to crimes; and despite the worldwide reach of the company and its products, no national coverage of any of this, the anechoic effect redux.

We have, not only in the US, but in other developed countries (for example, Switzerland, in which Synthes is based), and globally, health care based on marketing and hype, on deception and conflicts of interest, and on making a few privileged insiders fabulously wealthy.  Here is the essence of our health care dysfunction, and the inflating health care bubble.  When would be health care reformer stop arguing about insurance coverage, perhapds they can then pay some attention to these major reasons why health care is so expensive, inaccessible, and often does so little good for patients. 

PS - Earlier this year, we commented on the make-up of the Board of Trustees of the prestigious Hospital for Special Surgery in New York, which seemed split mainly among leaders of finance, including such troubled corporations that contributed to the global financial meltdown as Bank of America, AIG, Citigroup, and Wachovia, and orthopedic surgeons with ties to device makers.  One of the latter was Dr David Helfet, who listed his memberships on advisory boards for OHK Medical Devices Inc, Healthpoint Capital, and Orthobond Corp.  Just to add to the whiffs of lack of full disclosure surrounding this case, Dr Helfet still does not disclose on the hospital web-site that he is on the Board of Directors of Synthes.

Note (added 14 December, 2009) - Per comment below, the Vice President of Corporate Compliance and Internal Audit at the Hospital for Special Surgery claimed that an administrative error, not failure of disclosure by Dr Helfet, lead to the omission of the information about his membership on the Synthes board from the Hospital's web-site.

How Industry Views the Research It Sponsors

We have posted frequently about threats to the integrity of the clinical evidence-based, and to the practice of evidence-based medicine.  In particular, we have discussed how research may be manipulated in favor of vested interests, or suppressed when the results do not favor such interests.

Last week, the British Medical Journal electronically published a set of guidelines for how industry sponsored clinical research ought to be published, sponsored by the International Society for Medical Publication Professionals.  The authors came from pharmaceutical companies (Johnson & Johnson, AstraZeneca, Pfizer and Cephalon), medical device companies (LifeScan), and medical publishing and medical education and communication companies (John Wiley & Sons, Excerpta Medica, Field Advantage Medical Communications, PharmaWrite, and Knowledgepoint 360 Group).  [Graf C, Battisti WP, Bruce-Winkler V et al. Good publication practice for communicating company sponsored medical research: the GPP2 guidelines.  Brit Med J 2009; 339:b4430.  Link here.]

These guidelines are remarkable for the questions they raise about how people from industry view clinical research and how it should be reported in medical journals. 

Who's in Charge?

Nowhere does the article acknowledge that any one person has overall responsibility for a research project.  Clinical research projects funded by the US National Institutes of Health, Agency for Healthcare Quality and Research, and other federal agencies must have "principal investigators," who are the people who take overall scientific responsibility for the project.  The Graf et al article does not use this or an equivalent term.   There is no sense that they expect anyone to be in completely in charge of industry sponsored research. 

Particularly confusing is the following passage:

Before writing begins one author (a lead author, who may also be guarantor) should take the lead for writing and managing each publication or presentation. One author (identified as guarantor) should take overall responsibility for the integrity of a study and its report.

Here are some questions it raises.  If the guarantor could be chosen only when writing a paper is contemplated, which presumably could be years after the study that the paper would report was designed and implemented, who then would have been responsible for study "integrity" before the guarantor was chosen? Who would chose the guarantor for a particular paper? If a study generates more than one paper, could they each have a guarantor, and then how could they share responsibility for study "integrity?"  If the guarantor and lead author of a paper are different people, how would they share responsibilities, what would happen if they were not to agree, and who would be finally accountable?

So the guidelines seem to completely diffuse accountability for research projects, and the reports written about them.

Who is an Author?

In my experience with US federally and foundation funded research, research papers are written by the investigators, the people who actually did the research project.  However, in the guidelines by Graf et al, the concept of authorship is also ambiguous.  They suggest that "recognised criteria should be used to determine which of the contributors to an article should be identified as authors."  This is already confusing, since how could one be a "contributor" to an article without authoring it?  A later discussion of "contributors" as "investigators, sponsor employees, and individuals contracted by the sponsor" was not very helpful.  Why should a "sponsor employee" who was not an "investigator" be a "contributor" or an "author," whatever the distinction is between them? 

So the guidelines blur distinctions among people who do research, and people employed by companies that may have vested interests in the research favoring their products or services.

What do Professional Medical Writers Do?

There has been considerable recent controversy, not directly acknowledged by Graf et al, about the role of professional medical writers in the reporting of research and the writing of ostensibly scholarly medical publications, particularly in cases where the writers were paid by and reported to corporate sponsors, but were not recognized as such in the publications they wrote (a type of ghost-writing).  The guidelines by Graf et al do not clearly explain what the roles of professional medical writers ought to be:

Professional medical writers must be directed by the lead author from the earliest possible stage (for example, when the outline is written), and all authors must be aware of the medical writer’s involvement. The medical writer should remain in frequent contact with the authors throughout development of the article or presentation. The authors must critically review and comment on the outline and drafts, approve the final version of the article or presentation before it is submitted to the journal or congress, approve changes made during the peer review process, and approve the final version before it is published or accepted for presentation.

Note that this would not prevent a professional medical writer from writing an initial outline, the first draft, and all subsequent drafts. including the final draft of a paper. (The role of an "author" above might be restricted to simply commenting on and accepting the outline and all drafts.) Thus, the "authors" could function as distant editors, and the professional writer would assume authorship, as most people would define it. (Merriam-Webster: "1. one that originates or creates 2. the writer of a literary work.")

Furthermore, while the professional writer could take one the role of authorship, the guidelines do not require him or her to publicly acknowledge this role:

Professional medical writers, depending on the contributions they make, may qualify for authorship. For example, if a medical writer contributed extensive literature searches and summarised the literature discovered, and by doing so helped define the scope of a review article, and if he or she is willing to 'take public responsibility for relevant portions of the content' then he or she may be in a position to meet the remaining ICMJE criteria for authorship.

Presumably, a professional writer could dodge authorship by simply being "unwilling" to take such public responsibility.

So the guidelines apparently condone nearly all functions commonly assumed to be those of an author to be performed by a professional writer paid directly by the sponsor, without the writer being listed as an author.  The guidelines thus appear to condone ghost-writing in its most pernicious form.

Who Owns and Analyzes the Data?

Cases in which various the implementation and analysis of clinical research seems to have been manipulated to favor vested interests have raised concerns about the integrity of the data collected in the course of a research project, and how it is analyzed.  This is what Graf et al say about the ownership and use of the data:

Sponsors have a responsibility to share the data and the analyses with the investigators who participated in the study. Sponsors must provide authors and other contributors (for example, members of a publication steering committee or professional medical writers) with full access to study data and should do so before the manuscript writing process begins or before the first external presentation of the data. Information provided to the authors should include study protocols, statistical analysis plans, statistical reports, data tables, clinical study reports, and results intended for posting on clinical trial results websites. Sufficient time should be allowed for authors and contributors to review and interpret the data provided and to seek further information if they wish (for example, access to raw data tables or the study database).

The guidelines by Graf et al suggest that the company that sponsors the research should own the data. The investigators who collected the data and implemented the research project should not. At best, the company should "share" summaries, analyses, or pieces of the data, but at best investigators could have only "sufficient time" to "seek ... access to raw data tables or the study database."

So the guidelines would allow corporate research sponsors to analyze the data from studies evaluating their own products and services as they see fit, and the scientists who implemented the study and collected the data could only ask for access to it. 

Summary

The guidelines by Graf et al seem based on a very strange conceptualization of clinical research. In their view, no individual may be responsible for a clinical research project. Research data is controlled by the company that paid for the project, not scientists who implemented the research and collected the data. Research papers may be written by anonymous professional writers while the scientists who did the research only need to review and approve what they have written.

So why should anyone give credence to industry sponsored research?

We have discussed numerous instances in which clinical research was manipulated in favor of vested interests, and when clinical research whose results did not favor vested interests was suppressed. In most cases, the vested interests were held by for-profit pharmaceutical, biotechnology or device anufacturers acting as research sponsors. The guidelines by Graf et al seem to have been cleverly written to to employ comforting platitudes while licensing manipulation and suppression.  They should inspire no confidence in the integrity of industry sponsored research.

The Editorial that Wasn't: Evidence for Systematic Research Manipulation Undetectable by Critical Review

Woe to those of us who have been advocates for evidence-based medicine.  A short description of the evidence-based medicine is medicine whose practice is informed by critical, rigorous review of the best available evidence from clinical research as revealed by systematic search of the published research literature, as well as by the clinician's understanding of biology and the medical and biopsychosocial context, and by the patient's own values.

Evidence-based medicine is based on some key assumptions.  One is that a systematic review will reveal all the results of research studies that are relevant to the issue at hand.  A second is that while the research studies may be flawed and imperfect, they are reported honestly.

Unfortunately, as we have repeatedly discussed, there is more and more evidence that a systematic review will not reveal all relevant results, because research studies may be suppressed, perhaps often, when their results are unfavorable to vested interests.  (Look here for further discussion.)  There is also considerable anecdotal evidence that the design, implementation, and analysis of research studies may be manipulated to make the results more likely to favor vested interests.  (Look here for further discussion.)

An article by Vedula et al just published in the renowned New England Journal of Medicine suggests that such manipulation might be systemic, and that the reporting of manipulated studies may not clearly show what manipulation was done. (1)

To summarize, the authors got access through legal proceedings to internal study protocols and research reports from clinical trials sponsored by Parke-Davis (later merged into Pfizer Inc) of gabapentin (Neurontin) for a variety of clinical problems other than seizures.  Gabapentin was originally marketed as an anti-seizure drug, but Pfizer later "admitted guilt for off-label marketing."  (We discussed the stealth marketing campaign for Neurontin here.)  Vedula et al compared the primary outcome variables specified in the original research protocols, internal research reports, and any publications of the trials' results.  They identified 21 trials, 13 of which were published.  For 12 of the 13, the authors had access to the internal protocol, report, or both.  The main results were:

For 8 of the 12 published trials, there was a disagreement between the definition of the primary outcome in the protocol and that in the published report.... Sources of disagreement included the introduction of an entirely new primary outcome in the published report (in the case of 6 trials); failure to distinguish between primary and secondary outcomes in the published report, even though the protocol did distinguish between them (2 trials); relegation of a primary outcome in the protocol to a secondary outcome in the published report (2 trials); and failure to include in the published report one or more primary outcomes specified in the protocol (5 trials).

Furthermore, it appeared that failure of published articles to clearly and fully report results in terms of the original, pre-specified primary outcome variables occurred when these comparisons were not favorable to gabapentin. As the authors summarized:

Thus, trials with findings that were not statistically significant (P≥0.05) for the protocol-defined primary outcome, according to the internal documents, either were not published in full or were published with a changed primary outcome.

As shown in Figure 3, all the changes that took place between what was specified in the protocol, what was known before publication (as presented in the internal company research reports), and what was reported to the public led to a more favorable presentation in the medical literature of gabapentin's efficacy for unapproved indications.

They concluded:

We are concerned that the reporting practices observed in our analysis do not meet the ethical standards for clinical research or maintain the integrity of scientific knowledge. Fair and honest treatment of patients enrolled in clinical trials of any kind requires full, open, and unbiased reporting. Journal publication, a formalized platform for scientific discourse and dissemination of knowledge, should not be used as a marketing tool for off-label drug use.

Reporting biases such as those we describe here increase the likelihood that interventions will appear to be effective when they are not. Such biases can lead to the omission of negative findings in systematic reviews of intervention effectiveness and in evidence-based guidelines. For example, the 2005 Cochrane systematic review regarding the effectiveness of gabapentin for acute and chronic pain concluded that it is effective on the basis of published findings and should now be updated with the inclusion of unpublished information made available through litigation.

I believe that the article by Vedula et al is particularly important because it shows what appears to be systematic manipulation of the analysis and reporting of multiple clinical trials of the same drug (but for different indications) that had the effect of making the drug appear efficacious when it likely was not. Furthermore, the manipulation was concealed. The published research articles did not completely describe what the intended primary efficacy outcome variables were, did not provide results in terms of these variables, and instead provided results only in terms of variables that were chosen post-hoc as new primary outcome variables.

To address that latter point, I independently reviewed three of the research publications cited by Vedula et al.(2,3,4) All three were noted to have reported significant results favoring gabapentin in terms of a primary efficacy outcome variable that was not identified as such in the corresponding studies' original research protocols or reports (see the supplementary data provided with the New England Journal article.) Per my review, none of the three published articles offered any hint that what they reported as primary outcome variables were not the variables originally chosen in that capacity, nor did they identify what those original primary outcome variables were, or how comparisons made using them turned out.

The implications of the article by Vedula et al are very important. Hence, I was surprised that the article appeared without an accompanying editorial to discuss these implications, and that its publication did not generate much media interest.

So that gives me an opportunity to comment further.

One could start with the implications for evidence-based medicine. As noted above, a short description of the evidence-based medicine is medicine whose practice is informed by critical, rigorous review of the best available evidence from clinical research as revealed by systematic search of the published research literature, as well as by the clinician's understanding of biology and the medical and biopsychosocial context, and by the patient's own values. Evidence-based medicine depends on critical, rigorous review, but the review process is generally done under the assumptions that research publications honestly describe what was done and what its results were. The review process was never designed to detect dishonest reporting or find information that was deliberately concealed. Manipulation of research (design, analysis, and implementation), concealment of that manipulation, and outright suppression of research threaten the foundations of evidence-based medicine. Yet the article by Vedula et al is part of a growing body of evidence that such manipulation, concealment and suppression are widespread, and done to serve vested interests, often commercial.

That is a huge problem for proponents of evidence-based medicine, but also for physicians who want their practices to be based on science, for patients who want their care to be based on science, and for all those in society who see the advancement of medical science as a way to improve peoples' lives.

As senior author Professor Kay Dickersin noted in an interview with Bloomberg "The trouble is, as a scientist, the publication has always been held up to me as the truth. It's the scientific record. What this study indicated is we can't believe that record."

Furthermore, to be a bit more concrete, most physicians, patients, and policy makers depend on what appears to be honest clinical research to make decisions about individual care and health policy. Deliberate and deceitful manipulation of clinical research to favor sponsors' products has likely lead to excessive use of and payments for drugs and devices that are less effective than advertised, if not useless or dangerous. Thus, it is likely that such manipulation is partially responsible for ever increasing health costs and poor health outcomes.

Finally, we need to start thinking about how we can detect and compensate for manipulation of clinical research in the past, and deter such manipulation in the future. One possible deterrent would be, as was noted by Vedula et al, detailed clinical trials registries that contain complete information about trial protocols.  For this to be effective, there need to be mechanisms to assure compliance, and penalties for non-compliance.  Moreover, since clinical research is now global, the registries must have global scope, and enforced assurance of compliance must also be global.

Registries might decrease future manipulation and suppression of research.   No one has suggested, as far as I know, a systematic way to detect and correct for previous manipulation. It would require a major, global investigative effort to uncover manipulation, and it would be a major scientific and policy endeavor to reveal most suppressed research and correct most manipulations.

However, before anything is done, patients, physicians and policy makers must acknowledge and understand the problem.  Yet it seems that even discussing these may be topics that are very uncomfortable for some of us. The longer we shrink from addressing them, however, the worse will be the results for patients, physicians, science and society.

References


1. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored tirals of gabapentin for off-label use. N Engl J Med 2009; 361: 1963-1971. Link here.


2. Mathew NT, Rapoport A, Saper J et al. Efficacy of gabapentin in migraine prophylaxis. Headache 2001; 41: 119-128. Link here.


3. Vieta E, Goikolea JM, Martinea-Aran A et al. A double-blind, randomized, placebo-controlled, prophylaxis study of adjunctive gabapentin for bipolar disorder. J Clin Psychiatr 2006; 67: 473-477.  Link here.


4. Caraceni A, Zecca E, Bonezzi C et al. Gabapentin for neuropathic cancer pain: a randomized controlled trial from the gabapentin cancer pain study group. J Clin Oncol 2004; 22: 2909-2917. Link here.

Failing to Report Adverse Effects of Treatments

We have frequently advocated the evidence-based medicine (EBM) approach to improve the care of individual patients, and to improve health care quality at a reasonable cost for populations. Evidence-based medicine is not just medicine based on some sort of evidence. As Dr David Sackett, and colleagues wrote [Sackett DL, Rosenberg WM, Muir Gray JA, Haynes RB, Richardson WS. Evidence-based medicine; what it is and what it isn't. BMJ 1996; 312: 71-72. Link here. ]

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

One can find other definitions of EBM, but nearly all emphasize that the approach is designed to appropriately apply results from the best clinical research, critically reviewed, to the individual patient, taking into account that patient's clinical characteristics and personal values.

When making decisions about treatments for individual patients, the EBM approach suggests using the best available evidence about possible benefits and harms of treatment, so that the treatment chosen is most likely to maximize benefits and minimize harms for the individual patient. The better the evidence about specific benefits and harms applicable to a particular patient, the greater will be the likelihood that a particular decision based on this evidence will result in the best possible outcomes for the patient.

A new study in the Archives of Internal Medicine focused on how articles report adverse effects found by clinical trials. [Pitrou I, Boutron I, Ahmad N et al. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med 2009; 169: 1756-1761. Link here.] The results were not encouraging.

The investigators assessed 133 articles reporting the results of randomized controlled trials published in 2006 in six English language journals with high impact factors, that is, the most prestigious journals, including the New England Journal of Medicine, Lancet, JAMA, British Medical Journal, and Annals of Internal Medicine. They excluded trials with less common designs, such as randomized cross-over trials. The majority of trials (54.9%) had private, or private mixed with public funding.

The major results were:
15/133 (11.3%) did not report anything about adverse events
36/133 (27.1%) did not report information about the severity of adverse events
63/133 (47.4%) did not report how many patients had to withdraw from the trial due to adverse events
43/133 (32.3%) had major limitations in how they reported adverse events, e.g., reporting only the most common events (even though most trials do not enroll enough patients to detect important but uncommon events).

The authors concluded, "the reporting of harm remains inadequate."

An accompanying editorial [Ioannidis JP. Adverse events in randomized controlled trials: neglected, distorted, and silenced. Arch Intern Med 2009; 169: 1737-1739. Link here] raised concerns about why the reporting of adverse events is so shoddy:

Perhaps conflicts of interest and marketing rather than science have shaped even the often accepted standard that randomized trials study primarily effectiveness, whereas information on harms from medical interventions can wait for case reports and nonrandomized studies. Nonrandomized data are very helpful, but they have limitations, and many harms will remain long undetected if we just wait for spontaneous reporting and other nonrandomized research to reveal them. In an environment where effectiveness benefits are small and shrinking, the randomized trials agenda may need to reprogram its whole mission, including its reporting, toward better understanding of harms.

Pitrou and colleagues have added to our knowledge about the drawbacks of the evidence about treatments that is publicly available to physicians and patients when making decisions about treatment. Even reports of studies with the best designs (randomized controlled trials) in the best journals seem to omit important information about the harms of the treatments they test.

It appears that the majority of the reports that Pitrou and colleagues studied received "private" funding, presumably meaning most were funded by drug, biotechnology, or device companies and were likely meant to evaluate the sponsoring companies' products. However, note that this article did not analyze the relationship of funding source to the completeness of information about adverse effects.

Nonetheless, on Health Care Renewal we have discussed many cases in which research has been manipulated in favor of the vested interests of research sponsors (funders), or in which research unfavorable to their interests has been suppressed. Therefore, it seems plausible that sponsors' influence over how clinical trials are designed, implemented, analyzed and reported may reduce information about the adverse effects of their products reported in journal articles. Trials may be designed not to gather information about adverse events. Analyses of some adverse events, or some aspects of these events may not be performed, or if performed, not reported. The evidence from clinical research available to make treatment decisions consequently may exaggerate the ratios of certain drugs' and devices' benefits to their harms.

Patients may thus receive treatments which are more likely to hurt than to help them, and populations of patients may be overtreated. Impressions that treatments are safer than they actually are may allow their manufacturers to overprice them, so health care costs may rise.

The article by Pitrou and colleagues adds to concerns that we physicians may too often really be practicing pseudo-evidence based medicine when we think we are practicing evidence-based medicine. We cannot judiciously balance benefits and harms of treatments to make the best decisions for patients when evidence about harms is hidden. Clearly, as Ioannidis wrote, we need to "reprogram." However, what we need to reprogram is our current dependence on drug and device manufacturers to pay for (and hence de facto run) evaluations of their own products. If health care reformers really want to improve quality while controlling costs, this is the sort of reform they need to start considering.

NB - See also the comments by Merrill Goozner in the GoozNews blog.