BLOGSCAN - Wandering in the Wilderness

On the 1 Boring Old Man blog, "Mickey," the anonymous blogger, a retired academic psychiatrist, posted about how he figured out the extent that health care dysfunction affected psychiatry while wandering the "wildnerness" outside of academic psychiatry.  He proposed that what needs to die is "the shameful mockery many have made of their own rallying cry - 'evidence based medicine'…"  Dr Wally Smith would have called it pseudo-evidence based medicine.   (And thanks for the mention of Health Care Renewal.)

Has Ghostwriting Infected The "Experts" With Tainted Knowledge, Creating Vectors for Further Spread and Mutation of the Scientific Knowledge Base?

I could have titled this piece "have the deceivers been themselves deceived?"

Recent revelations have shown that biomedical ghostwriting is a scourge that some physicians and scientists -- who are clearly lacking ethics, insight and perhaps an understanding of psychiatric defense mechanisms such as rationalization -- engage in all too often, as it taints the corpus of worldwide scientific literature.

(I covered the issues related to ethical and justified claims of provenance for scientific writing in my post "Wyeth: Ghostwritten Papers Fake, But Accurate" here along with a discussion of why lay leadership of biomedical R&D organizations was not a good idea. Such leaders cannot vouch for the scientific fairness and accuracy emanating from their own organizations.)

One should examine the evidence in a first-person manner. The New York Times has made available secret documents and emails secured by Senator Grassley on Wyeth's ghostwriting practices. These documents were unsealed by a judge in a personal injury lawsuit and chronicle Wyeth's use of a medical writing company to prepare articles for publication in medical journals.

Wyeth said all the articles are "scientifically accurate" and that "it is a common practice in the pharmaceutical industry to work with such firms" (and, by implication, to falsify claimed authorship). As justification for clearly unethical practices, this is a rather poor argument, being a classic logical fallacy known as the Appeal to Common Practice:

Description of Appeal to Common Practice
The Appeal to Common Practice is a fallacy with the following structure:

1. X is a common action.
2. Therefore X is correct/moral/justified/reasonable, etc.

That said, these documents raise an important question of a systemic nature:

Have the scientific ghostwriting deceivers themselves been deceived regarding biomedical reality by these practices?

Might they genuinely believe they are contributing to the scientific literature based on their own erroneous notions, the latter based upon a scientific literature contaminated by other ghostwriters?

To illustrate this point, in the New York Times online document pool, we discover that Dr. Gloria Bachmann, Associate Dean for Women's Health and Professor of OB/GYN and Medicine at UNDNJ-Robert Wood Johnson Medical School was presented with a tidy ghostwritten article entitled "The Importance of Treating Vasomotor Symptoms" (in peri- and post-menopausal women) by authors "TBD" (to be decided):

(Original ghostwritten draft. Authors "TBD." Click to enlarge)

about which Bachmann wrote the following in an email to the true authors' organization:

"Outline is excellent as written. I would like to try for publication as a review article in Obstetrics and Gynecology. We would have to change the title to "Vasomotor Symptoms: Evidence Based Treatment Options" or a title that emphasizes the data driven writing style."

Here is an image of the email:

(Approval email. Click to enlarge)

She found the content OK and agreed to review it.

Apparently, only minor changes were made. (Bachmann should have received accreditation in the acknowledgments section for reviewing the article, and/or a position far down the authorship hierarchy if she actually added some material).

Instead, the reverse occured.

The published document appeared with Bachmann as first (and only) author after Bachmann submitted it to a medical journal under her name, itself unethical. See what "Authors TBD" morphed into in the image below:

(Article submission cover page. Click to enlarge)

Here is how the true authors, medical writers for DesignWrite, the medical writing and communications company retained by Wyeth to write the article, received credit:

(How the "ghosts" received credit for writing the paper. Click to enlarge)

Unbelievable.

Bachmann has since stated that she believed she was adding to science by engaging in this deceit. From the New York Times:

The article, a nearly verbatim copy of the DesignWrite draft, appeared in 2005 in The Journal of Reproductive Medicine, with Dr. Bachmann listed as the primary author. It described hormone drugs as the “gold standard” for treating hot flashes and was less enthusiastic about other therapies.

The acknowledgments thanked several medical writers for their “editorial assistance,” not disclosing that those writers worked for DesignWrite, which charged Wyeth $25,000 to generate the article.

Dr. Bachmann, who has 30 years of research and clinical experience in menopause, said she played a major role in the publication by lending her expertise. Her e-mail messages do not reflect contributions she may have made during phone calls and in-person meetings, she said. ["May have made?" This gossamer excuse is embarrassing to me, as an academic myself - ed.]

“There was a need for a review article and I said ‘Yes, I will review the draft and make sure it is accurate,’ ” Dr. Bachmann said in an interview Tuesday. “This is my work, this is what I believe, this is reflective of my view.”…

She is quite off base regarding scientific authorship and "this being her work", but what also needs examination is her "lending her expertise" and her agreeing with the paper as "reflective of her view."

The paper itself lists many references, for example these:

(References: click to enlarge)

Here is, then, the dilemma, another major problem posed by ghostwriting:

How many of the cited references, which contributed to Bachmann's beliefs regarding these drugs, were themselves ghostwritten and potentially biased, or otherwise influenced by drug companies?

Bachmann's own "expert opinions" about these matters may be based upon false or misleading information she's read in other journals, the percentage of which are genuine vs. ghostwritten or industry-influenced being an unknown at this point in time.

While I believe Bachmann is most likely rationalizing away the unethical nature of attribution without fair and justifying contribution, perhaps fueled by additional motives of academic portfolio building and perhaps present and future potential pecuniary gain, it is possible she is being genuine about agreeing with the "science" in the paper.

However, her agreement may be predicated upon her own tainted expertise -- tainted by reading ghostwritten or industry-influenced articles in her area of specialty.

Will we ever be able to peel back all the layers of the ghostwriting onion to get to the core of impartial and objective scientific articles related to drugs and medical devices? Perhaps not, but the practice must stop going forward.

Tainted literature creates tainted scientific knowledge, the carriers of which may then further taint the knowledge base (with the best of intentions and with firm belief in the fairness and accuracy of their activities, of course).

Practitioners of evidence-based medicine may be unwittingly practicing "evidence-tainted medicine", or "pseudo-evidence based medicine" as described by others on this blog.

Such are the systemic risks of the scourge of ghostwriting.

-- SS

Pseudo-Evidence Based Medicine Threatens Health Care Reform Based on "What Works"

As I posted yesterday, the increasingly noisy debate about health care reform in the US has not dealt much with the issues we often discuss on Health Care Renewal. These include problems in how health care organizations are led which threaten physicians' and other health care professionals' core values using tactics including perverse incentives, deception, and intimidation.
Last night, however, President Obama held a news conference mostly devoted to health care issues, in which he stressed the importance of changing not just how health insurance works, but how health care decisions are made. As Newsweek's "The Gaggle" blog reported,

Can I guarantee that there are going to be no changes in the health care delivery system? No. The whole point of this is to try to encourage changes that work for the American people and make them healthier. The government already is making some of these decisions. More importantly, insurance companies right now are making those decisions. And part of what we want to do is to make sure that those decisions are being made by doctors and medical experts based on evidence, based on what works, because that's not how it's working right now.

So what the President seems to be advocating is making health care more evidence-based, perhaps in the formal sense of evidence-based medicine.

As a card-carrying evidence-based medicine advocate, I certainly agree, but let me reiterate that evidence-based medicine is not just medicine based on some sort of evidence. As Dr David Sackett, and colleagues wrote [Sackett DL, Rosenberg WM, Muir Gray JA, Haynes RB, Richardson WS. Evidence-based medicine; what it is and what it isn't. BMJ 1996; 312: 71-72. Link here. ]

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice. Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients' predicaments, rights, and preferences in making clinical decisions about their care. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens.

Furthermore,

Evidence based medicine is not 'cookbook' medicine. Because it requires a bottom up approach that integrates the best external evidence with individual clinical expertise and patients' choice, it cannot result in slavish, cookbook approaches to individual patient care. External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision.

One can find other definitions of EBM, but nearly all emphasize that the approach is designed to appropriately apply results from the best clinical research, critically reviewed, to the individual patient, taking into account that patient's clinical characteristics and personal values.

So far, so good. I believe the proper application of "real" (as described above) evidence-based medicine has the potential to improve patient outcomes while moderating health care costs. However, we have pointed out how problems arising from concentration and abuse of power in health care threaten the evidence-based medical ideal.

First, there are major problems with the development of the sort of clinical research evidence required by the EBM process. We have discussed how clinical research is frequently manipulated by those with vested interests in producing results that favor the products or services that they sell. The critical review process inherent in EBM is meant to cope with less than optimally designed and implemented research. However, the process was designed to cope with honest mistakes and inevitable trade-offs, not deliberate manipulation by vested interests.

Worse, we have discussed how vested interests may engineer the suppression of research when manipulation fails to produce the desired results. The EBM process assumes that the research on which decisions should be based is an unbiased sample of research that was done (and done to advance science, not commercial or ideological interests). When research whose results are unwanted by vested interests is suppressed, the resulting distortion of the evidence base may irretrievably bias the EBM process.

Finally, we have posted about how vested interests have distorted the discussion, dissemination, and teaching of the results of clinical research. They may develop systematic stealth marketing campaigns, often employing supposed "key opinion leaders," who are paid on the side by marketers, and using "medical education and communication companies"as marketing fronts whose publication strategies include deceptive tactics such as "ghost-writing."

Thus, a rising tide of "pseudo-evidence based medicine" threatens to overwhelm even the most conscientious physicians trying to practice evidence-based medicine.

So, while I applaud President Obama's advocacy of reforming health care to emphasize what the best evidence suggests really works, I do not think this effort will get far unless we deal with the rising tide of pseudo-evidence based medicine. As a minimum, we need full and detailed disclosure of all the relationships among vested interests and medical research and education, and a much greater role for research and education that is not subsidized by corporations bent on using research and education to market their products and services.

Another Misuse of EBM

We agree with Roy Poses' comment on the recent JAMA article about EBM, particularly regarding some of its unfortunate "misuses." In the article that stands as the defining credo of Health Care Renewal, Roy wrote:

Most physicians believe that medicine should be based on science. Evidence based medicine, (EBM) ‘‘the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients’’ [74], is a refinement of this view. Meanwhile, the scientific basis of medicine is under increasingly severe attack.

We'd like to take this opportunity to identify a misuse of EBM that, ironically, contributes to the attack on the scientific basis of medicine, and that authors Montori and Guyatt didn't consider--probably because they are unaware of it. We explain it in the following letter, submitted to JAMA in response to their article. We discuss it at more length on the other blog to which we contribute, Science-Based Medicine, referenced from the letter:

To the editor:

In their recent article, “Progress in Evidence-Based Medicine,” Drs. Montori and Guyatt include important caveats about misuses of EBM. They mention trials funded by industry. They list several, documented problems with such trials, and correctly warn that these can result in erroneous interpretations by “unsophisticated users of the medical literature.” They correctly worry about “abandoning appropriate skepticism.”

There is a related misuse of EBM that leads to erroneous interpretations even by sophisticated authors, and that calls attention to systematic problems in EBM: trials of highly implausible hypotheses. Thus, regarding the homeopathic preparation Oscillococcinum, the Cochrane Collaboration writes:

Though promising, the data were not strong enough to make a general recommendation to use Oscillococcinum for first-line treatment of influenza and influenza-like syndromes. Further research is warranted…[1]

The data, however, are not promising, and further research is not warranted. The reason is that for highly implausible treatments, existing knowledge easily trumps modestly “positive” trial data. In the case of homeopathy, each of its tenets has been more than adequately disproved, and its claim that the potency of a “remedy” increases with each dilution of the original substance, continuing well past the point at which not a molecule of that substance remains in the preparation, contradicts physical reality.[2]

Such considerations reveal at least two, related problems with the evidence of EBM as it pertains to clinical trials. The first is explicit: by relegating “physiology, bench research or ‘first principles’” to Level 5 of its “Levels of Evidence” hierarchy, EBM fails to acknowledge that consistency with firmly established knowledge is a necessary, even if insufficient, basis for accepting a hypothetical treatment as effective.[2]

The second problem is part of the culture, if not the letter of EBM: by eschewing Bayesian in favor of “frequentist” statistics, its practitioners can excuse themselves from justifying estimates of prior probability, and erroneously conclude that trial data tell us more about reality than they do.[3-5]

In January, 2008, several colleagues began the blog Science-Based Medicine, referenced below.* The name, which could just as well have been “knowledge-based medicine,” is a frank recognition of the EBM weaknesses described above. Because of our interests, Science-Based Medicine’s content is predominantly, though not entirely, concerned with implausible health claims. Its larger message, however, is that these claims call attention to fundamental ways in which EBM can be improved, such as to justify its own claim to being an arbiter of real evidence.

Kimball C. Atwood, IV

Wallace I. Sampson

10/19/2008

* Steven Novella, David Gorski, Harriet Hall, Wallace Sampson, Kimball Atwood

[1] Vickers A, Smith C. Homoeopathic Oscillococcinum for preventing and treating influenza and influenza-like syndromes. Cochrane Database of Systematic Reviews 2006, Issue 3. Art. No.: CD001957. DOI: 10.1002/14651858.CD001957.pub3.

[2] Atwood K. Homeopathy and Evidence-Based Medicine: Back to the Future Part V. Science-Based Medicine 2008 (Feb. 8). http://www.sciencebasedmedicine.org/?p=42. Accessed October 19, 2008

[3] Atwood K. Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine.” Science-Based Medicine 2008 (Feb. 15). http://www.sciencebasedmedicine.org/?p=48. Accessed October 19, 2008

[4] Goodman SN. Toward evidence-based medical statistics. 1: The P value fallacy. Ann Intern Med. 1999;130:995-1004.

[5] Goodman SN. Toward evidence-based medical statistics. 2: The Bayes factor. Ann Intern Med. 1999;130:1005-13.

BLOGSCAN - Duplicate Publication about an SSRI

On the Clinical Psychology and Psychiatry Blog, the anonymous blogger described an interesting example of apparent dual publication on a selective serotonin reuptake inhibitor (SSRI) anti-depressant drug, duloxetine (Cymbalta, manufactured by Eli Lilly). Important aspects of the story include how both articles draw conclusions that go well beyond the (same) data in their enthusiasm for the drug in question, and how the second article incorporated an author not found in the first, an author who also happened to be the co-editor in chief of the journal in which the second article appeared, and the leader of an important medical society. This author has been the subject of previous posts (like this one), on Health Care Renewal.

Dr Marcia Angell on Reforming a "Broken System" of Clinical Research

This week's Journal of the American Medical Association (3 September, the paper version of which seems to arrive early at my house) is notable for having three commentaries on relationships among physicians, academic medicine, and industry. One, by David Rothman and Susan Chimonas, seemed rather optimistic, while two, by Dr Arnold Relman and by Dr Marcia Angell, were quite dark.

The latter [Angell M. Industry-sponsored clinical research: a broken system. JAMA 2008; 300: 1069-1071. Link here when available] should be required reading for anyone who worries about the issues we raise on Health Care Renewal. Dr Angell provided an excellent summary of how commercial "sponsors" have come to dominate clinical research, and thus ensure that the resulting studies favor their vested interests. My only criticism of her summary of the data was that it may actually have been too optimistic. For example, Dr Angell focused almost exclusively on pharmaceutical companies' influence on research, while the problem is even larger. Biotechnology companies, device manufacturers, and multiple miscellaneous commercial suppliers of health care goods and services also may influence research. Dr Angell's admittedly extensive list of the kinds of relationships among researchers and industry (grant support, paid consulting, service on speakers' bureaus and advisory panels, and stock ownership) was incomplete, leaving out, for example, the even more acute conflicts of interest generated by service on companies' boards of directors and service as companies' officers.

But those are quibbles. Most striking was Dr Angell's summary:

Looking at this picture altogether, it would be naive to conclude that bias is only a matter of a few isolated instances. It permeates the entire system. Physicians can no longer rely on the medical literature for valid and reliable information. This is the conclusion I reluctantly reached toward the end of my 2 decades as an editor of the New England Journal of Medicine, and it has been reinforced in subsequent years. Clinicians just do not know anymore how safe and effective prescription drugs really are, but these products are probably nowhere near as good as the published literature indicates.

I am afraid I must concur, and also concur with her summary of the effects of the substitution for valid clinical research of pseudo-evidence favoring the vested interests of "research" sponsors:

Physicians learn to practice a very drug-intensive style of medicine.

Physicians are also led to believe that the newest, most expensive brand-name drugs are superior to older drugs or generics, even though there is seldom any evidence to the effect because sponsors do not usually compare their drugs with old drugs at equivalent doses.

Physicians learn to use drugs for off-label uses without good evidence of effectiveness.

Keep in mind that the same considerations apply to devices, so that physicians also learn to practice a test- and procedure-intensive style of medicine; are led to believe that the newest tests, devices, and procedures are superior to older ones without any evidence comparing the new to the old; and physicians learn to extend the use of tests, devices, and procedures far beyond what the evidence would support.

In addition, I agree with Dr Angell's advocacy of an independent body that would administer all clinical trials of drugs. Dr Angell suggested it be an Institute within the NIH. I think it might be possible to form a not-for-profit body that would be independent of government that would do the same thing. Furthermore, that body, or another in parallel should also administer all clinical trials of diagnostic tests, devices and procedures, for analogous reasons.

Finally, Dr Angell urged academic investigators and their institutions to "be at the forefront of efforts to reform the system of clinical research and not leave it to the government and legal profession." However, as Dr Angell previously noted, a majority of medical school faculty have financial ties to drug, device, biotechnology and other health care companies, as do a majority of department chairs. As we have noted, some top leaders of academic medicine serve also as top leaders of for-profit health care corporations (see this post), and other corporations whose products have profound effects on health, even tobacco companies (see this post). Many academic leaders make a small fortune working on the side for such companies. It may be too much to expect that such leaders would be at all interested in reforming a system that has so effectively lined their pockets. For better or worse, we may need a wholesale change in the leadership of academic organizations before we can expect any real cooperation from them in transforming clinical research from being mainly a marketing exercise to once more being real clinical science.

ADDENDUM (5 September, 2008) - See also comments by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma blog.

The NIH Trial to Assess Chelation Therapy (TACT): an unethical study sponsored by taxpayers

Two occasional HCR bloggers are co-authors of an article published this week:

Why the NIH Trial to Assess Chelation Therapy (TACT) Should Be Abandoned

Kimball C. Atwood IV, MD; Elizabeth Woeckner, AB, MA; Robert S. Baratz, MD, DDS, PhD; Wallace I. Sampson, MD

Medscape J Med. 2008;10(5):115. ©2008 Medscape
Posted 05/13/2008

Abstract:

The National Institutes of Health (NIH) Trial to Assess Chelation Therapy (TACT) was begun in 2003 and is expected to be completed in 2009. It is a trial of office-based, intravenous disodium ethylene-diamine-tetra-acetic acid (Na2EDTA) as a treatment for coronary artery disease (CAD). A few case series in the 1950s and early 1960s had found Na2EDTA to be ineffective for CAD or peripheral vascular disease (PVD). Nevertheless, a
few hundred physicians, almost all of whom advocate other dubious treatments, continued to peddle chelation as an office treatment. They claim that chelation dramatically improves symptoms and prolongs life in 80% to 90% of patients. In
response, academics performed 4 controlled trials during the 1990s. None favored chelation, but chelationists repudiated those findings.

We have investigated the method and the trial. We present our findings in 4 parts: history, origin and nature of the TACT, state of the evidence, and risks. We present evidence that chelationists and their organization, the American College for Advancement in Medicine, used political connections to pressure the NIH to fund the TACT. The TACT protocols justified the trial by misrepresenting case series and by ignoring evidence of risks. The trial employs nearly 100 unfit co-investigators. It conflates disodium EDTA and another, somewhat safer drug. It lacks precautions necessary to minimize risks. The consent form reflects those shortcomings and fails to disclose apparent proprietary interests. The trial's outcome will be unreliable and almost certainly equivocal, thus defeating its stated purpose.

We conclude that the TACT is unethical, dangerous, pointless, and wasteful. It should be abandoned.

The entire article is available at: http://journal.medscape.com/viewarticle/570625_print

You may be asked to "register"; don't worry, it's free. The article is very long, but the Introduction, Executive Summary, Discussion, and Conclusion are reasonably succinct and make the important points. Readers who want to learn more details, who want to see more evidence for our assertions, or who are compelled by an odd fascination with crackpotism (my own weakness) will want to read more.

I've posted a similar announcement on Science-Based Medicine.

"A Catastrophic Failure" - More on the Ketek Trial Debacle

We previously posted (also here, here and here) about the ill-fated clinical trial, study 3014, of the antibiotic telithromycin (Ketek) made by Sanofi-Aventis, run by Pharmaceutical Product Development Inc. (PPD). Problems with the trial included fabrication of data at one clinical site, and allegations of manipulation of data at another. The physician in charge of the first site was convicted of mail fraud, and the physician in charge of the second had his license suspended. Although the results of this trial were never published, it still crept into the clinical literature: it was cited in a review article in the New England Journal of Medicine.

The case is now the subject of a US congressional investigation. As reported by the Associated Press (via the International Herald Tribune),

Democrats in the U.S. House of Representatives are threatening to hold a member of President George W. Bush's Cabinet in contempt for not turning over documents in a probe of a Sanofi-Aventis antibiotic.

Dingell, who chairs the House Energy and Commerce Committee, said he would support holding Health and Human Services Secretary Michael Leavitt in contempt for refusing to turn over FDA briefing documents subpoenaed by the committee.

The documents were used to prepare FDA Commissioner Andrew von Eschenbach for his appearance before lawmakers last year. Von Eschenbach testified the FDA did not use the flawed safety study to approve Ketek. Dingell and other Democrats say that statement may be untrue.

Robert West, an FDA agent who first investigated Ketek, said he tried to get permission in 2002 to look into whether Aventis was aware of fraudulent data when it submitted the study. West said his request was blocked by senior FDA officials, although he said he did not know which ones.

"A catastrophic failure" was how another FDA investigator, Douglas Loveland, described the company's handling of the study.

"The decision-making process Aventis used to investigate these problems was illogical and ineffective and it could have led them to come to the wrong conclusion," Loveland told House lawmakers.

The investigator stopped short of saying Aventis, Sanofi-Aventis' corporate predecessor, knew the results were false when they submitted them but said it "should have known." Loveland said the results contained all the hallmarks of sham data, including forged signatures and crossed out information.

Sanofi President for Research and Development Paul Chew testified that the company submitted the study in "good faith."

The company was not able to spot the fake patient results....

It seems clear that this case involved a very sloppy clinical trial sponsored and run by a pharmaceutical company. Such sloppy science breaks promises made to human subjects of clinical research that their participation would go to advance science and improve health care. Sloppy trials that produce dubious data will do neither. Furthermore, sloppy science that produces dubious results provides patients and doctors pseudoevidence rather than the best possible evidence relevant to decisions about health care interventions, and hence may also lead to bad decisions that harm patients.

It is bad enough that clinical trials sponsored by pharmaceutical, biotechnology, and device companies are often sloppy, and designed to show their products in the most favorable light, rather than provide valid and unbiased data testing hypotheses most important for patients' and physicians' decisions. This case now raises the more disturbing possibility that high government officials entrusted with protecting the public's health and safety may have hid the sloppiness and unreliability of the Ketek trial. Time and further investigation will tell if this possibility is more than speculation. But the latest events in the Ketek saga clearly show the need for more open and transparent clinical research on drugs and devices, and more open and transparent regulation of such companies by the government.