What the November, 2009, Breast Cancer Screening Argument Obscured

In November, 2009, a rancorous argument about screening mammography for women aged 40-49 was touched off by the publication of updated guidelines(1), supported by a systematic literature review(2) by the US Preventive Services Task Force (USPSTF).  The guidelines suggested that yearly mammographic screening for women in that age group should not be automatic, but a decision made for individual patients after discussion between the patients and their doctors.  This was based on a critical review of the best available data which suggested that the benefits of screening acrue to only a few patients.  1904 women would have to start screening and continue for multiple rounds to prevent one cancer death over 11-20 years of follow-up.  These benefits had to be balanced against a number of potential risks, including the risks of treatment of cancers that might never behave malignantly, and the at least theoretical risk of generating new cancers through radiation exposure from mammography. 

These seemingly reasonable recommendations generated heated responses.  The debate, to be charitable, seemed to be at its core about how one should weigh benefits and harms in making individual and health policy decisions.  Since different people value different outcomes differently, I was not sure at the time how to make a meaningful contribution to this debate, or whether the debate had to do with the issues we usually discuss on Health Care Renewal.

I should note that the USPSTF guidelines never said "do not screen" women under age 50, or that the government should not pay for such screening.  They did say "the decision to start regular, biennial screening mammography before the age of 50 years should be an individual one and take pateint context into account, including the patient's values regarding specific benefits and harms."  It is hard to see how anyone could argue with that as an expression empowering patients' choices and values.  (For further discussion about how the recommendations were actually modest and reasonable, see Partridge and Winer.)(3)

Now that the dust has settled, it may be useful to reflect further on this.  Doing so suggests that the rancorous US debate mainly obscured rather than illuminated the major issues regarding mammography screening, particularly about our lack of clear evidence from clinical research needed to make the best individual and  policy decisions about mammography. 

It seems to me that the main questions one must answer to make an individual or policy decision about screening mammography are:

Does it improve longevity?

This is not the same as asking whether mammography reduces mortality from breast cancer.  It is theoretically possible that while decreasing mortality due to breast cancer, screening and its downstream consequences increases mortality from other causes.  At least in theory, screening may detect small tumors that would never grow or metastasize.  Treating such tumors could sometimes lead to premature death due to complications of surgery, radiation, or chemotherapy.  Furthermore, screening also involves periodically exposing large numbers of women to radiation, which may sometimes cause new tumors.  So reducing breast cancer mortality does not automatically mean that overall longevity would be improved.

There have been eight major trials of breast cancer screening which included women younger than 50. (See reference 2.)  None demonstrated a statistically significant increase in overall survival (that is, an increase unlikely to have been due to chance alone) due to screening.

Does it reduce suffering, improve functioning or generally improve quality of life?

To my knowledge, no major trial attempted to answer this question.  No such data is mentioned in the USPSTF systematic review.

Do the above benefits outweigh all its potential harms and risks?

So we cannot answer this question, because the benefits that might be most meaningful to patients (overall survival, symptom reduction, functional improvement, overall quality of life improvement) have not been clearly measured.

A Lack of Relevant Evidence

So the USPSTF guidelines, like other relevant guidelines, were based on the evidence that is available.  Since the evidence did not directly answer the most important questions, the guideline writers were left doing the best they could with evidence that only indirectly addressed the main issues.  No wonder they ended up unable to make a clear recommendation, and leaving the decisions to individual discussions, and individual discussions that would necessarily hinge on guesses about the unknown. 

One would think that a big point of discussion about breast cancer screening would be why after eight trials enrolling a total of about 350,000 patients reported over 20 years we still cannot answer the big clinical questions.  A related point for discussion in the US is why only one, and the earliest trial was conducted here.  If we here in the US think breast cancer screening is such a major concern (and we should think so), why have we been unable to mount a single important trial of it since the HIP trial conducted more than 30 years ago?

Instead, the rancorous debate in the US included...

Anecdotes, Some Irrelevant

The press found a number of women who said they would not be alive were it not for screening mammography before age 50.  With all due respect, one cannot tell whether an individual whose tumor was found on screening mammography would still have been diagnosed early enough for succesful treatment in the absence of screening mammography.  (And also with all due respect, we have no idea whether there also are cases of women who died as a result of treatments of tumors that never would have progressed, or cases of women who died of tumors caused by radiation from multiple mammograms.)  Reasoning from single cases when people, diseases, and treatment results vary so much is likely to mislead. 

It is somewhat ironic that some of the cases cited were of women who had breast cancer diagnosed before age 40, even though the debate was supposedly about screening from ages 40-49.  For example, in an inflammatory article that suggested that some "oncologists" might want the USPSTF sent to the prison at Guantanamo Bay, Washington Post editorialist Dana Milbank cited cancer activist Nancy Brinker, who mentioned her sister "whose breast cancer was found with a mammogram at age 37," (and apparently who tragically is no longer alive).(4) 

Going Well Beyond the Evidence

As noted above, no trial has shown that screening mammography for women under 50 increases overall longevity.  We all hope it does, but so far, there is no clear evidence that it does. 

Yet multiple media reports included assertions that screening mammography saves lives.   For example, the breast cancer activist mentioned above said, "mammography saves lives," apparently including mammography under age 50.(4)  An op-ed column by Dr Alan Kaye, chairman of radiology at Bridgeport Hospital, asserted "large, multinational research studies have shown that mammography saves lives in all age groups covered by the current guidelines."(5)  I would challenge him to show me a single such study that found a statistically significant increase in overall survival for patients under 50.  A Texas radiologist stated, "I diagnosed a 40-year old woman with breast cancer last week.  If she had waited 10 years, with pre-menopause breast cancer she would have been dead."(6)  Unfortunately, since she was just diagnosed, how can he be certain that she will survive any given amount of time?  How could he know that the cancer might not have become manifest, absent that single mammogram, later while still treatable? 

I do not want to be too hard on patients who do not appreciate that the outcomes of testing and treatment for breast cancer are not certain.  However, one would hope that physicians would be able to deal with this uncertainty.

Conflicted Opinions

Some of the more strident discourse came from those who may have had financial vested interests in promoting screening mammography.  Fugh-Berman and Bell pointed out numerous "fact-free emotionally charged statements" made by people who appeared to "reading from the same script-book."(7)  They identified that many of the loudest critics of the USPSTF guidelines were affiliated with not-for-profit organizations with impressive names, but also with substantial financial support from corporations that make products used in mammography.  Also, some had personal financial relationships with such corporations. 

An op-ed article by former US Food and Drug Agency (FDA) commissioner Dr Andrew von Eschenbach and Ms Nancy Desmond distorted the USPSTF guidelines to mean "most women should delay screening until they are 50," and claimed that was based on cost concerns, not clinical evidence.(8).  Desmond is the CEO of and von Eschenbach is now a senior advisor to the Center for Health Transformation.  The Center's members include numerous pharmaceutical and device manufacturing corporations, including several that make mammography equipment (e.g., GE Healthcare and Siemens).

Summary

Cancer, especially breast cancer, has major emotional connotations, and can be a difficult issue to deal with from many people.  The conflicting emotions cancer brings out in many patients may understandably affect their physicians, as well as friends and family.  Nonetheless, physicians, other health policy professionals, and health policy experts can serve patients better if they do not allow the patients' understandable affective responses cloud their understanding of the clinical and scientific issues. 

Yet the late 2009 debate in the US about screening mammography included many responses in which emotion seemed to overwhelm reason.  It may also be that some such responses came from people who had vested interests, or whose employers had vested interests that supported the emotional, rather than the reasoned approach.  Meanwhile, no one seemed to acknowledge that a big reason we are still debating this topic is that we have not made the effort or expended the resources to do good trials of sufficient size to answer the questions that need answering.  Of course, such trials might provide answers that would upset some people, or threaten others' incomes.  (As one news article pointed out, mammography is now a $5 billion a year industry in the US.)(6)

So my end of annus horribilis 2009 message on Health Care Renewal is to better serve our patients, from 2010 onward we health care professionals need  to try harder to put evidence and logic ahead of our own emotions, and certainly ahead of our financial self-interest.  

Note that numerous bloggers have taken on this topic, so see posts on Respectful Insolence, GoozNews,  Health Care Organizational Ethics, and the Evidence in Medicine blog.
References

1. US Preventive Services Task Force. Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement.  Ann Intern Med 2009; 151: 716-726. [link here]
2. Nelson HD, Tyne K, Naik A et al. Screening for breast cancer: an update for the U.S. Preventive Services Task Force.   Ann Intern Med 2009; 151: 727-737.  [link here]
3. Partridge AH, Winer EP. On mammography - more agreement than disagreement.  N Engl J Med 2009; 361: 2499-2501. [link here]
4. Milbank D. Feeling farther from the finish.  Washington Post, Nov 24, 2009.  [link here] 
5. Kaye A. An alarming retreat on early detection.  Hartford Courant, Nov 25, 2009 [link here]
6.  Jacobson SJ. Dallas-area clinics ignore proposed rules, still push for mammograms. Dallas News, Nov 27, 2009.  [link here]
7. Fugh-Berman A, Bell A. Mammography and the corporate breast.  Bioethics Forum, Nov 24, 2009.  [link here]
8. von Eschenbach A, Desmond N. Government panels can't put price on human life. Associated Press, Nov 24, 2009.  [link here]

Spun Silly: Academic Medical Center Cancer Treatment Advertising in the Era of Hype and Flim-Flam

Over the weekend, the New York Times reported on how prestigious academic medical centers advertise cancer care.  Here are some examples,

Prostate Cancer Surgery at Mount Sinai

A print advertisement for prostate cancer surgery at Mount Sinai Medical Center in Manhattan is typical of the way many elite research and teaching hospitals sell hope to the public.

'Our newest prostate specialist, Dr. David Samadi, has pioneered a minimally invasive approach that allows him to retain the highest cancer cure rates with the lowest risk of side effects,' says the ad.

Highest cure rates. Lowest risk. What evidence does the medical center have to back up such superlatives?

The ad’s claims are based on the successful results of Dr. Samadi’s operations and testimonials from his patients, said Jane Zimmerman, Mount Sinai’s chief marketing officer.

However, the article noted that the hospital could provide no studies that showed that its or Dr Samadi's results were superior to those of other hospitals or other surgeons.

... the ad with the superlative prostate cancer claims ... was later revised to say that Dr. Samadi’s approach gives 'high rates of success coupled with lowered risks of side effects.' Ms. Zimmerman said Dr. Samadi was not available to be interviewed.

Also,the people who concocted the advertisement said it was not really meant to tell prospective patients that the surgeon had better results than all others:

But marketing executives defend their approach, saying cancer treatment ads tend to play more heavily on emotion than on medical statistics because the ads are not intended to inform people who already have the disease. They are meant to make an impression on future patients, who may decide on treatments years after they have seen an ad, or to sway influential people who might advise a future patient.

'This isn’t retail advertising,' said Ellis Verdi, president of the DeVito/Verdi Agency in Manhattan.

The agency produced the Mount Sinai ad, which ran in The New York Times, and has created cancer ads for other hospital clients. 'This is reputation advertising,' Mr. Verdi said. 'There is a very big difference.'

But the advertisement said that the hospital's prostate cancer specialist had the highest survival and lowest adverse event rates.  How would a patient with prostate cancer realize that the advertisement was only meant to enhance the hospital's reputation, but not meant to speak to him?  

Radiation for Brain Tumors at Massachusetts General Hospital

'We gave Nick something he couldn’t find anywhere else in the Northeast. Life without cancer.'

That was the text of a print ad last year by the Massachusetts General Hospital Cancer Center in Boston, promoting its $50 million center for proton beam therapy, a kind of high-energy radiation to treat brain tumors and other cancers.

The hospital was the only medical center in the region with a proton therapy center, the ad said, enabling doctors there to successfully treat the brain tumor of a young man named Nick.

The ad’s concept was that Nick had a greater chance of survival because the precise proton beam could destroy malignant brain tissue while leaving surrounding healthy brain tissue intact, said Jodie Justofin, the marketing director at Mass General’s cancer center.

Dr. Thomas F. DeLaney, the medical director of the Francis H. Burr Proton Therapy Center at Mass General, said he had no involvement in the ad and did not have any information about Nick.

However, the article also noted that "no rigorous studies have shown that proton beam therapy has higher brain-cancer cure rates than other treatment methods, said Dr. [John D] Birkmeyer of Michigan [a professor at the University of Michigan and cancer outcomes researcher]. 'The ad might be accurate that they are the only hospital in the Northeast with this particular widget,' he said. 'But it could be misleading that the availability of this particular widget gave this patient better odds of survival.'"

Again, the advertisement said that the patient got "life without cancer," something he could not get anywhere else in the Northeast.  How would a patient with a brain tumor realize that the advertisement was merely based on a "concept," rather than scientific evidence that his  or her only hope for "life without cancer" could come from proton beam therapy at the Massachusetts General Hospital?

Surgery for Cervical Cancer at Memorial Sloan-Kettering Cancer Center

'Cancer, You said I’d never bear children,' reads the handwritten letter, held out by a pretty, healthy-looking woman, as a toddler peeks from behind the paper. 'My daughter says you’re wrong.'

That recent print ad from Memorial Sloan-Kettering Cancer Center in Manhattan tells the story of Michelle Rogala, a patient with cervical cancer.

Ms. Rogala’s hospital in New Jersey could offer her only a hysterectomy, an operation that would have left her unable to have children. Instead, she went to Memorial Sloan-Kettering, where she entered a clinical trial that was studying less invasive surgery. Ms. Rogala now has a little girl named Maddie.

Ellen Miller-Sonet, vice president for marketing at Memorial Sloan-Kettering, said consumers seeing the ads realizes that these were individual stories. 'They know that no two people are the same,' she said.

However, Ms Rogala told the NY Times, "hers had indeed been a special case. She had early-stage cervical cancer, she said, making her eligible for a novel operation that has now become a standard treatment at the center. After her operation, doctors told her she would need fertility treatments to conceive. But she said she turned out to be one of the few patients in the study who did not need radiation — which can cause fertility problems. She later became pregnant without medical intervention."

Again, why "consumers," much less patients with cervical cancer, would realize that the advertisement was just an "individual story," not a promise that the hospital's treatment of cervical cancer would not prevent future pregnancies, was entirely obscure.

Summary

The three advertisements described in the NY Times article had some features in common. All seemed to promise exceptional results. None were based on clear scientific evidence. All seemed to have been products of marketers and advertising agencies working without input from the physicians who actually provide the treatments they were advertising. All the marketers defended their work by saying that the advertisements did not actually mean what they appeared to mean.

My most obvious comment is that hospitals, even the most prestigious teaching hospitals, now seem to be willing to market their services like the used car salespeople seen on late night television.  Such advertisements, of course, are unseemly and undignified coming from such august institutions.  Worse, they seem to promise more than what these or any hospitals can be proved to deliver, and the only defense of the marketers who produced the advertisements were that they did not mean what they seemed to mean.

This shows the sad, and ultimately deceptive and unethical effects of turning the leadership of our best medical institutions over to businesspeople with little knowledge or understanding of the values of  health care.

It also shows what has happened to health care in an age of hype, scam, sham, spin and flim-flam.  It all seems part of what Frank Rich just wrote about in the NY Times:

If there’s been a consistent narrative to this year and every other in this decade, it’s that most of us, Bernanke included, have been so easily bamboozled. The men who played us for suckers, whether at Citigroup or Fannie Mae, at the White House or Ted Haggard’s megachurch, are the real movers and shakers of this century’s history so far. That’s why the obvious person of the year is Tiger Woods. His sham beatific image, questioned by almost no one until it collapsed, is nothing if not the farcical reductio ad absurdum of the decade’s flimflams, from the cancerous (the subprime mortgage) to the inane (balloon boy).

What makes the golfing superstar’s tale compelling, after all, is not that he’s another celebrity in trouble or another fallen athletic 'role model' in a decade lousy with them. His scandal has nothing to tell us about race, and nothing new to say about hypocrisy. The conflict between Tiger’s picture-perfect family life and his marathon womanizing is the oldest of morality tales.

What’s striking instead is the exceptional, Enron-sized gap between this golfer’s public image as a paragon of businesslike discipline and focus and the maniacally reckless life we now know he led. What’s equally striking, if not shocking, is that the American establishment and news media — all of it, not just golf writers or celebrity tabloids — fell for the Woods myth as hard as any fan and actively helped sustain and enhance it.

People wanted to believe what they wanted to believe. Tiger’s off-the-links elusiveness was no more questioned than Enron’s impenetrable balance sheets, with their 'special-purpose entities' named after 'Star Wars' characters. Fortune magazine named Enron as America’s 'most innovative company' six years in a row. In the January issue of Golf Digest, still on the stands, some of the best and most hardheaded writers in America offer 'tips Obama can take from Tiger,' who is typically characterized as so without human frailties that he 'never does anything that would make him look ridiculous.'

I would note that the health care precursor to all this was how the former CEO of the Allegheny Health Education and Research Foundation (AHERF), the biggest health care system in Pennsylvania in the 1990s, was hailed as a visionary in the medical press and scholarly literature, which later ignored AHERF's bankruptcy and its former CEOs criminal conviction (see post here.)  So my one disagreement with Mr Rich is that the problems are much older than the 21st century.
Rich concluded,

after a decade of being spun silly, Americans can’t be blamed for being cynical about any leader trying to sell anything. As we say goodbye to the year of Tiger Woods, it is the country, sad to say, that is left mired in a sand trap with no obvious way out.

The way out of our sand trap in health care, of course, is to refuse to be spun any more. We need to stop believing the hype propogated by all the clever marketers, and all the self-interested CEOs who hire them.

Meanwhile, I would suggest to any cancer patient who failed to get the wonderful results promised by some slick hospital advertisement, there may be some lawyers who with whom you ought to speak.

StandUp4Pharma?

Many people in the US watched a telethon carried on all three major television networks meant to raise money for cancer research. The telethon was sponsored by StandUp2Cancer.org, a not-for-profit organization apparently run by a number of Hollywood celebrities, whose mission is to sponsor cancer research.

On the Schwitzer Health News blog, this post raised some questions about the purpose of the organization, and in particular noted that its "heroes" included GlaxoSmithKline and Amgen.

On perusal of the organization's poorly organized and not very functional web-site, I found a listing of the membership of its Scientific Advisory Committee, who will determine who gets research money from the organization. The committee included many notable scientists, but I also noted that it included four people who worked full-time for pharmaceutical or biotechnology companies:

• Julian Adams, Chief Scientific Officer, Infinity Pharmaceuticals Inc
• Richard B Gaynor, Vice President, Cancer Research and Clinical Investigation, Eli Lilly and Company
• Cecil B Pickett, President, Research and Development, Biogen Idec
• Laura K Shawver, Chief Executive Officer, Phenomix Corp

In addition, the board member biographies disclosed other members with financial ties to other such corporations:

• Philip A Sharp, Institute Professor, David H Koch Institute for Integrative Cancer Research, Massachusetts Institute of Technology also "co-founded Biogen (now Biogen Idec), Alnylam Pharmaceuticals, an early-stage therapeutics company and Magen Biosciences Inc., a biotechnology company developing agents to promote the health of human skin."
• Vicki L Sato, Professor of Management Practice, Harvard Business School, also is the retired President of Vertex Pharmaceuticals, and a current business advisor to Atlas Ventures "and other enterprises in the biotechnology and pharmaceutical industry." She is also a current member of the boards of directors of Bristol Myers Squibb, PerkinElmer Corporation, Infinity Pharmaceuticals, and Alnylam Pharmaceuticals.

Furthermore, a bit of searching the web revealed other financial ties of the scientific advisory board:

• Philip A Sharp is also, according to his CV, a Member of the Scientific Advisory Board of Sirtris Pharmaceuticals, 2003- ; Member of the Scientific Advisory Board of Fidelity Biosciences Group, 2004- ; Member of the Board of Advisers, Polaris Venture Partners, 2002- ; Chairman of the Scientific Board (to 2002) and member of the
  Board of Directors of Biogen Idec; Chairman of the Scientific Board and member of the Board of Directors of Alnylam Pharmaceuticals; and member of the Scientific Board and member of the Board of Directors of Magen Biosciences.
• Arnold J Levine, Professor, Institute of Advanced Study and Cancer Institute of New Jersey is also a member of boards of directors of Theravance, Applera Corporation and Infinity Pharmaceuticals, Inc, according to the Theravance web-site.
• Brian J Druker, Professor of Medicine, Oregon Health Sciences University Cancer Institute, is also on the scientific advisory board and clinical advisory board of Avalon Pharmaceuticals, per the company web-site; on the clinical oncology advisory board of Ambit Biosciences, according to this news item; and in 2003, disclosed consulting for and receiving honoraria from Novartis, in this article.
• Richard N DuBois, Provost and Executive Vice President, UT MD Anderson Cancer Center, also has been a consultant to Pfizer, and received honoraria from Axcan Pharmaceuticals, disclosed in this article; and disclosed consulting for Abbott, Novartis, and Salix in this article.
• Waun Ki Hong, Head, Division of Cancer Medicine, UT MD Anderson Cancer Center, is a member of the medical oncology advisory panel for Omnitura, according to the company web-site; a member of the medical oncology advisory panel for Genyous, according to the company web-site; on the scientific advisory board of GenVault, according to this news item; and on the scientific advisory board of Agennix, according to this news item.
• William G Kaelin, Professor, Dana-Farber Cancer Institute, is also a founder of Imigen Systems, according to this article; a member of the scientific advisory board of Nextech Ventures, according to the company web-site; and a member of the scientific advisory board of Arqule, according to the company web-site.
• Ronald Levy, Chief, Division of Oncology, Stanford University, is also a member of the
  scientific advisory boards of XTL Therapeutics, Therion Incorporated, Coley Pharmaceutical Group Inc., Xcyte Therapeutics, Agensys, Pointilliste, Cell Genesis, and Five Prime, according to his CV; and is also on the scientific advisory board of InNexus Biotechnology, according to this news article.
• Tak W Mak, Chief Scientific Officer, Princess Margaret Hospital, Campbell Family Institute for Breast Cancer Research is also a founder of Agios Pharmaceuticals, according to the company web-site, and on the scientific advisory board of Entremed, according to the company web-site.

The explanation by StandUp4Cancer.org of where the money it raises goes, and why, includes,

SU2C believes that it can benefit the most patients by accelerating the course of cancer research through raising philanthropic dollars and developing unique mechanisms to utilize these funds.

SU2C is committed to transparency and high-quality science in its funding activities and believes that this can be accomplished within the framework of a truly innovative approach to support SU2C-funded research.

However, of 19 people on the board that will control how this research money is spent, 4 were disclosed to work full-time for pharmaceutical or biotechnology companies, and two others were disclosed to have ties to such companies. Seven other board members had multiple, undisclosed ties to pharmaceutical or biotechnology companies. Thus, 13 of 19, a clear majority, including the board chair and vice-chair, had multiple and significant financial ties to pharmaceutical and device companies. Most of those ties were not disclosed on the organization's web-site. I did not watch all of the organization's telethon, but did not see anything approaching such disclosure in the part I did watch.

Where is the transparency there?

A major new effort to raise money to do research on the extremely important topic of cancer, supposedly devoted to transparency, should at least make clear what financial ties the people in charge of the effort have to companies that have commercial interests in selling cancer tests and treatments, and stand to benefit from particular research directions. This is even more important when appeals for funding are widely broadcast to an audience which may not be very sophisticated about biomedical and clinical research, and not in an easy position to determine who may be benefited from research that goes in particular directions.

Cancer Data Collection Outsourced, and Then Manipulated

I have a favorite quote about the impossibly complex bureaucratic structure of the American health care system from Oxymorons by JD Kleinke,(1) (which I used in my "Cautionary Tale" article in the European Journal of Internal Medicine[2]):

Tens of thousands of well-meaning people work throughout the health care system, none of whom ever see a patient or deliver any actual medical care. They preside over an infinity of rules, regulations, forms, processes, contract outsourcing, financial brokering, benefit plan tinkering, analytical processes, incompatible data systems, and dead forests of paperwork. Health care administration in America is a tower of Babel that reaches to the moon....

It seems impossible to keep track of all the administrative, managerial and bureaucratic organizations that now impact health care. All of them, however, seem greatly susceptible to mismanagement, and worse.

Here is the latest example, from the Baltimore Sun,

A state contractor tampered with Maryland's cancer registry, a database used by researchers to track the disease's impact, counting hundreds of patients as having cancer when they did not, according to a legislative audit released yesterday.

The company, Macro International Inc., found in an internal investigation that data were deliberately altered between August 2004 and December of that year. The company fired the employee responsible for the cancer registry. State officials said that Macro employees apparently overreported the incidence of cancer to ensure that the database met standards set by a national certification association, which closely monitors registries to ensure that states have a complete count of cases.

The misinformation led researchers to send an estimated 400 women letters beginning in 2005 asking them to participate in a cervical cancer study when they did not have the disease. About 10 of those women called the state Family Health Administration, part of the state's Department of Health and Mental Hygiene - one of the first indications that the cancer registry was inaccurate.

Guy Garnett, a Macro vice president, declined to comment, citing company policy not to discuss client issues. The company completed the contract term, earning $1.9 million to manage the registry for 18 months through January. The state hired a new vendor, Westat, beginning in February.

In 2003, when Macro submitted the 2001 cancer database to the North American Association of Central Cancer Registries, it did not receive certification as required under its contract. Auditors faulted the state agency for not taking action, such as terminating the contract.

The next year, Macro did receive certification, but the auditors noted that information was later found to be erroneous. Auditors outlined several red flags that were not raised until years later.

First, the state agency did not review the 2002 data until August 2005, when it found they showed a 90 percent increase in cervical cancer cases and a 70 percent increase in melanoma cases from 1998 to 2002. At the time, auditors said, Macro could not provide an adequate explanation for the staggering rise in cancer incidence.

Around the same time, as part of the state health department's study of breast and cervical cancer diagnosis in Maryland women, about 2,300 patients in the cancer registry were asked to participate in the study. The Family Health Administration later worked with Macro to comb through medical histories, comparing original laboratory reports with data in the cancer registry, and determined that 400 of them did not have cancer. The study was suspended.

Then in May 2006, a former Macro employee informed the state agency that data were deliberately altered. Macro's investigation found that more than 13 percent of all cases in 2002 showed signs of tampering, according to the audit. Macro concluded that the changes were 'methodical and were made by one or more persons with broad access to the system,' the audit said.

I am of course familiar with the concept of a state-wide cancer registry. I must say it did not previously occur to (even) me that states would out-source the operation of such registries, and if so, that they would not bother to supervise the companies entrusted with collecting and analyzing the data.

Macro International Inc, is a for-profit, apparently privately held company. (I cannot find it on the Edgar database.) It boasts that its"mission is to deliver high-quality, research-based solutions to complex problems, integrating objective information with the advisory and implementation tasks needed to improve real world performance." That wording is so full of buzz words and management/ bureaucratic-speak that I am not sure what it means. So I cannot accuse Macro International's performance of violating its own mission. But it certainly violated the values of public health and health care research.

So we have yet another example of how health care activities have been transferred from people and organizations which were supposed to be fulfilling a (in this case, public) health mission, to (in this case, for-profit) organizations which do not have an overt commitment to the mission, a governance structure designed to fulfill the mission, or effective oversight or regulation to ensure that they fulfill the mission.

So the result was mismanagement, and fabricated data. The latter had direct impact on health care research, and on individual patients.

To take back the future of health care, we will have to ensure that health care is carried out by people and organizations pledged to uphold its core values, and working transparently and accountably under explicit and enforceable codes of ethics.

References

1. J.D. Kleinke. Oxymorons: the myth of a US health care system, Jossey-Bass, San Francisco (2001).

2. Poses RM. A cautionary tale: the dysfunction of the American health care system. Eur J Int Med 2003; 14: 123-130.

Criticism of Surrogate Endpoints in Whose Interests?

Last week, in an op-ed in the Wall Street Journal, Mark Thornton, lit into an inquiry launched by US Senator Charles Grassley (R-Iowa). Dr Thornton's concern was that Grassley was attacking the use of surrogate endpoints in clinical trials of drugs used to treat life-threatening conditions. Thornton was described as a "former medical officer in the Office of Oncology Products at the Food and Drug Administration," who also volunteers as "president of the Sarcoma Foundation of America."

Before discussing Dr Thornton's points, let me take a brief detour to explain surrogate endpoints. Ideally, physicians ought to prescribe treatments to alleviate pain and suffering, improve function, and delay death. So ideally treatments ought to be assessed on the degree they affect such endpoints. Often, however, trials designed to measure such endpoints are difficult, and often require prolonged observation of multiple patients. Thus, surrogate endpoints, which are those thought to correlate with such clinical endpoints, are studied instead.

Here is what Dr Thornton wrote about such endpoints:

At issue is the concept of 'surrogate endpoints' and the FDA's 'accelerated approval' regulations.

New laws and regulations ... created an accelerated approval mechanism by which a drug could be allowed on the market if it showed early evidence of an effect on a surrogate endpoint. For cancer, examples of surrogate endpoints are tumor shrinkage or a delay in the disease's progression.

This kind of measurement – as opposed to an assessment of a drug's impact on a patient's overall survival – has dramatically increased the pace of cancer clinical trials. It also has won near-universal acceptance within the cancer community.

The FDA does require follow-on studies to assure that a surrogate finding shows clinical benefit. But if all cancer clinical trials were required to show a survival benefit from the get-go, progress in cancer-drug development would slow to an absolute crawl.

The damage done by Mr. Grassley's decision to make an issue of this decision cannot be understated.

U.S. cancer-drug development stands on a precipice overlooking a new dark age in which each new product's development is longer and costlier than the last. Companies may decide it is not financially viable to even bother developing new drugs, and the pipeline for new products to treat cancer could slow even more. Mr. Grassley's legacy could be thousands of additional cancer deaths.

I cannot comment on the political or bureaucratic issues raised.

But I do question Dr Thornton's uncritical acceptance of surrogate endpoints, and his assertion that any questioning of the use of such endpoints could lead to death and disaster.

The problem with surrogate endpoints is that they are surrogates for the real thing. In many cases, a treatment may appear beneficial when measured by its affect on such endpoints, but not turn out to be beneficial when measured by its affect on real clinical outcomes, e.g., alleviation of symptoms, improvement of function, and prolongation of survival. There are many reasons why this may be the case.

Consider, for example, cancer "progression-free surival," cited by Dr Thornton in his article. Progression-free survival means the time a patient survives without measurable evidence that his or her tumor has gotten worse. There are some obvious reasons why progression-free survival may not correlate with overall survival. First, a tumor may progress but its progression may go undetected. Second, although a treatment may inhibit progression for a while, a tumor could escape its effects, and thereafter progress more rapidly than it did before. Third, a treatment might retard tumor growth, but cause severe adverse effects that hasten death on their own.

Thus, the FDA usually requires studies not only of progression free survival, but eventually of overall survival. Approving a drug only on the basis of its effects on progression-free survival risks allowing the use of a drug that may eventually prove to have no effect on overall survival, and hence whose benefits may not outweigh its harms.

So, regardless of the politics involved, Dr Thornton's dire warning that Senator Grassley's inquiry could lead to "thousands of additional cancer deaths" appears way over the top.

Incidentally, soon after the article came out, Ed Silverman on the Pharmalot blog reported that Dr Thornton, in addition to his voluntary post with the Sarcoma Foundation of America, serves as Senior Vice President of Product Development for the biotechnology firm GenVec, which is attempting to develop drugs to treat cancer. Also, the Sarcoma Foundation of America is supported in part by pharmaceutical and biotechnolgy companies including Ariad, Ziopharm, Bristol Myers Squibb, Ortho Biotech, and Novartis.

So readers of Dr Thornton's op-ed on health policy vis a vis how treatments for cancer and other life-threatening diseases are evaluated by the FDA deserved to know that Dr Thornton works for a biotech companies whose products could end up being evaluated by the FDA, and that Dr Thornton leads a not-for-profit organization which appears to be sponsored in part by other drug and biotech companies whose products also are being evaluated by the FDA.

Whether or not these firms' concerns that limitation of the use of surrogate endpoints in drug approval could affect their profts affected Dr Thornton's thinking is unknown. However, this question would not likely have even occurred to readers who did not know about Dr Thornton's ties, direct and indirect, to these firms.

In lieu of such disclosures, Dr Thornton's original op-ed, appears to be a new and prominent example of stealth health policy advocacy.

Health care is in crisis in many ways. Policy-level solutions deserve to be discussed widely by all sorts of people representing all sorts of opinions, with all sorts of expertise and backgrounds. However, the discussion would be a lot more productive if those taking part were completely transparent about who they are, for whom they work, and whose interests they may serve.

Cancer Screening, CT Scans, and Patent Applications

We are getting to this story a bit late, but perhaps can provide some new insight.

In 2006, an article that supported the value of CT scans to screen for lung cancer caused quite a stir. [The International Early Lung Cancer Action Program Investigators. Survival of patients with stage I lung cancer detected on CT screening. N Engl J Med 2006; 355: 1763-1771. Link here.]

The article reported a case series of over 30,000 patients who were entered into a program using annual CT scans to screen for lung cancer. Over 10 years of enrollment, 484 patients were found to have cancer. The investigators estimated the10-year lung-cancer specific survival of these patients was 80%, concluding "annual spiral CT screening can detect lung cancer that is curable."

A few days later, the article's lead author, Dr Claudia Henschke, was quoted in the New York Times as not understanding why CT screening was not the standard of care, "'I don’t get what the resistance is,' Dr. Henschke said. To her, it is a matter of simple logic: the earlier cancer is found, the better the odds of a cure. CT finds lung cancer early. So why not use it?"

In line with that sentiment, advocacy for widespread screening using CT scanning has continued under the auspices of the Lung Cancer Alliance. (See their news release, "Lung Cancer Alliance stands behind CT screening for lung cancer" from last fall.) The LCA asserted, "the mounting evidence which continues to mature from the largest international, protocol-driven screening effort (I-ELCAP) [is] showing that CT screening in a high risk population has the potential to reverse the current 15% five year survival rate." Furthermore, as reported in the May 4, 2007 edition of the Cancer Letter, legislation providing funding for screening using I-ELCAP protocols has been introduced in several state legislatures.

Of course, this seems to be another example of the introduction of an expensive, high-technology intervention that could save lives, but once again driving up the already staggering cost of health care. But these sorts of advances seem inevitable, don't they?

Maybe not. Dr Henschke's enthusiasm for screening seemed to go well beyond the data her study provided. In fact, the multiple flaws of the study were nicely summarized in a special article in the Archives of Internal Medicine [Welch HG, Woloshin S, Schwartz LM et al. Overstating the evidence for lung cancer screening: the International Early Lung Cancer Action Program (I-ELCAP) study. Arch Intern Med 2007; 167: 2289-2295. Link here.]

• A Case-Series, not a Randomized Controlled Trial - Because the study did not contain a control group, it is impossible to tell whether its apparently favorable results were due to the intervention, to the selection of patients with unusually good prognoses, or to chance. As Welch et al put it, "The I-ELCAP study design is a case series. Because there was no control group, readers cannot know what would have been observed in the absence of screening."
• Lead-Time Bias - The major result of the NEJM study was that patients with cancer seemed to live longer after their diagnosis than experience suggests is the norm for lung cancer patients. However, this could have been due to lead-time bias. Per Welch et al, "because survival is measured from the time of diagnosis, any screening test that advances the time of diagnosis will bias the measure of survival—the lead-time bias." For example, consider a person who developed cancer at age 60, was diagnosed at age 65, and died at age 66. His survival after diagnosis was one year. But if he had been diagnosed at age 60, and died at age 66, his survival after diagnosis would have been six years, but his life-span would not have been altered.
• "Overdiagnosis Bias" - This means that more intensive efforts at diagnosis may reveal cases of disease with more favorable prognoses. For example, it is well known that some patients who appear to have early prostate cancer will never have progressive disease. But detecting such patients by screening will not improve their survival. Welch et al noted that previous studies that used chest x-rays to screen for lung cancer demonstrated improved survival after diagnosis, presumably as a result of combined lead-time and overdiagnosis biases, since overall survival was not improved in patients who were screened compared to controls.
• Failure to Assess Harms of Screening and Subsequent Treatment - Welch et al noted problems with how the I-ELCAP study assessed deaths due to surgical treatment of cancer. The study also failed to report adverse effects of invasive diagnostic efforts for patients who had positive CT scans, most of whom did not prove to have cancer. For example, needle biopsy of lesions found by CT scan can cause pneumothorax (lung collapse), but these events were not reported in the NEJM article.

Also, a series of letters to the editor of the NEJM brought up similar concerns, plus additional flaws, e.g., although the I-ELCAP study reported 10-year disease specific survival, it only followed patients for a median of 40 months, and less than one-fifth of subjects were followed for five years or more. A response by lead author Henschke and two other authors failed to reassure. It first seemed to simply deny the importance of lead-time bias,

Screening for cancer is supposed to provide for lead time in diagnosis and treatment. A bias is introduced when one compares relatively short-term survival rates as of diagnosis to assess the effectiveness of treatment with lead time relative to treatment without lead time. We did not do this.

Henschke et al also seemed to misunderstand overdiagnosis bias. They responded to the relevant critique by asserting that all cases of cancer were confirmed by pathology. This begged the question, because pathological diagnosis does not necessarily determine whether a tumor will progress.

Finally, they seemed to deny that randomized controlled trials are necessary to assess screening programs.

Berg and Aberle remark that our study had no comparison group, but unlike interventional research, diagnostic research does not inherently require a comparison, much less a comparison group. The attainable frequency of diagnoses of stage I tumors can be assessed only in the framework of screening, and our principal diagnostic result was the 85% frequency of diagnoses of clinical stage I tumors among all diagnoses. The principal interventional result was the 92% cause-specific 10-year survival rate after prompt treatment of stage I tumors. The relevant comparison group would consist of patients who received early diagnoses but were treated late, principally to learn about the timing of deaths from lung cancer.

Every scientist tends to be an enthusiast about his or her own work, but I was struck by how Henschke et al seemed to shrug off their study's obvious methodologic problems.

(It might seem to be beating a dead horse, but I found even more problems with the I-ELCAP article beyond those noted above: failure to describe the clinical characteristics of the patient population; not blinding assessment of whether deaths were cancer-related; failure to have a credible method to verify that patients who did not have a positive CT scan did not have cancer; and failure to report loss to follow-up.)

More concerns about the origins of the I-ELCAP investigators' enthusiasm, which seems to have infected such groups as the Lung Cancer Alliance, were raised by a new report in the Cancer Letter, (from Jan 18, 2007, but not available without a subscription, and just made available to me last week),

The leaders of a research consortium that advocates computed tomography scanning to detect early-stage lung cancer have built an estate of 27 patent applications worldwide covering technologies and methods of screening.

According to publicly available databases, the leaders of the International
Early Lung Cancer Action Project are listed as inventors on patent applications
and one issued U.S. patent that cover innovations in lung cancer screening,
from clinical trial methodology, to software for interpretation of scans, to
technology of biopsy needles.

The first of these patent applications was submitted on April 11, 2000. In the U.S., group leaders are listed as inventors on one patent and 10 published applications. At least one of the inventions has been licensed by General Electric, a maker of CT scanners, and, in another instance, patent rights were reportedly exchanged for stock of a start-up company that markets lung biopsy devices.

Disclosure statements that accompanied the publications by the group’s leaders didn’t reflect these activities as potential conflict of interest, a literature
search shows.

Furthermore,

Both GE and Cornell declined to disclose the details of a licensing agreement covering Henschke’s and Yankelevitz’s inventions.

'We do have a license agreement with the Cornell Research Foundation for certain CT lung cancer patents,' Corey Miller, a spokesman for GE Healthcare Americas,
said to The Cancer Letter.

'The researchers—Henschke and Yankelevitz— have received some royalty payments from GE for a computer algorithm equation that they developed to detect lung cancer on diagnostic scans,' Miller said. 'That is, as far as I know, the extent of our relationship with those two.'

Note that the 2006 NEJM article included no disclosures of conflicts of interest.

Finally, the Cancer Letter noted that I-ELCAP investigators have worked actively with the Lung Cancer Alliance to promote screening.

As they spar with skeptics, I-ELCAP scientists work closely with LCA, a group that advocates for screening former and current smokers in accordance with I-ELCAP protocols.

So here we go again. Flawed research that supports an expensive, high-technology intervention for which millions of people would be eligible appears in a major journal. The intervention is hyped in news articles, and by a grass roots patient advocacy group. Politicians soon jump on the bandwagon. Citing the potential for saving lives, enthusiasts ignore the flaws in the data that started it all. Only much later do we find out that the researchers who wrote the flawed study may stand to make money if the high-technology intervention is widely adopted.

At the minimum, this case suggests that academic medicine's current method of managing conflicts of interest through disclosure is not working well. Further, in the absence of such disclosure, it seems that individual clinical decisions and public policy may too often be based on apparently unbiased research that in fact is being purveyed not merely by enthusiasts, but by people who may make more money when the results turn out a certain way.

People have been wringing their hands about ever rising health care costs for a long time. Expensive high-technology interventions are often cited as responsible, but not further addressed, apparently due to the assumption that their costs are an inevitable reflection of their advantages. Yet maybe this is not so inevitable. We may have placed to much credence in research that exaggerated the value of high tech gizmos because it was done by by people who stood to gain from the sale of such gizmos.

At the very least, we need to make sure we know when clinical research to evaluate screening methods, diagnostic tests, drugs, or devices was done by people with financial interests in their success. It might be, however, that the only way to get unbiased evaluations of drugs and devices is to have the evaluation done by people who have no financial interest in the results turning out one way or the other.

See this editorial in the Boston Globe by Dr Jerry Kassirer, and this post in the WSJ Health Blog for earlier discussion of what the Cancer Letter revealed.

ADDENDUM (11 March, 2008) - See also previous comments after publication of article by Welch et al by Merrill Goozner in GoozNews blog.