I have not written much about the seemingly endless health care reform debate in the US, because much of it has not been relevant to the issues we discuss on Health Care Renewal. Now that the current phase of the debate is done, and legislation has been passed, let me offer my opinions on the few aspects that do seem relevant to this blog.
The Sunshine Act
For Health Care Renewal readers, the most important part of the legislation is that containing the provisions of the Sunshine Act, championed by Senators Grassley and Kohl. (See this summary on Postscript, the Prescription Project blog.) The act requires that all drug, device, biologic, and medical supply manufacturers report essentially all payments to physicians or teaching hospitals to the goverment, and on the internet. It does not appear that the rules apply to other health care related non-profit organizations, e.g., medical schools, disease advocacy groups, health care related charities, medical societies, etc, or to payments made by for-profit health insurers, clinical research organizations, and some other corporations. Unfortunately, the provisions only take effect in 2013. However, despite these quibbles, this still may be one of the most important advances promoting disclosure of health care related conflicts of interest made in the 21st century.
Comparative Effectiveness Research
As best as I can tell at this point, the current legislation used the wording from the bill previously passed in the US Senate, which we discussed here and here, regarding comparative effectiveness research. Although its goal of setting up a not-for-profit comparative effectiveness organization seems laudable, the devil will be in the details. The Senate version gave considerable oversight of this organization to those with vested interests in selling particular products or services, threatening the impartiality of the organization and the research it would sponsor, and perhaps thus wholly defeating its ostensible purpose. Furthermore, the Senate bill included curious wording that seems to threaten the ability of those getting funding from the organization to express views that might disturb the organization's leadership, again threatening the integrity of their dissemination of its work, and perhaps violating the First Amendment of the US Constitution. Whether these provisions provide benefits that outweigh their harms is highly questionable.
Payments to Physicians
We have criticized how the process of setting payments to physicians by the US Medicare system has been captured by a secretive committee of the American Medical Association that is dominated by physicians who do procedures, the RBRVS Update Committee, or RUC. The results have been payments for primary care and other cognitive services that have failed to keep up with inflation, a major cause of the continuing decline of generalist/ primary care medicine in the US. (See most recent post here about this.) According to the summary provided by the American College of Physicians (here), the new legislation would enable review of payments made for specific services, and would reconsideration of the process used to set physician payments by an independent advisory group. However, the bill would not mandate any changes in payments, or in the processes used to set them, including the pivotal role of the RUC. So there is some chance that the legislation would lead to a more transparent, accountable, honest, and rational process for setting physician payments and hence eliminating perverse incentives, but no guarantee of such favorable changes.
Summary
The legislation seemingly will result in one major advance fostering disclosure of some conflicts of interest, and perhaps some progress in terms of reducing perverse incentives generated by Medicare's payments to physicians, and possibly reducing regulatory capture of this process. It likely will result in more comparative effectiveness research, but how badly it will be biased in favor of vested interests is unclear. As far as I can tell, the legislation will leave most of the other problems we discuss on Health Care Renewal untouched. We thus have one or two small steps for mankind, but no reason for complacency.
the news is not bad, but we are still a long way from meaningfully addressing concentration and abuse of power in health care. There will be no rest for the weary bloggers of Health Care Renewal.
Also, see comments here and here by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma blog.
ADDENDUM (25 March, 2010) - Also see comments on the Sunshine Act by Alison Bass on the Alison Bass Blog.
Showing posts with label comparative effectiveness research. Show all posts
Showing posts with label comparative effectiveness research. Show all posts
Wednesday, March 24, 2010
Monday, January 18, 2010
Not Just an American Disease
We have written about attacks on rigorous evidence-based medicine, and particularly on comparative effectiveness research from those with vested interests in having clinical research come out a certain way (e.g., see this most recent relevant post). Those who see such research primarily as a marketing opportunity tend to be offended by the notion of rigorous, unbiased research that may not be so easily turned to marketing purposes. Since I, like the other current Health Care Renewal bloggers, am based in the US, we tend to focus on local examples. But it turns out that the American malady described above has spread to Germany.
From the Science blog, ScienceInsider:
And it turns out that the American-based pharmaceutical industry has jumped right in.
Parenthetically, "innovation" seems to be a favorite term that those with vested interests in selling products or services use to describe those products, sometimes in the absence of any data that shows them to be superior to the alternatives in terms of important clinical outcomes, that is, outcomes that patients may care about. "Innovative" was also how complex financial products which contributed to the global economic meltdown were described by those who stood to make money selling, or sometimes simultaneously short-selling them, - but maybe that's guilt by association.
I hope the Germans are able to preserve their stake in honest, comparisons of tests and treatments that are not influenced by those with vested interests in selling those tests and treatments.
From the Science blog, ScienceInsider:
A long-running feud between pharmaceutical companies and the German institute that evaluates the effectiveness of medical treatments could cost the institute director his job. Although the post is supposed to be apolitical, members of Germany’s new coalition government have called for Peter Sawicki, founding director of the Institute for Quality and Efficiency in Health Care (known by its German acronym IQWiG, pronounced ICK-vig), to be replaced with someone who is friendlier to the pharmaceutical industry. The institute’s board of directors are expected to decide on 20 January whether Sawicki, a clinical researcher and diabetes expert, will be replaced when his contract runs out later this year.
Sawicki’s supporters say the move would endanger the institute’s reputation for impartial and rigorous science, and earlier this month a petition signed by 600 doctors and clinical researchers called on the health minister and the board to keep Sawicki on. Gerd Antes, director of the German Cochrane Centre in Freiburg, a not-for-profit organization that analyzes health care effects, says that replacing Sawicki would significantly undermine IQWiG and its work. Antes views the anti-Sawicki push as 'part of the political game to soften and to weaken rigorous procedures for new drugs and medical devices in Germany.'
And it turns out that the American-based pharmaceutical industry has jumped right in.
Big pharma’s attacks have even come from outside Germany. In March 2009, the Pharmaceutical Research and Manufacturers of America petitioned the Obama Administration to put Germany on a trade and intellectual property 'priority watch list' chiefly because of IQWiG’s influence on the German drug market. The petition complained that the institute has 'inadequately taken into account the value of innovative pharmaceuticals,' among other complaints. The Obama Administration declined to put Germany on its watch list.
Parenthetically, "innovation" seems to be a favorite term that those with vested interests in selling products or services use to describe those products, sometimes in the absence of any data that shows them to be superior to the alternatives in terms of important clinical outcomes, that is, outcomes that patients may care about. "Innovative" was also how complex financial products which contributed to the global economic meltdown were described by those who stood to make money selling, or sometimes simultaneously short-selling them, - but maybe that's guilt by association.
I hope the Germans are able to preserve their stake in honest, comparisons of tests and treatments that are not influenced by those with vested interests in selling those tests and treatments.
Tuesday, November 24, 2009
No Free Speech for Comparative Effectiveness Researchers?
We have repeatedly argued why comparative effectiveness research, under ideal circumstances, would be a good idea. As I said before:
Those opposed to the sort of comparative effectiveness research I described above then seemingly decided, "if you can't beat 'em, join 'em." Thus, a provision appeared in a recent version of health care reform legislation proposed in the US Senate for comparative effectiveness research to be sponsored by an "independent" institute whose board of directors would have to include a substantial minority of representatives of industry (that is, drug, biotechnology, device, health insurance corporations, and other corporations as "payers.") This would seems to be a fairly shameless form of "regulatory capture," that is, an instance in which a government agency whose mission seems to be to improve health care is "captured" by those with vested interests in promoting certain health care products and services. (See post here.)
My concern has now seemingly gone mainstream, in that it was addressed in a commentary published on-line in the prestigious New England Journal of Medicine. [Selker HP, Wood AJJ. Industry influence on comparative-effectiveness research funded through health care reform. N Engl J Med 2009. Link here.]
Selker and Wood addressed the issue of regulatory capture thus.
Note that the situation is worse considering that the insurance industry and other "payers" also have seats on the board.
However, Selker and Wood discovered an even more outrageous provision:
As Selker and Wood noted, it is unprecedented for a US government agency that is meant to sponsor science to be empowered to punish researchers for conclusions or opinions with which the agency disagrees. This suggests that the new agency would be meant to produce only results that support the vested interests of its leadership, that is, that favor the latest, and most expensive drugs and devices. The research sponsored by such an agency would not only be biased, it would likely be of poor quality, because researchers of integrity would likely avoid sponsorship by an agency that would be so threatening to their scientific independence.
This part of the bill does not promote health reform, but blatantly attempts to serve health care corporations while sacrificing the interests of patients and doctors.
As Selker and Wood politely put it:
It seems to be almost gilding the lilly to note that the provision cited above seems to violate the free speech and free press provisions of the 1st amendment of the US Constitution, since they threaten government punishment of private citizens (e.g., by withdrawal of existing funding) purely for speech that the government does not like.
So I ask the anonymous Senate aide who drafted this provision, and the anonymous lobbyist(s) who influenced him or her, have they no shame?
Finally, I have yet to see coverage of the Selker and Wood article in the mainstream media. I hope they will eventually conclude that this attempt to co-opt clinical science and mock the 1st amendment is actually news and comment worthy.
Physicians spend a lot of time trying to figure out the best treatments for particular patients' problems. Doing so is often hard. In many situations, there are many plausible treatments, but the trick is picking the one most likely to do the most good and least harm for a particular patient. Ideally, this is where evidence based medicine comes in. But the biggest problem with using the EBM approach is that often the best available evidence does not help much. In particular, for many clinical problems, and for many sorts of patients, no one has ever done a good quality study that compares the plausible treatments for those problems and those patients. When the only studies done compared individual treatments to placebos, and when even those were restricted to narrow patient populations unlike those patient usually seen in daily practice, physicians are left juggling oranges, tomatoes, and carburetors.However, when comparative effectiveness research was proposed as an element of US health care reform, it was attacked as a vehicle for the dreaded rationing of health care (even though in the US health care is already rationed, especially to those without generous insurance or the means to pay for expensive tests and treatments), using arguments based more on emotions, or outright fallacies than on logic and evidence. For example, see our blog posts here, here, here, and here.
Comparative effectiveness studies are simply studies that compare plausible treatments that could be used for patients with particular problems, and which are designed to be generalizable to the sorts of patients usually seen in practice. As a physician, I welcome such studies, because they may provide very useful information that could help me select the optimal treatments for individual patients.
Because I believe that comparative effectiveness studies could be very useful to improve patient care, it upsets me to see this particular kind of clinical study get caught in political, ideological, and economic battles.
Those opposed to the sort of comparative effectiveness research I described above then seemingly decided, "if you can't beat 'em, join 'em." Thus, a provision appeared in a recent version of health care reform legislation proposed in the US Senate for comparative effectiveness research to be sponsored by an "independent" institute whose board of directors would have to include a substantial minority of representatives of industry (that is, drug, biotechnology, device, health insurance corporations, and other corporations as "payers.") This would seems to be a fairly shameless form of "regulatory capture," that is, an instance in which a government agency whose mission seems to be to improve health care is "captured" by those with vested interests in promoting certain health care products and services. (See post here.)
My concern has now seemingly gone mainstream, in that it was addressed in a commentary published on-line in the prestigious New England Journal of Medicine. [Selker HP, Wood AJJ. Industry influence on comparative-effectiveness research funded through health care reform. N Engl J Med 2009. Link here.]
Selker and Wood addressed the issue of regulatory capture thus.
Although most observers agree on the value of funding CER, many are unaware that embedded in the legislation are provisions ceding substantial influence to the medical products industries that have a major interest in the outcomes of such research.
The Senate Finance Committee bill mandates the creation of an entirely new private–public research entity and, owing to industry lobbying, guarantees industry three seats on this entity’s 15-member governing board, as well as representation on its methodology committee
Note that the situation is worse considering that the insurance industry and other "payers" also have seats on the board.
However, Selker and Wood discovered an even more outrageous provision:
The Finance Committee bill also includes language requested by industry lobbyists (pages 1138–1139) that threatens to withdraw federal funding for 5 years from any investigator who publishes a report on research funded by the proposed institute that is not within the bounds of and entirely consistent with the evidence.' Determinations regarding such consistency would be made by the newly created research entity, which would have industry involvement both in its governance and in study design. To allow scientists — and their institutions, which receive the support for the conduct of research — to be punished for the publication of work that is not approved by this entity is essentially to cede authority over the dissemination of government-funded research to a body that is at least partially controlled by persons with a potential commercial interest in its outcome.
As Selker and Wood noted, it is unprecedented for a US government agency that is meant to sponsor science to be empowered to punish researchers for conclusions or opinions with which the agency disagrees. This suggests that the new agency would be meant to produce only results that support the vested interests of its leadership, that is, that favor the latest, and most expensive drugs and devices. The research sponsored by such an agency would not only be biased, it would likely be of poor quality, because researchers of integrity would likely avoid sponsorship by an agency that would be so threatening to their scientific independence.
This part of the bill does not promote health reform, but blatantly attempts to serve health care corporations while sacrificing the interests of patients and doctors.
As Selker and Wood politely put it:
If health care reform legislation does not promote CER that is free of the potential taint of commercial and political meddling, the public will have little confidence in the results of such research. This outcome would be extremely unfortunate, since such research has the potential to improve patients’ lives by leading to more effective medical care. The U.S. biomedical research enterprise has a long and storied history that has made it a model for other countries. It would be a tragedy if we were to squander its achievements for political expediency, in the service of short-term commercial interests. The current proposals for controlling CER in a manner unlike anything we have seen in federally sponsored biomedical research therefore should be rejected.
It seems to be almost gilding the lilly to note that the provision cited above seems to violate the free speech and free press provisions of the 1st amendment of the US Constitution, since they threaten government punishment of private citizens (e.g., by withdrawal of existing funding) purely for speech that the government does not like.
So I ask the anonymous Senate aide who drafted this provision, and the anonymous lobbyist(s) who influenced him or her, have they no shame?
Finally, I have yet to see coverage of the Selker and Wood article in the mainstream media. I hope they will eventually conclude that this attempt to co-opt clinical science and mock the 1st amendment is actually news and comment worthy.
Monday, October 26, 2009
Who Should Sponsor Comparative Effectiveness Research?
We have tried to argue why comparative effectiveness research is a good idea. To cut and paste what I wrote in a previous post,
In particular, we have discussed a number of high profile attacks on comparative effectiveness research, which often have featured arguments based on logical fallacies. While some of the people making the attacks have assumed a conservative or libertarian ideological mantle, one wonders whether the attacks were more driven by personal financial interests. For example, see our blog posts here, here, here, and here. On the other hand, we discussed a clear-headed defense of comparative effectiveness research by a well-known economist most would regard as libertarian here.
Comparative effectiveness research has been discussed as an element of health care reform in the US. It turns out that the current version of the health care reform bill in the US Senate has a provision to create a Patient Centered Outcome Research Institute, which presumably would become the major organization which could sponsor comparative effectiveness research.
This institute, however, would not be a government agency (despite the name that makes it sound like it would be part of the National Institutes of Health). Moreover, here is a description of the Board of Governors who would run the institute from the current version of the bill :
Thus, only 3/15 members of the governing board would represent the patients who ultimately reap the benefits or suffer the harms produced by medical diagnosis and treatment. Further, 6/15 members represent for-profit corporations which stand to make more or less money depending on how particular comparative effectiveness studies come out. Also, 3/15 members would be physicians, some of who may get paid more to deliver particular treatments (e.g., procedures) than others (e.g., providing advice about diet and exercise).
We often discuss how clinical research sponsored by organizations with vested interest in the research turning out to favor their products or services may be manipulated to favor these interests, and sometimes suppressed if it does not. In the US, there are few unconflicted sources of sparse funds to support comparative effectiveness research. (The most significant current source is the Agency for Healthcare Research and Quality, AHRQ. For full disclosure, I have been an ad hoc reviewer of grants for that agency.)
The current draft of legislation would create the largest potential sponsor for comparative effectiveness research, but would make that organization report to representatives of for-profit companies whose profits may be affected by the results of such research. In my humble opinion, this is not much of an advance. Comparative effectiveness research controlled by corporations that stand to profit or lose depending on its results will forever be suspect.
If the government is going to support comparative effectiveness research, it ought to make sure such research is not run by people with vested interests in the outcomes coming out a certain way. I may be biased myself, but why not let the research be sponsored by AHRQ, an agency with relevant experience and no axe to grind vis a vis any particular product or service?
Physicians spend a lot of time trying to figure out the best treatments for particular patients' problems. Doing so is often hard. In many situations, there are many plausible treatments, but the trick is picking the one most likely to do the most good and least harm for a particular patient. Ideally, this is where evidence based medicine comes in. But the biggest problem with using the EBM approach is that often the best available evidence does not help much. In particular, for many clinical problems, and for many sorts of patients, no one has ever done a good quality study that compares the plausible treatments for those problems and those patients. When the only studies done compared individual treatments to placebos, and when even those were restricted to narrow patient populations unlike those patient usually seen in daily practice, physicians are left juggling oranges, tomatoes, and carburetors.
Comparative effectiveness studies are simply studies that compare plausible treatments that could be used for patients with particular problems, and which are designed to be generalizable to the sorts of patients usually seen in practice. As a physician, I welcome such studies, because they may provide very useful information that could help me select the optimal treatments for individual patients.
Because I believe that comparative effectiveness studies could be very useful to improve patient care, it upsets me to see this particular kind of clinical study get caught in political, ideological, and economic battles.
In particular, we have discussed a number of high profile attacks on comparative effectiveness research, which often have featured arguments based on logical fallacies. While some of the people making the attacks have assumed a conservative or libertarian ideological mantle, one wonders whether the attacks were more driven by personal financial interests. For example, see our blog posts here, here, here, and here. On the other hand, we discussed a clear-headed defense of comparative effectiveness research by a well-known economist most would regard as libertarian here.
Comparative effectiveness research has been discussed as an element of health care reform in the US. It turns out that the current version of the health care reform bill in the US Senate has a provision to create a Patient Centered Outcome Research Institute, which presumably would become the major organization which could sponsor comparative effectiveness research.
This institute, however, would not be a government agency (despite the name that makes it sound like it would be part of the National Institutes of Health). Moreover, here is a description of the Board of Governors who would run the institute from the current version of the bill :
BOARD OF GOVERNORS.—
(1) IN GENERAL.—The Institute shall have a Board of Governors, which shall consist of 15 members appointed by the Comptroller General of the United States not later than 6 months after the date of enactment of this section, as follows:
(A) 3 members representing patients and health care consumers.
(B) 3 members representing practicing physicians, including surgeons.
(C) 3 members representing private payers, of whom at least 1 member shall represent health insurance issuers and at least 1 member shall represent employers who self-insure employee benefits.
(D) 3 members representing pharmaceutical, device, and diagnostic manufacturers or developers.
(E) 1 member representing nonprofit organizations involved in health services research.
(F) 1 member representing organizations that focus on quality measurement and improvement or decision support.
(G) 1 member representing independent health services researchers.
Thus, only 3/15 members of the governing board would represent the patients who ultimately reap the benefits or suffer the harms produced by medical diagnosis and treatment. Further, 6/15 members represent for-profit corporations which stand to make more or less money depending on how particular comparative effectiveness studies come out. Also, 3/15 members would be physicians, some of who may get paid more to deliver particular treatments (e.g., procedures) than others (e.g., providing advice about diet and exercise).
We often discuss how clinical research sponsored by organizations with vested interest in the research turning out to favor their products or services may be manipulated to favor these interests, and sometimes suppressed if it does not. In the US, there are few unconflicted sources of sparse funds to support comparative effectiveness research. (The most significant current source is the Agency for Healthcare Research and Quality, AHRQ. For full disclosure, I have been an ad hoc reviewer of grants for that agency.)
The current draft of legislation would create the largest potential sponsor for comparative effectiveness research, but would make that organization report to representatives of for-profit companies whose profits may be affected by the results of such research. In my humble opinion, this is not much of an advance. Comparative effectiveness research controlled by corporations that stand to profit or lose depending on its results will forever be suspect.
If the government is going to support comparative effectiveness research, it ought to make sure such research is not run by people with vested interests in the outcomes coming out a certain way. I may be biased myself, but why not let the research be sponsored by AHRQ, an agency with relevant experience and no axe to grind vis a vis any particular product or service?
Wednesday, June 17, 2009
A Clear-Headed Defense of Comparative Effectiveness Research
We have tried to argue why comparative effectiveness research is a good idea. To cut and paste what I wrote in a previous post,
In particular, we have discussed a number of high profile attacks on comparative effectiveness research, which often have featured arguments based on logical fallacies. While some of the people making the attacks have assumed a conservative or libertarian ideological mantle, one wonders whether the attacks were more driven by personal financial interests. For example, see our blog posts here, here, here, and here.
Therefore, it was refreshing to see this defense of comparative effectiveness research in the opinion pages of the New York Times, which demonstrated that the issues here are really not ideological.
This was written by Tyler Cowen, a well known academic economist, a Professor in that department at George Mason University with impeccable libertarian credentials. (Prof Cowen also blogs on Marginal Revolution.) Prof Cowen reminded us how the current health care reform debate could benefit from some clear thinking that eschews ideological posturing.
Physicians spend a lot of time trying to figure out the best treatments for particular patients' problems. Doing so is often hard. In many situations, there are many plausible treatments, but the trick is picking the one most likely to do the most good and least harm for a particular patient. Ideally, this is where evidence based medicine comes in. But the biggest problem with using the EBM approach is that often the best available evidence does not help much. In particular, for many clinical problems, and for many sorts of patients, no one has ever done a good quality study that compares the plausible treatments for those problems and those patients. When the only studies done compared individual treatments to placebos, and when even those were restricted to narrow patient populations unlike those patient usually seen in daily practice, physicians are left juggling oranges, tomatoes, and carburetors.
Comparative effectiveness studies are simply studies that compare plausible treatments that could be used for patients with particular problems, and which are designed to be generalizable to the sorts of patients usually seen in practice. As a physician, I welcome such studies, because they may provide very useful information that could help me select the optimal treatments for individual patients.
Because I believe that comparative effectiveness studies could be very useful to improve patient care, it upsets me to see this particular kind of clinical study get caught in political, ideological, and economic battles.
In particular, we have discussed a number of high profile attacks on comparative effectiveness research, which often have featured arguments based on logical fallacies. While some of the people making the attacks have assumed a conservative or libertarian ideological mantle, one wonders whether the attacks were more driven by personal financial interests. For example, see our blog posts here, here, here, and here.
Therefore, it was refreshing to see this defense of comparative effectiveness research in the opinion pages of the New York Times, which demonstrated that the issues here are really not ideological.
Drawing upon the ideas of the Harvard economist David Cutler, the Obama administration talks of empowering an independent board of experts to judge the comparative effectiveness of health care expenditures; the goal is to limit or withdraw Medicare support for ineffective ones. This idea is long overdue, and the critics who contend that it amounts to 'rationing' or 'the government telling you which medical treatments you can have' are missing the point. The motivating idea is the old conservative chestnut that not every private-sector expenditure deserves a government subsidy.
This was written by Tyler Cowen, a well known academic economist, a Professor in that department at George Mason University with impeccable libertarian credentials. (Prof Cowen also blogs on Marginal Revolution.) Prof Cowen reminded us how the current health care reform debate could benefit from some clear thinking that eschews ideological posturing.
Sunday, May 24, 2009
BLOGSCAN - Comparative Effectiveness Research, the Partnership to Improve Patient Care, and PhRMA
On the Hooked: Ethics, Medicine and Pharma blog, Dr Howard Brody dissected a campaign to redirect comparative effectiveness research by making it responsible to a new governing board that would include "insurance" and "industry" members. And surprise, surprise, the campaign is run by the Partnership to Improve Patient Care, a group that seems to have multiple connections to PhRMA, the pharmaceutical industry trade organization. More stealth health policy advocacy?
Thursday, April 9, 2009
Have we suffered a complete breakdown in the scientific method with regard to EHR and clinical IT?
Have we suffered a complete breakdown in the scientific method with regard to EHR and other clinical IT?
I read announcements like this with trepidation:
Recovery Act funds have been allocated to NIH specifically for comparative effectiveness research. NIH has further specified the definition of CER as:
NIH states that such research may include "the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcomes data as they apply to CER."
The problems I foresee concern the word "rigorous" as in the above definition.
The use of EHR data to reliably detect uncommon (but strong, discrete) early warning signals from a single drug or treatment -- to then be subject to more rigorous study with reasonable controls -- is itself a Medical Informatics "Grand Challenge." An example would be finding VIOXX's association with myocardial infarction earlier than we did, via an EHR-based automated postmarket surveillance process.
Doing this is a "grand challenge" due to the nature of EHR data, which is as far from "clinical trials clean" as possible. It is what might be called highly uncontrolled. The statistical methods needed to reliably pull signals out of the muck for even a single drug are still exploratory, the problems formidable if one wants to stay scientifically sound. I wrote about the experimental nature of such efforts a few years ago here, and believe an effort got underway at U. Indiana/Regenstrief to test such methodologies for postmarket surveillance about the same time.
Now we have had what appears to be a leap of faith and logic of irrationally exuberant proportions, and probably a deviation from sound science as well. The government has announced enthusiasm for EHR data-based comparative effectiveness research (CER) not to aid science, but to cut costs (implying skipping the rigorous confirmatory phases) through elimination of more costly drugs and treatments deemed less effective or at effectiveness parity compared to less expensive choices. Following this thinking, perhaps in the future a metric will be developed for an "acceptable" improved benefit/cost ratio for expensive drugs that are better than cheaper alternatives?
This overconfidence in EHR data is of concern. To detect relatively less concrete (i.e., than major ADE) "outcomes differences" between two or more drugs or treatments via EHR data - did treatment A lower blood pressure more than drug B, did drug C lessen depression more than drug D - rises to the level of "grand overconfidence in computing." To accomplish this task with reasonable scientific certainty from reams of EHR data, originating from different vendor systems, input by myriad people of different backgrounds with differing interpretations of terminologies (students/MD's/RN's etc) under different pressures (time, reimbursement maximization), and so forth, seems a stretch. What will the p values and predictive values be for such studies? Yet our incoming HHS secretary touts such methods?
Ironically, the gold standard in medical science is the controlled clinical trial, yet EHR-based comparative effectiveness research itself as a research methodology, now touted by our government, seems to have gotten a pass.
Even what I would consider minimum requirements for scientific treatment comparisons, such as well designed and reasonably controlled registries as developed here for interventional cardiology, with hundreds of granular, finely defined and "tuned" data elements, appear to be bypassed in EHR "miracle claims." Such precise registries take months or years to develop, implement, and train users to interact with properly. Further, such registries are not portable and must be created for individual medical domains and subdomains. Uncontrolled EHR data is no substitute for such efforts.
The following question arises:
Where are the comparative effectiveness studies that compare 1) EHR-based comparative effectiveness studies of drugs and treatments to 2) controlled clinical trials-based comparative effectiveness studies?
In other words, where are the meta-clinical trials that compare EHR data mining-based comparative effectiveness research as a methodology, vs. the "traditional" gold standard methodology of controlled clinical trials to compare drugs or treatments? How do we know EHR-based CER studies will not produce GIGO that will cause harm through ham-fisted elimination or defunding of useful treatment options?
While there are initial efforts underway to increase understanding of CER, e.g., "Broad Challenge Area 5" (PDF) of the NIH RC1 Challenge Grants in Health and Science Research, ominously, there is a lot of potential advantage to be had with terabytes of uncontrolled data and a political agenda.
I fear that what will come from "comparative effectiveness research" that draws upon uncontrolled EHR data will be politics masquerading as comparative effectiveness research. Good luck to private practitioners and medical innovators. Good luck, pharma. Good luck, patients.
This movement towards EHR uncontrolled data alchemy represents a further deviation from medical science towards the Syndrome of Inappropriate Over-Confidence in Computing (a.k.a. SICC Syndrome) writ large.
It seems the IT industry has now rendered a scientific approach to HIT and its use obsolete. We see this "post scientific era" phenomenon in the takeover of clinical IT by vendors who contractually demand suppression of sharing of problems, we see it in a remarkably uncritical push for EMR's by 2014 now involving force of government (only financial at present, but will punitive licensure issues and other measures be off the table?) despite a growing body of literature advising caution, we see a consortium of big business/payers/vendors/myriad secondary feeder organizations gunning full blast for this technology without consideration of the possible downsides.
Biomedical informatics, a scientific discipline (at least those parts of it not yet compromised by conflicts of interest), as a relevant field is very much a minority player in today's health IT.
Even the contributions from experts and pioneers in the field of Biomedical Informatics, in the form of the Jan. 2009 National Research Council's report that "Current Approaches to U.S. Health Care Information Technology are Insufficient" (here) has not had much impact.
I see Biomedical Informatics' death as a relevant discipline that anyone of importance pays attention to, not too far down the road as well.
-- SS
Addendum April 20:
We've seen this phenomenon in our economy. WSJ "Information Age" writer L. Gordon Crovitz notes:
I believe EMR data at best is a very loose approximation to the real world. It contains many "unknowns" regarding quality and reliability that cannot be turned into probabilities no matter how fancy the math. Asking too much of EHR data becomes a vice, not a virtue.
-- SS
I read announcements like this with trepidation:
http://govhealthit.com/articles/2009/03/31/sebelius-confirmation.aspx
“The goal,” Sebelius said, “is to provide every American with a safe, secure electronic health record by 2014." The nominee also endorsed efforts to use data gleaned from electronic medical records to conduct “comparative effectiveness research" (CER) to provide information on the relative strengths and weaknesses of alternative medical interventions to health providers and consumers.”
“The goal,” Sebelius said, “is to provide every American with a safe, secure electronic health record by 2014." The nominee also endorsed efforts to use data gleaned from electronic medical records to conduct “comparative effectiveness research" (CER) to provide information on the relative strengths and weaknesses of alternative medical interventions to health providers and consumers.”
Recovery Act funds have been allocated to NIH specifically for comparative effectiveness research. NIH has further specified the definition of CER as:
"[A] rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy."
NIH states that such research may include "the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcomes data as they apply to CER."
The problems I foresee concern the word "rigorous" as in the above definition.
The use of EHR data to reliably detect uncommon (but strong, discrete) early warning signals from a single drug or treatment -- to then be subject to more rigorous study with reasonable controls -- is itself a Medical Informatics "Grand Challenge." An example would be finding VIOXX's association with myocardial infarction earlier than we did, via an EHR-based automated postmarket surveillance process.
Doing this is a "grand challenge" due to the nature of EHR data, which is as far from "clinical trials clean" as possible. It is what might be called highly uncontrolled. The statistical methods needed to reliably pull signals out of the muck for even a single drug are still exploratory, the problems formidable if one wants to stay scientifically sound. I wrote about the experimental nature of such efforts a few years ago here, and believe an effort got underway at U. Indiana/Regenstrief to test such methodologies for postmarket surveillance about the same time.
Now we have had what appears to be a leap of faith and logic of irrationally exuberant proportions, and probably a deviation from sound science as well. The government has announced enthusiasm for EHR data-based comparative effectiveness research (CER) not to aid science, but to cut costs (implying skipping the rigorous confirmatory phases) through elimination of more costly drugs and treatments deemed less effective or at effectiveness parity compared to less expensive choices. Following this thinking, perhaps in the future a metric will be developed for an "acceptable" improved benefit/cost ratio for expensive drugs that are better than cheaper alternatives?
This overconfidence in EHR data is of concern. To detect relatively less concrete (i.e., than major ADE) "outcomes differences" between two or more drugs or treatments via EHR data - did treatment A lower blood pressure more than drug B, did drug C lessen depression more than drug D - rises to the level of "grand overconfidence in computing." To accomplish this task with reasonable scientific certainty from reams of EHR data, originating from different vendor systems, input by myriad people of different backgrounds with differing interpretations of terminologies (students/MD's/RN's etc) under different pressures (time, reimbursement maximization), and so forth, seems a stretch. What will the p values and predictive values be for such studies? Yet our incoming HHS secretary touts such methods?
Ironically, the gold standard in medical science is the controlled clinical trial, yet EHR-based comparative effectiveness research itself as a research methodology, now touted by our government, seems to have gotten a pass.
Even what I would consider minimum requirements for scientific treatment comparisons, such as well designed and reasonably controlled registries as developed here for interventional cardiology, with hundreds of granular, finely defined and "tuned" data elements, appear to be bypassed in EHR "miracle claims." Such precise registries take months or years to develop, implement, and train users to interact with properly. Further, such registries are not portable and must be created for individual medical domains and subdomains. Uncontrolled EHR data is no substitute for such efforts.
The following question arises:
Where are the comparative effectiveness studies that compare 1) EHR-based comparative effectiveness studies of drugs and treatments to 2) controlled clinical trials-based comparative effectiveness studies?
In other words, where are the meta-clinical trials that compare EHR data mining-based comparative effectiveness research as a methodology, vs. the "traditional" gold standard methodology of controlled clinical trials to compare drugs or treatments? How do we know EHR-based CER studies will not produce GIGO that will cause harm through ham-fisted elimination or defunding of useful treatment options?
While there are initial efforts underway to increase understanding of CER, e.g., "Broad Challenge Area 5" (PDF) of the NIH RC1 Challenge Grants in Health and Science Research, ominously, there is a lot of potential advantage to be had with terabytes of uncontrolled data and a political agenda.
I fear that what will come from "comparative effectiveness research" that draws upon uncontrolled EHR data will be politics masquerading as comparative effectiveness research. Good luck to private practitioners and medical innovators. Good luck, pharma. Good luck, patients.
This movement towards EHR uncontrolled data alchemy represents a further deviation from medical science towards the Syndrome of Inappropriate Over-Confidence in Computing (a.k.a. SICC Syndrome) writ large.
It seems the IT industry has now rendered a scientific approach to HIT and its use obsolete. We see this "post scientific era" phenomenon in the takeover of clinical IT by vendors who contractually demand suppression of sharing of problems, we see it in a remarkably uncritical push for EMR's by 2014 now involving force of government (only financial at present, but will punitive licensure issues and other measures be off the table?) despite a growing body of literature advising caution, we see a consortium of big business/payers/vendors/myriad secondary feeder organizations gunning full blast for this technology without consideration of the possible downsides.
Biomedical informatics, a scientific discipline (at least those parts of it not yet compromised by conflicts of interest), as a relevant field is very much a minority player in today's health IT.
Even the contributions from experts and pioneers in the field of Biomedical Informatics, in the form of the Jan. 2009 National Research Council's report that "Current Approaches to U.S. Health Care Information Technology are Insufficient" (here) has not had much impact.
I see Biomedical Informatics' death as a relevant discipline that anyone of importance pays attention to, not too far down the road as well.
-- SS
Addendum April 20:
We've seen this phenomenon in our economy. WSJ "Information Age" writer L. Gordon Crovitz notes:
... In a paper for the scientific journal of the Royal Society back in 1994, Harvard economist Robert Merton wrote that "any virtue can become a vice if taken to extreme, and just so with the applications of mathematical models in finance practice." We know even better now that some risks can be calculated and thus reduced, while some unknowns cannot be turned into probabilities. "The mathematics of the models are precise, but the models are not, being only approximations to the complex, real world."
I believe EMR data at best is a very loose approximation to the real world. It contains many "unknowns" regarding quality and reliability that cannot be turned into probabilities no matter how fancy the math. Asking too much of EHR data becomes a vice, not a virtue.
-- SS
Wednesday, February 11, 2009
The Attack on Government Funded Comparative Effectiveness Research
A provision in the massive US stimulus bill passed by the US Senate to fund comparative effectiveness research has generated considerable criticism. As Alicia Mundy, reporting in the Wall Street Journal, wrote:
And a Wall Street Journal editorial warned:
Of course, it is one thing to put comparative effectiveness research funding in a bill, and another thing to implement those funds to support research. There are all sorts of ways comparative effectiveness could go wrong. There is room for debate about how it ought to be done.
However, comparative effectiveness research, done right, has the potential to help doctors make better decisions that will help patients have better outcomes of care. As I have argued before,
It may be that some who protest comparative effectiveness are more worried about promoting products which good comparative effectiveness research might find are not as effective as their advertising says they are, than about physicians being hassled and micro-managed. We posted previously (here and here) about some particularly poorly crafted arguments against comparative effectiveness research. Some especially illogical arguments against it were made by people with important, albeit sometimes indirect financial relationships with health care corporations whose products may not prove so comparatively effective (here and here).
As Alicia Mundy's article noted above, the Partnership to Improve Patient Care seems to be at least partially controlled and funded by elements of the pharmaceutical, biotechnology, and device industries. Not only does its steering committee include the Advanced Medical Technology Association, the Biotechnology Industry Organization, and the Pharmaceutical Research and Manufacturers of America (PhRMA), but spot checks reveal that several of the disease advocacy organizations in its steering committee have significant relationships with pharmaceutical, biotechnology and/or device companies. For example, the 2007 annual report from the Alliance for Aging Research listed the organization's board as including executives of Omnicare Clinical Research, GlaxoSmithKline, Procter & Gamble, Guidant, Merck, Johnson & Johnson, and Novartis. The 2007 annual report of the National Alliance for Mental Illness (NAMI) lists corporate partners including Abbott Laboratories, AstraZeneca, Bristol-Myers-Squibb, Eli Lilly, Forest Laboratories, GlaxoSmithKline, Janssen (part of Johnson & Johnson), McNeil, Otsuka America Pharmaceutical, Pfizer, PhRMA, Solvay Pharmaceuticals, Vanda Pharmaceuticals, and Wyeth.
Not noted in Betsy McCaughey's op-ed article was that she is currently on the board of directors of Cantel Medical, a device company, and formerly on the board of Genta, a biotechnology company.
Comparative effectiveness research controlled by government bureaucracies could be done clumsily. Government oversight has the potential to tilt comparative effectiveness research more towards cost-containment than to finding the best tests and treatments for individual patients. But if oversight of government comparative effectiveness is transparent, and its leadership excludes people with vested interests in the research producing particular results, we can be hopeful that its results could be unbiased and helpful.
Up to now, however, we have left industry to fund, control, and too often suppress and manipulate clinical research about its own products, so that the results are better at putting particular products in a good light than providing doctors and patients with the best, most unbiased information needed to make decisions. Unless we really make a hash of its leadership and governance, government funded comparative effectiveness research has the potential to improve health and health care.
In an op-ed column for Bloomberg News that got wide attention, Betsy McCaughey, who is described as, "former lieutenant governor of New York and is an adjunct senior fellow at the Hudson Institute," wrote:
The drug and medical-device industries are mobilizing to gut a provision in the stimulus bill that would spend $1.1 billion on research comparing medical treatments, portraying it as the first step to government rationing.
The $1.1 billion in research funding would be doled out to the National Institutes of Health and other government bodies. 'We should focus on producing the best unbiased science possible,' said Rep. Henry Waxman (D., Calif.), a strong proponent of the House language.
Mr. Obama supported research into comparative effectiveness during his campaign. Administration officials and leading Democrats in Congress say the idea will help government programs direct their dollars to treatments that are worth the money.
Officially, drug and device makers don't object to that sentiment. But they warn of a slippery slope where the government ends up axing useful treatments just because they cost too much. They have lined up patient groups that get industry funding to lobby Capitol Hill.
A coalition called the Partnership to Improve Patient Care includes the lobbying arms of the drug, device and biotechnology industries as well as patient-advocacy groups and medical-professional societies. Coalition spokesman David Di Martino says the research envisioned in the House bill may be used 'in an inappropriate manner that may limit treatment options for patients.'
A public-relations firm that is part of one of Washington's most influential lobby shops, Barbour Griffith Rogers, is representing the coalition. A major goal is to give industry a seat at the table when federal officials decide what to research with the $1.1 billion.
Tragically, no one from either party is objecting to the health provisions slipped in without discussion. These provisions reflect the handiwork of Tom Daschle, until recently the nominee to head the Health and Human Services Department.
Senators should read these provisions and vote against them because they are dangerous to your health.
In his book, Daschle proposed an appointed body with vast powers to make the 'tough' decisions elected politicians won’t make.
The stimulus bill does that, and calls it the Federal Coordinating Council for Comparative Effectiveness Research (190-192). The goal, Daschle’s book explained, is to slow the development and use of new medications and technologies because they are driving up costs.
The elderly will bear the brunt.
Medicare now pays for treatments deemed safe and effective. The stimulus bill would change that and apply a cost- effectiveness standard set by the Federal Council.
And a Wall Street Journal editorial warned:
The true political goal is cost control. For the Pete Stark Democrats whose ambition is Medicare for all -- no exceptions -- giving government exclusive control over electronic health information and reporting is a step toward "comparative effectiveness" research. That in turn will be used to impose price controls and deny some types of medical treatment and drugs. And because government is able to skew the whole health system through Medicare and Medicaid, comparative effectiveness could end up micromanaging the practice of medicine.
Of course, it is one thing to put comparative effectiveness research funding in a bill, and another thing to implement those funds to support research. There are all sorts of ways comparative effectiveness could go wrong. There is room for debate about how it ought to be done.
However, comparative effectiveness research, done right, has the potential to help doctors make better decisions that will help patients have better outcomes of care. As I have argued before,
Physicians spend a lot of time trying to figure out the best treatments for particular patients' problems. Doing so is often hard. In many situations, there are many plausible treatments, but the trick is picking the one most likely to do the most good and least harm for a particular patient. Ideally, this is where evidence based medicine comes in. But the biggest problem with using the EBM approach is that often the best available evidence does not help much. In particular, for many clinical problems, and for many sorts of patients, no one has ever done a good quality study that compares the plausible treatments for those problems and those patients. When the only studies done compared individual treatments to placebos, and when even those were restricted to narrow patient populations unlike those patient usually seen in daily practice, physicians are left juggling oranges, tomatoes, and carburetors.
Comparative effectiveness studies are simply studies that compare plausible treatments that could be used for patients with particular problems, and which are designed to be generalizable to the sorts of patients usually seen in practice. As a physician, I welcome such studies, because they may provide very useful information that could help me select the optimal treatments for individual patients.
It may be that some who protest comparative effectiveness are more worried about promoting products which good comparative effectiveness research might find are not as effective as their advertising says they are, than about physicians being hassled and micro-managed. We posted previously (here and here) about some particularly poorly crafted arguments against comparative effectiveness research. Some especially illogical arguments against it were made by people with important, albeit sometimes indirect financial relationships with health care corporations whose products may not prove so comparatively effective (here and here).
As Alicia Mundy's article noted above, the Partnership to Improve Patient Care seems to be at least partially controlled and funded by elements of the pharmaceutical, biotechnology, and device industries. Not only does its steering committee include the Advanced Medical Technology Association, the Biotechnology Industry Organization, and the Pharmaceutical Research and Manufacturers of America (PhRMA), but spot checks reveal that several of the disease advocacy organizations in its steering committee have significant relationships with pharmaceutical, biotechnology and/or device companies. For example, the 2007 annual report from the Alliance for Aging Research listed the organization's board as including executives of Omnicare Clinical Research, GlaxoSmithKline, Procter & Gamble, Guidant, Merck, Johnson & Johnson, and Novartis. The 2007 annual report of the National Alliance for Mental Illness (NAMI) lists corporate partners including Abbott Laboratories, AstraZeneca, Bristol-Myers-Squibb, Eli Lilly, Forest Laboratories, GlaxoSmithKline, Janssen (part of Johnson & Johnson), McNeil, Otsuka America Pharmaceutical, Pfizer, PhRMA, Solvay Pharmaceuticals, Vanda Pharmaceuticals, and Wyeth.
Not noted in Betsy McCaughey's op-ed article was that she is currently on the board of directors of Cantel Medical, a device company, and formerly on the board of Genta, a biotechnology company.
Comparative effectiveness research controlled by government bureaucracies could be done clumsily. Government oversight has the potential to tilt comparative effectiveness research more towards cost-containment than to finding the best tests and treatments for individual patients. But if oversight of government comparative effectiveness is transparent, and its leadership excludes people with vested interests in the research producing particular results, we can be hopeful that its results could be unbiased and helpful.
Up to now, however, we have left industry to fund, control, and too often suppress and manipulate clinical research about its own products, so that the results are better at putting particular products in a good light than providing doctors and patients with the best, most unbiased information needed to make decisions. Unless we really make a hash of its leadership and governance, government funded comparative effectiveness research has the potential to improve health and health care.
Tuesday, August 5, 2008
BLOGSCAN - A Proposed Comparative Effectiveness Institute
On the Health Beat Blog, Maggie Mahar discussed a bill in the US Senate proposing the formation of a comparative effectiveness institute. It has one striking drawback. The board of the proposed institute, which would include representatives of "private payers; [and] pharmaceutical, device and technology companies." Both groups have financial interests favoring comparative effectiveness studies with particular groups. Private payers make more money when they can avoid paying for expensive tests, treatments, and other forms of care. They would prefer to see comparative effectiveness studies that find that more expensive tests, treatments, etc do not work so well. Pharmaceutical, biotechnology, and device companies would prefer to see comparative effectiveness studies that find their products work well. Because of these conflicts of interest, neither representatives of private payers nor representatives of pharmaceutical, biotechnology or device companies should be on the board of this institute.
Subscribe to:
Posts (Atom)