Monday, October 26, 2009

Who Should Sponsor Comparative Effectiveness Research?

We have tried to argue why comparative effectiveness research is a good idea. To cut and paste what I wrote in a previous post,

Physicians spend a lot of time trying to figure out the best treatments for particular patients' problems. Doing so is often hard. In many situations, there are many plausible treatments, but the trick is picking the one most likely to do the most good and least harm for a particular patient. Ideally, this is where evidence based medicine comes in. But the biggest problem with using the EBM approach is that often the best available evidence does not help much. In particular, for many clinical problems, and for many sorts of patients, no one has ever done a good quality study that compares the plausible treatments for those problems and those patients. When the only studies done compared individual treatments to placebos, and when even those were restricted to narrow patient populations unlike those patient usually seen in daily practice, physicians are left juggling oranges, tomatoes, and carburetors.
Comparative effectiveness studies are simply studies that compare plausible treatments that could be used for patients with particular problems, and which are designed to be generalizable to the sorts of patients usually seen in practice. As a physician, I welcome such studies, because they may provide very useful information that could help me select the optimal treatments for individual patients.

Because I believe that comparative effectiveness studies could be very useful to improve patient care, it upsets me to see this particular kind of clinical study get caught in political, ideological, and economic battles.

In particular, we have discussed a number of high profile attacks on comparative effectiveness research, which often have featured arguments based on logical fallacies. While some of the people making the attacks have assumed a conservative or libertarian ideological mantle, one wonders whether the attacks were more driven by personal financial interests. For example, see our blog posts here, here, here, and here. On the other hand, we discussed a clear-headed defense of comparative effectiveness research by a well-known economist most would regard as libertarian here.

Comparative effectiveness research has been discussed as an element of health care reform in the US. It turns out that the current version of the health care reform bill in the US Senate has a provision to create a Patient Centered Outcome Research Institute, which presumably would become the major organization which could sponsor comparative effectiveness research.

This institute, however, would not be a government agency (despite the name that makes it sound like it would be part of the National Institutes of Health). Moreover, here is a description of the Board of Governors who would run the institute from the current version of the bill :

BOARD OF GOVERNORS.—
(1) IN GENERAL.—The Institute shall have a Board of Governors, which shall consist of 15 members appointed by the Comptroller General of the United States not later than 6 months after the date of enactment of this section, as follows:
(A) 3 members representing patients and health care consumers.
(B) 3 members representing practicing physicians, including surgeons.
(C) 3 members representing private payers, of whom at least 1 member shall represent health insurance issuers and at least 1 member shall represent employers who self-insure employee benefits.
(D) 3 members representing pharmaceutical, device, and diagnostic manufacturers or developers.
(E) 1 member representing nonprofit organizations involved in health services research.
(F) 1 member representing organizations that focus on quality measurement and improvement or decision support.
(G) 1 member representing independent health services researchers.


Thus, only 3/15 members of the governing board would represent the patients who ultimately reap the benefits or suffer the harms produced by medical diagnosis and treatment. Further, 6/15 members represent for-profit corporations which stand to make more or less money depending on how particular comparative effectiveness studies come out. Also, 3/15 members would be physicians, some of who may get paid more to deliver particular treatments (e.g., procedures) than others (e.g., providing advice about diet and exercise).

We often discuss how clinical research sponsored by organizations with vested interest in the research turning out to favor their products or services may be manipulated to favor these interests, and sometimes suppressed if it does not. In the US, there are few unconflicted sources of sparse funds to support comparative effectiveness research. (The most significant current source is the Agency for Healthcare Research and Quality, AHRQ. For full disclosure, I have been an ad hoc reviewer of grants for that agency.)

The current draft of legislation would create the largest potential sponsor for comparative effectiveness research, but would make that organization report to representatives of for-profit companies whose profits may be affected by the results of such research. In my humble opinion, this is not much of an advance. Comparative effectiveness research controlled by corporations that stand to profit or lose depending on its results will forever be suspect.

If the government is going to support comparative effectiveness research, it ought to make sure such research is not run by people with vested interests in the outcomes coming out a certain way. I may be biased myself, but why not let the research be sponsored by AHRQ, an agency with relevant experience and no axe to grind vis a vis any particular product or service?

Sunday, October 25, 2009

Clinic's medical files vanish

At "Data Malpractice on T-Mobile Sidekick: But Don't Worry, Your Medical Data is Safe", on Oct. 16 I wrote:

One of the promises made about healthcare IT is that your medical data is "safer" in electronic form than in paper form. The Hurricane Katrina example of paper records being destroyed is often used as a poster example of the dangers of paper records.

However, the risk of electronic storage of information, especially the talk of national EMR's stored on the "cloud" (an amorphous term meaning distributed storage "out there" whose physical sites and boundaries are supposedly irrelevant from the user's perspective) has also been under-reported.

Personal customer data had been "lost" from many of T-Mobile USA's Sidekick devices due to a computer malfunction, although the data was apparently recovered eventually, apparently through luck rather than good engineering.

I expressed concern that such mishaps could affect clinical IT. I did not have to wait long for such a case to appear. Less than one week.

Below is a story of a Canadian clinic that lost two years of electronic health records:

Clinic's medical files vanish

By Ryan Cormier, Edmonton Journal

October 21, 2009

During a recent investigation into whether a patient's confidentiality had been breached at the Fairview Medical Clinic, an investigator asked for a log of who had accessed the complainant's file. When the clinic responded that it had automated his records in 2004 but only had files from 2006 on, alarm bells rang.

"That raised a lot of questions," said Leahann McElveen, an investigator with the office of the information and privacy commissioner.

The clinic had permanently lost two years worth of health files that include patient information on visits, prescriptions, lab reports, doctor's notes and other information. The loss happened when the clinic switched from one electronic medical records system to another.

"They were two similar systems intended to do the same thing," McElveen said. "However, they weren't coded the same way behind the scenes. It's not that the records fall into the wrong hands, they just don't exist anymore."


*POOF*.

Deinstallation of one system in favor of another is not uncommon. EHR data may become unavailable due to lack of data portability and the expense of data migration, or in this case apparently due to preventable technical problems.

It is essential for clinical IT users to have robust disaster recovery and business continuity solutions, and take great care when performing actions that can lose large amounts of data very fast. This adds to clinical IT cost, and a concern is that some users might skimp on these capabilities.

This must be discouraged.

(To the reader: do you back up your own PC or Mac reliably?)

-- SS

Washington Post Article: Electronic medical records not seen as a cure-all

Regarding the very well done Sunday Oct. 25, 2009 story in the Washington Post "Electronic medical records not seen as a cure-all" by staff writer Alexi Mostrous - signup may be required for access - I have several observations.

(Not including the observation that Mr. Mostrous probably deserves an award for being the first major newspaper reporter to broach this topic in a serious and balanced manner.)

First, I believe healthcare IT can live up to all the predictions made about its benefits - but only if done well. There is massive complexity behind those two words "done well", and that is HIT's key stumbling block in 2009. I believe we are only in the adolescent stage of knowing how to "do health IT well."

Second, I should point out that the intended consequences of health IT include, among many other things, the following "hiding in plain sight" (i.e., not often verbalized) intended consequences:

  • The improvement of medicine ... in the context of protection of patient rights established over centuries of development of modern medicine.
  • The improvement of IT itself through cross disciplinary collaboration between IT and medicine, of the science of IT (computer science), the social aspects of computerization ("sociotechnical issues"), and improvement of the our understanding of the intersection of medicine and computers.

Instead, we largely have the opposite. Patients' rights are trampled, and hostility and territoriality has arisen between clinicians (including medical informaticists) and IT, groups that rarely if ever interacted in hospitals ten or twenty years ago.

Of concern, when scientific study sections evaluate NIH grant proposals calling for testing of new IT that involves patients, patient protections and informed consent processes are a paramount concern since such activities are considered research. Yet, in implementing large clinical IT system in a hospital with new features, there are no formal regulations, and I'm not sure there's even IRB involvement in most cases. Patients do not get the chance to give informed consent to the use of these IT devices mediating their care. Why the difference?

The unintended results of computerization efforts have also included suppression of research on sociotechnical issues and on informatics, which must include study of the downsides of HIT, and of the failures in addition to the study of the successes. That is scientific fact - there is no room for debate, no room for spin on the need for careful study of the downsides of any mission critical domain. One would think there to be a vibrant literature on these issues, Yet searches on massive biomedical databases such as PubMed on, say, "cerner electronic health record" (or other vendors as well) are disappointing to say the least. Further, my own website on HIT difficulties remains nearly unique (PPT) after ten years online. That is not bragging; it is a disturbing finding to me - symptomatic of an industry that somehow has managed to avoid serious scrutiny.

In a field with downsides, there are:

1) those who know about the problems but fail to speak,
2) those who see the problems but fail to act, and
3) those who see, know and speak/write/research the problems.

That said, now on to the Washington Post article:

... bipartisan enthusiasm has obscured questions about the effectiveness of health information technology products, critics say. Interviews with more than two dozen doctors, academics, patients and computer programmers suggest that computer systems can increase errors, add hours to doctors' workloads and compromise patient care.

I would include the bipartisan enthusiasm under the subject header of "irrational exuberance", which itself falls under the header of "lack of domain knowledge." That itself is a consequence of both failure to study the issues, and suppression of those issues by those with an interest in doing so.

health IT's effectiveness is unclear.

The literature is indeed conflicting, and the need for rigorous scientific study has never been more essential considering the commitment of tens of billions of dollars towards health IT. The time for story telling, marketing based on opinion, name calling, leap-of-faith extrapolations of light year dimensions, and other forms of pseudoscience and non-science are over. The time for objective study is now.

The Senate Finance Committee has amassed a thick file of testimony alleging serious computer flaws from doctors, patients and engineers unhappy with current systems.

Being the ranking member of that committee, Sen. Grassley has a fiduciary responsibility to protect Medicare and Medicaid patients (and one might argue, to protect all patients since those programs often serve as models for private insurers). In that regard, the investigation he has initiated is part of his responsibility as a ranking member of Congress. Politics aside (and there are those who will resort to ad hominem "political witch hunt" arguments), he would have been negligent if he had not initiated an inquiry.

Sen. Grassley has taken on the pharma industry and the government's Food and Drug Administration itself, such as in this recent article "FDA fails to follow up on unproven drugs" where he concluded from a GAO study he ordered that "FDA has fallen far short of where it should be for patient safety." He seems quite serious about medical safety.

If only others in Congress had done their job similarly regarding national finance, we might now not be in the worst economic crisis since '29 with many major industries failing.

David Blumenthal, the head of health technology at the Department of Health and Human Services, acknowledged that the systems had flaws. "But the critical question is whether, on balance, care is better than before," he said. "I think the answer is yes."

This sounds uncomfortably like how a pharmaceutical company might respond to doubts about drug effectiveness and safety. In reality it's really irrelevant what he "thinks." Where's the data? Is this a political statement, a personal belief, or a statement backed up by scientific fact that is not cast into doubt by other research results? Our own National Research Council, Joint Commission, and other international organizations have written about their doubts and concerns about HIT [as that IT is designed and implemented in 2009]. If there is rigorous, systematic research weighing pro's and con's to back this assertion, I wish it would be published.

For his statement is really saying "we don't really know how many systems have flaws, we know some do, and we don't really know the full extent of the impact of those flaws, but because there can be some benefits, let's spend $50+ billion before we know the extent of the problems and fix them." I point out research from Harvard forty years ago, when Harvard informatics pioneer Dr. Octo Barnett published the "Ten Commandments of HIT." Two of those commandments were:

... 8. Thou shall be concerned with realities of the cost and projected benefit of the computer system [i.e., ROI - ed.]

10. Be optimistic about the future, supportive of good work that is being done, passionate in your commitment, but always guided by a fundamental skepticism.

The full set is in this post. Somewhere in the past 40 years, the rigorous ROI evaluations (which also must include systematic evaluations of risk, as any businessperson knows) and the fundamental skepticism seem to have gotten lost.

Over the next two months, Blumenthal will finalize the definition of "meaningful use," the standard that hospitals and physicians will have to reach before qualifying for health IT stimulus funds.

This is an example of putting the cart before the horse, and is a semantically-based, self contained logical fallacy of sorts. If a health IT system is harmful, the term "meaningful use" is itself Orwellian. If we don't know if HIT is beneficial, or have doubts, then such as term presupposes that health IT is inherently beneficial. A better term would have been "good faith use" - use based on the faith or hope that health IT will have an overall positive effect. The term "meaningful use" jumps the gun and is more a political slogan than a "meaningful term."

"If you look at other high-risk industries, like drug regulation or aviation, there's a requirement to report problems," said David C. Classen, an associate professor of medicine at the University of Utah who recently completed a study on health IT installations.

This is obvious, the reasons for it are obvious, and the reasons why health IT needs a requirement for problems reporting (one aspect of post-marketing surveillance, the "Phase IV" study as it is known in pharma) is obvious. Yet in 2009, no such requirement exists (see my post "Our Policy Is To Always Have Unabashed Faith In The Computer" for more on why we need reporting requirements.) Why do these requirements simply not exist in HIT?

"It's been a complete nightmare," said Steve Chabala, an emergency room physician at St. Mary Mercy Hospital in Livonia, Mich., which switched to electronic records three years ago. "I can't see my patients because I'm at a screen entering data." Last year, his department found that physicians spent nearly five of every 10 hours on a computer, he said. "I sit down and log on to a computer 60 times every shift. Physician productivity and satisfaction have fallen off a cliff."

The industry in the past has called such physicians "luddites", "resistant to change", "stubborn" etc. However, argumentum ad hominem is a fallacious mode of argument that has no place in a scientific field such as biomedicine. There also seems to be quite a lot of such concerns expressed by a large number of physicians, nurses, etc., and dismissing their concerns with a wave of the hand is cavalier in the extreme - again, these are first principles, without room for argument or debate. Let's study the issues rigorously and scientifically before resorting to ad hominem.

Other doctors spoke of cluttered screens, unresponsive vendors and illogical displays. "It's a huge safety issue," said Christine Sinsky, an internist in Dubuque, Iowa, whose practice implemented electronic records six years ago.

See my eight part series on mission hostile clinical IT here for examples of what Dr. Sinsky is referring to.

She emphasized that electronic records have improved her practice. "We wouldn't want to go back," she said. "But EHRs are still in need of significant improvement."

Yes, not cancellation, but improvement. And, quite importantly, before tens of billions of dollars are spent. Hospitals and physician offices are not an IT development laboratory, since the users of these facilities (patients) have very special rights and the clinicians, very special obligations and responsibilities.

Legal experts say it is impossible to know how often health IT mishaps occur. Electronic medical records are not classified as medical devices, so hospitals are not required to report problems.

That after decades of HIT development, sales and implementation we cannot know with certainty how often mishaps occur is, quite simply, a scandal of major proportions. Quoting an old House of God law, #10: "if you don't take a temperature, you can't find a fever." Another applicable aphorism seen on another discussion board: "you can only be so negligent or craven before the only remaining rationale is that you intended the result."

"Doctors who report problems can lose their jobs," Hoffman said.

I've taken risks with my own career in criticizing health IT, as have my colleagues. Hoffman is not exaggerating.

"Hospitals don't have any incentive to do so [speak out about problems with HIT] and may be in breach of contract if they do."

Imagine the outcry if the same prevailed regarding drugs or medical devices. The cemeteries would be lined with people whose epitaph could read "we bury our mistakes."

While orange-shirted vendor employees "ran around with no idea how to work their own equipment," the internist said, doctors struggled to keep chronically ill patients alive. "I didn't go through all my training to have my ability to take care of patients destroyed by devices that are an impediment to medical care."

This gets to issues I first raised in my website on HIT difficulties: who are these IT personnel, and what are their qualifications, exactly, to be working in mission critical medical environments? How is their competence evaluated?

I think these are questions that need to be answered.

-- SS

Saturday, October 24, 2009

Washington Post: EMR's No Cure-All; Sen. Grassley Sends Letter of Inquiry to health IT vendors

Senator Grassley has sent a letter out to ten major health IT vendors seeking information on health IT problems and flaws. The letter can be retrieved from this link (PDF) at the Washington Post. Read the whole thing.

It is linked via a Washington Post story "Electronic medical records not a cure-all", Sunday Oct. 25, 2009, by staff writer Alexi Mostrous. The WaPo article itself seems to require registration. I will comment on the article in future posts.

Addendum 10/25: I have now commented on the article here.

-- SS

Friday, October 23, 2009

"Organisational Ethics Policies; A Primer"

I regret that it took me so long to find an essay on "Organisational Ethics Policies" by Howard Whitton, available from the European U4 Anti-Corruption Resource Center. While it was written with international non-governmental organisations (NGOs) who "administer aid programs" in mind, it seems applicable to all kinds of NGOs and not-for-profit organizations, including those in health care. In the US, most medical schools and their parent universities, most hospitals and academic medical centers, essentially all medical societies and disease advocacy groups, and some insurance companies and managed care organizations are not-for-profit.

The main points of the paper are its summaries of the basic elements of "effective ethics policies."

First, such a policy

- must first have unequivocal authority and the endorsement of boards and senior management, and must be:
o founded on the organisation’s core values, mandate, and ethical principle
o developed in consultation with those affected by it
o realistically achievable
o written in plain language, coherent with other policies, and easily available
o clearly understood by staff, and by other stakeholders
o consistent with the organisation’s policies on rewards and sanctions
o regularly reviewed and evaluated with all stakeholders
o universally applied, and transparently enforced.


The main content areas might include such "major areas of ethical risk" as:
o financial management and accountability standards
o internal and external audit processes
o professional ethics, conduct, and conflict of interest standards
o fair treatment rules for staff and clients
o processes for the prevention of fraud and other abuse of trust
o integrity mechanisms governing proper decision-making
o provision of transparent information to stakeholders
o complaints and whistleblower disclosure processes
o principled policy dispute processes
o transparent and objective evaluation mechanisms.

So, specific policies should include the following functional elements:

o a code of conduct/ethics based on the organisation’s core values
o professional practice standards interpreting the code’s principles
o procedures for managing conflict of interest situations (including the registration of relevant interests and assets of decision makers)
o procedures for offering and accepting gifts and business courtesies
o criteria for the proper use of organisational assets and authority
o prohibition of harassment and discrimination in the workplace
o criteria for protected reporting of unethical or illegal behaviour
o rights of clients to obtain service, including complaint procedures for failure to meet standards
o obligations for accountability and transparency,and information provision
o standards for dealing with confidential and privileged information
o constraints on ancillary and post-separation employment
o standards for providing reasons for administrative decisions.


Also the policies should include:

• A commitment to training staff in the full range of ethics-related activities. Training will improve personal awareness and strengthen the ability to define and manage improper conduct, whether by co-workers, managers, or external stakeholders.
The range of training themes should include the organisation’s integrity system, specific anticorruption measures, harassment-free workplaces, non-discrimination principles, financial management and audit, integrity in procurement practices, donor relations, personal and institutional conflict of interest, accountability, responsibility, procedural fairness, and strategic problem-solving.

• Policies and procedures for regular management reporting to boards and executives, in particular to enable monitoring of matters which may be of particular concern from time to time.

• Independent, external scrutiny of policies provide an important resource for boards and executives for ensuring that espoused core values and actual behaviours are aligned, and to identify areas of policy and management practice requiring
improvement.

• Policies and procedures for protected reporting of improper conduct, both to enhance worker and stakeholder confidence in the integrity of an organisation, and to provide avenues for early detection of inappropriate behaviour. Genuine
whistleblowing must be effectively endorsed, and effectively protected, to ensure the organisation’s credibility.

• Procedures for the sanctioning of improper conduct and failure to meet relevant standards by staff, structured so as to enhance management’s capacity to deal effectively with ethical issues in the workplace.

Such policies cannot be considered ethical panaceas, but in my humble opinion (and based, I believe, on at least a little cognitive psychology), visible, reasonable, clear ethics policies could reduce the sort of bad behavior that Health Care Renewal often discusses on the part of leaders of major health care not-for-profit organizations and NGOs.

So, those of you who work for or are otherwise affiliated with a not-for-profit university, medical school, hospital, academic medical center, medical organization, disease advocacy organization, or insurance company/ managed care organization might want to go through the exercise of answering these questions:
1 - Does your organization have anything that resembles an ethics policy?
2 - If so, which of the characteristics listed above does it have?
3 - Which of the content areas listed above does it include?
4 - Which of the functional and additional elements listed above does it include?

If much is missing, is there an obvious reason for what was omitted? If the policy seems poorly characterized or incomplete, why should it not be improved? Would you feel comfortable suggesting improvements? If not, why not, and what does that say about the organization?

Academic Freedom Curtailed: Censorship Down Under On EHR's for the Emergency Department?

In a stunning development:

The essay on Emergency Department electronic health record (EHR) problems in the Australian state of New South Wales (NSW) by medical informatics professor Dr. Jon Patrick, Health Information Technologies Research Laboratory (HITRL), University of Sydney, that I referenced in my posts "The Story of the Deployment of an ED Clinical Information System ‐ Systemic Failure or Bad Luck" and "NSW Nightmare and Overuse of Computers" appears to have been censored. This apparently occurred at the level of the the government.

The essay was available as item 6 at http://www.it.usyd.edu.au/~hitru/index.php?option=com_content&task=view&id=91&Itemid=146 . Attempts to download now provide a message "This document is not currently available." I do not know if the vendor was involved.

(A copy of vers. 3 of Professor Patrick's report is still available here in PDF; it was a work in progress when it was pulled.) [Oct. 24 - see addendum below - ed.]

This appears to be an overt example of suppression of academic freedom that I believe should receive widespread attention, most especially if there is retaliation of any kind against the professor.

I also believe censorship of a document such as this could escalate to a human rights issue regarding what appears to be deliberate censorship of critical information about IT systems that could possibly be putting large numbers of patients at risk. Imagine if this was a report about potentially defective nuclear reactor control systems or containment processes at biological hazards research sites.

I believe addressing the issues raised in the report would have been a far more responsible approach than censorship.

-- SS

Addendum 10/24: The author of the paper asked me to remove it from my server as well. It is currently unavailable. This situation is in flux and hopefully will be resolved soon. Hopefully, the issues in the report will be addressed.

Thursday, October 22, 2009

Note to readers referred here from the Oct. 20 NY Times

Note to readers referred here from the Oct. 20 NY Times story "Steps to Greater Accountability in Medical Education" by Duff Wilson:

This is a multi-author blog. The essay "Nemeroff, Seroquel, and ACCME" by Dr. Bernard Carroll is at this link on the blog.