Comparative Effectiveness Research: A Report From the Institute of Medicine

Medicine and Public Issues4 August 2009Comparative Effectiveness Research: A Report From the Institute of MedicineFREEHarold C. Sox, MD, Editor and Sheldon Greenfield, MDHarold C. Sox, MD, EditorFrom the American College of Physicians, Philadelphia, Pennsylvania, and University of California, Irvine, Irvine, California. and Sheldon Greenfield, MDFrom the American College of Physicians, Philadelphia, Pennsylvania, and University of California, Irvine, Irvine, California.Author, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-151-3-200908040-00125 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Join the dialogue on health care reform. Comment on the perspectives published in Annals and offer ideas of your own. All thoughtful voices should be heard.On 30 June 2009, another milestone was achieved in progress toward a national system to promote medical research that focuses on decision making by physicians and patients. The first steps occurred in late 2007 and early 2008 with a seminal article (1) and an Institute of Medicine (IOM) report that called for a national initiative of research that would support better decision making about interventions in health care (2). One of us was a coeditor of the IOM report. A third milestone was reached when both presidential candidates endorsed this concept. A fourth milestone came when the president signed into law the American Recovery and Reinvestment Act of 2009 (ARRA); this act allotted $1.1 billion to support this form of research, which had become known as comparative effectiveness research (CER). The legislation created a federal council on CER and asked the IOM to elicit input from a broad array of stakeholders on which research topics should have the highest-priority for funding through the ARRA and to then develop a list of the highest-priority topics for the Secretary of Health and Human Services to consider. By law, the Federal Coordinating Council and the IOM committee to set priorities for comparative effectiveness research reported to the Secretary on 30 June 2009. This issue contains both this article, which is a commentary on the IOM committee report (3) by the committee's cochairs, and a perspective on better research methods for CER (4).Definition of CERThe IOM committee quickly settled on a working definition of CER, which consisted of the elements of earlier definitions reduced to 2 sentences:CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition, or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.Two key elements that are embedded in this definition are the direct comparison of effective interventions and their study in patients who are typical of day-to-day clinical care. These features would ensure that the research would provide information that decision makers need to know, as would a third feature, research designed to identify the clinical characteristics that predict which intervention would be most successful in an individual patient. The same research design would also help policymakers by identifying subpopulations of patients that are more likely to benefit from one intervention than the other.High-Priority Research TopicsThe IOM committee sought advice from a broad range of stakeholders and received it in 3 forms: approximately 90 letters; 54 oral presentations at a day-long hearing in Washington, DC; and a Web-based nomination form. Overall, the committee received more than 2606 nominations from 1758 individual responders within 3 weeks. In a 3-step voting process, the committee identified 100 high-priority topics, which it ranked in quartiles. At each stage, the committee sought to have a balanced portfolio of topics. Table 5-1 in the report (available at www.iom.edu/cerpriorities) lists the 100 highest-ranked topics.Establishing a Sustainable National Program of CERThe U.S. Congress is currently debating legislation that would guarantee health insurance to more Americans while reforming the U.S. health system. A key aim of the legislation is to moderate the rate at which the cost of health care increases while improving the quality of care. Many leaders believe that reaching this goal will require a sustained effort to produce better evidence to inform decision making. Government officials have characterized the $1.1 billion allocated to CER as a down payment on a national program of CER. As of this writing, the House of Representatives bill proposes a national program of CER, and the Senate Finance Committee white paper does likewise. Accordingly, the IOM committee made several recommendations aimed at a sustainable, trustworthy national CER initiative. The Appendix provides the full text of these recommendations.GovernanceThe 2008 IOM committee recommended establishing a national program "with authority, overarching responsibility, sustained resources, and adequate capacity to ensure production of credible, unbiased information about what is known and what is not known about clinical effectiveness" (2). The current committee expanded this recommendation by calling upon the secretary of the Department of Health and Human Services "to establish a mechanism—such as a coordinating advisory body—with the authority to strategize, organize, monitor, evaluate, and report on the implementation and impact of the CER Program." The committee recognized that many agencies of the federal government are engaged in research that fits some or all of the key elements of CER. To make the best use of the funds allocated for CER, these agencies should be jointly accountable for spending CER funds on research that reflects the same conceptualization of CER. Therefore, these disparate efforts should use a single definition of CER and the same definitions of outcomes and measures of function and illness. This form of cross-agency coordination is compatible with each agency focusing on the form of research with which it has the greatest experience.Active Involvement of Consumers in CERThe 2009 ARRA created a means for coordinating federal CER efforts: the Federal Coordinating Council. Seeking to complement the functions of the Council, the IOM committee focused on the potential contribution of consumers of health care to CER and recommended involving them at all levels of the national CER initiative. The recommendation is as follows: "The CER Program should fully involve consumers, patients and their caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination."The concept that animates this recommendation is a radical but logical departure from past practice: When a principal aim of research is to inform decision makers, listen to the decision makers. Meaningful participation will require consumers to learn more about research, just as they will teach researchers about how to frame their investigations to satisfy the needs of consumers who are also decision makers. This approach has precedent. For example, the National Institute for Health and Clinical Effectiveness in the United Kingdom teaches decision making to consumers who participate in the institute's program.Methodology of CERRemembering that the purpose of CER is to inform decision making, consider the principal forms of clinical research. Systematic reviews of the literature summarize a body of evidence. Over the past 2 decades, they have become indispensable to expert panels that formulate practice guidelines or policymakers who make insurance coverage decisions. These "macro" decision makers are starting to use decision models for the same purposes (5). Two broad categories of research are the starting point for systematic reviews: observational research and randomized trials. Large established databases provide an opportunity to link current health care practices to the outcome of care. These databases are representative of critical groups often omitted from randomized trials, but they contain limited data; of note, they seldom specify the rationale for medical decisions. These observational research methods have many advantages—speed, real-world decisions, large numbers of decisions and outcomes, and low cost—but they cannot escape a key limitation: Characteristics of the patient that drive real-life clinical decisions may also influence clinical outcomes, leading to uncertainty about whether they or the intervention itself causes the outcomes. Overcoming the limitations of observational research is the most important frontier of research on study methods.To overcome these limitations, researchers often randomly assign patients to different interventions. This simple action eliminates much of the uncertainty that plagues the interpretation of observational research. However, we have too often used the conceptual elegance of randomization to answer the wrong questions. We ask, "Does this work?" when our readers want to know, "Is this better than that?" Thus, too often, randomized trials are so-called efficacy studies: They are designed to create near-ideal circumstances to see whether the intervention can possibly work. To this end, researchers exclude many types of patients from efficacy trials and often compare the intervention with a placebo. In these respects, efficacy trials are unlike the conditions that physicians and patients face in daily practice. Another article in this issue (4) describes several exciting new methods for making randomized trials more suitable for CER.The examples of observational research and new randomized trial methods are the basis for the IOM committee's strong recommendation that a national CER program should support research in the methods of clinical research.The Infrastructure for Observational ResearchThe committee believes that very large collections of the electronic records of patients can be a valuable resource for CER. Using these data sets, it is possible to compare the outcomes of using several effective interventions in a population that is representative of daily care, 2 features that are aligned with the goals of CER. The great numbers of patients in these data sets also makes it possible to study subgroups with precision and perhaps identify key predictors of response to an intervention, both of which would facilitate decision making at the individual and population level. However, using these data sets will require researchers to surmount several technical barriers, to say nothing of the analytic challenges of making crisp inferences from observational data.The committee's recommendation is worded as follows: "The CER Program should help to develop large-scale, clinical and administrative data networks to facilitate better use of data and more efficient ways to collect new data to yield CER findings."The committee recognized the need to protect patients' privacy; it therefore called for development of methods for linking patient-level data from multiple sources. It also recognized that participating in a data network requires institutions to invest in the technical requirements and make a commitment to provide high-quality data. Accordingly, the national CER program should create incentives to participate and rewards for good performance.The CER WorkforceDepending on the scale of the public's investment in CER, the workforce for CER may prove to be inadequate. The committee recommended that a national CER program invest in developing a research workforce with the requisite skills for CER. The national CER program should develop a strategic plan for meeting the CER workforce needs and invest in career development support at every level from graduate school to mid-career.Translating Research Findings into Patient CareThe main reason for public interest in CER is the belief that better information will result in better health outcomes and more effective use of resources. Comparative effectiveness research is an important cog in a larger mechanism for promoting better health care practices. The committee believes that a national CER program will be perceived as successful if it funds research that improves the uptake of new knowledge and its translation into better decision making. Indeed, many of the high-ranked research topics addressed questions about the best methods to translate research results into action. In the end, whether CER will realize its potential will depend on other features of health care reform legislation, the profession's ability to mobilize the best in its members, and the ability of individual patients to engage their physicians in a dialogue about their own care.Personal ReflectionsResearch that informs clinical decisions is everywhere, yet a national research initiative to improve decision making by patients and their physicians is a novel concept. We should all hope that the program goes well beyond the goal of providing better information for decision making. That mission is important. Indeed, it is the principal motivation for a national CER program. However, we believe that it is not enough. Somehow, and in concert with national professional organizations and local opinion leaders, a national program should promote the professional ethos that places the interests of patients and the larger community above all other considerations (6). Together, a national initiative that engages the medical profession and patients can justify a large public investment in CER.References1. Wilensky GR. Developing a center for comparative effectiveness information. Health Aff (Millwood). 2006;25:w572-85. [PMID: 17090555] CrossrefMedlineGoogle Scholar2. Eden J, Wheatley B, McNeil B, Sox H, eds. Institute of Medicine. Knowing What Works in Health Care: A Roadmap for the Nation. Washington, DC: National Academies Pr; 2008. Google Scholar3. Ratner R, Eden J, Wolman D, Greenfield S, Sox H, eds. Institute of Medicine. Initial national priorities for comparative effectiveness research. Washington, DC: National Academies Pr; 2009. Google Scholar4. Luce BR, Kramer JM, Goodman SN, Connor JT, Tunis S, Whicher D, et al. Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change. Ann Intern Med. 2009;151:206-9. LinkGoogle Scholar5. Zauber AG, Lansdorp-Vogelaar I, Knudsen AB, Wilschut J, van Ballegooijen M, Kuntz KM. Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med. 2008;149:659-69. [PMID: 18838717] LinkGoogle Scholar6. Gawande A. The cost conundrum. The New Yorker. 2009;1 June. Accessed at www.newyorker.com/reporting/2009/06/01/090601fa_fact_gawande on 23 June 2009. Google ScholarAppendix: Recommendations to Establish the Infrastructure for a Successful CER InitiativeRecommendation 5: The HHS Secretary should establish a mechanism—such as a coordinating advisory body—with the authority to strategize, organize, monitor, evaluate, and report on the implementation and impact of the CER Program.Recommendation 6: The CER Program should fully involve consumers, patients and their caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination.The CER Program should develop strategies to reach out to, engage, support, educate, and, as necessary, prepare consumers, patients and their caregivers for leadership roles in these activities.The CER Program should also encourage broad participation in CER research in order to create a representative evidence base that could help identify health disparities and inform decisions by patients in special population groups.Recommendation 7: The CER Program should devote sufficient resources to research and innovation in the methods of CER, including the development of methodological guidance for CER study design such as the appropriate use of observational data and more informative, practical, and efficient clinical trials.Recommendation 8: The CER Program should help to develop large-scale, clinical and administrative data networks to facilitate better use of data and more efficient ways to collect new data to yield CER findings.The CER Program should ensure that CER researchers and institutions implement privacy and security protections in all their activities.The CER Program should support the development of methodologies for linking patient-level data from multiple sources, and for analyzing the resulting data for CER.The CER Program should promote widespread participation and provide incentives to data holders to participate in CER.Recommendation 9: The CER Program should develop and support the workforce for CER to ensure the nation's capacity to carry out the CER research mission. Important next steps include:Development of a strategic plan for research workforce development.Long-term, sufficient funding for career development including expanding grants for graduate and postgraduate training opportunities in comparative effectiveness methods as well as career development grants and mid-career merit awards.Recommendation 10: The CER Program should promote rapid adoption of CER findings and conduct research to identify the most effective strategies for disseminating new and existing CER findings to health care professionals, consumers, patients, and their caregivers and for helping them to implement these results in daily clinical practice. Comments0 CommentsSign In to Submit A Comment Author, Article, and Disclosure InformationAuthors: Harold C. Sox, MD, Editor; Sheldon Greenfield, MDAffiliations: From the American College of Physicians, Philadelphia, Pennsylvania, and University of California, Irvine, Irvine, California.Disclosures: None disclosed.Corresponding Author: Harold C. Sox, MD, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.Current Author Addresses: Dr. Sox: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.Dr. Greenfield: Center for Health Policy Research, University of California, Irvine, 111 Academy, Suite 220, Irvine, CA 92697. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoRethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change Bryan R. Luce , Judith M. Kramer , Steven N. Goodman , Jason T. Connor , Sean Tunis , Danielle Whicher , and J. Sanford Schwartz Metrics Cited byTranslational ScienceSequential and Comparative Evaluation of Pain Treatment Effectiveness Response (SCEPTER), a pragmatic trial for conservative chronic low back pain treatmentIntegrated rapid-cycle comparative effectiveness trials using flexible point of care randomisation in electronic health record systemsComputer clinical decision support that automates personalized clinical care: a challenging but needed healthcare delivery strategySlow recruitment in the HIMALAIA study: lessons for future clinical trials in patients with delayed cerebral ischemia after aneurysmal subarachnoid hemorrhage based on feasibility dataAn evaluation of stakeholder engagement in comparative effectiveness research: lessons learned from SWOG S1415CDImpact of Missing Data on Identifying Risk Factors for Postoperative Complications in Hand SurgeryImpact of energy saving on the financial performance of industrial enterprises in China: An empirical analysis based on propensity score matchingThe Pain Outcomes Comparing Yoga vs. Structured Exercise (POYSE) Trial in Veterans With Fibromyalgia: Study Design and MethodsComparative efficacy trials in inflammatory bowel disease: current and future implications for practiceDevelopment of an Efficient ETL Technique for Data WarehousesEthical care requires pragmatic care research to guide medical practice under uncertaintyRisk factors for progression or death after first-line platinum-based chemotherapy in real-world patients in the USA with ovarian cancer from 2011 to 2018Optimizing the Start Time of Biologics in Polyarticular Juvenile Idiopathic Arthritis: A Comparative Effectiveness Study of Childhood Arthritis and Rheumatology Research Alliance Consensus Treatment PlansPragmatic Considerations in Incorporating Stakeholder Engagement Into a Palliative Care Transitions StudyInforming Healthcare Decisions with Observational Research Assessing Causal Effect: An American Thoracic Society Statement Not Ready for ImplementationDesign and methods of the Care Management for the Effective Use of Opioids (CAMEO) trialPostextubation Respiratory Support: Is High-Flow Oxygen Therapy the Answer?*A comparison of parametric propensity score‐based methods for causal inference with multiple treatments and a binary outcomeFactors associated with participation over time in the Avon Longitudinal Study of Parents and Children: a study using linked education and primary care dataEvidence-Based Global HealthEvidence-Based Global HealthPlayers in the Development of of efficacy better clinical outcomes for patients with than with to real-world evidence of comparative effectiveness research to decision for of Veterans and of Outcomes for Comparative Effectiveness Research Using Development and the practice of data quality evaluation in a national clinical data research network through a systematic in the of real-world of to the of intervention with real-world from pragmatic clinical of in for Evaluation Research: trial design in the of associated with clinical the of and on chronic a randomized Data in the National Program It the of Risk Factors for to of for multiple a Effectiveness quality of and after a systematic of the systematic and and for patients medical from the United results from Health on in the Use of in the United evidence Comparative effectiveness research and of 2 Characteristics of Randomized in the in A Comparative Effectiveness Study effectiveness of a and a for the of to Health from Health of and for in the of Comparative effectiveness outcomes aspects of clinical trials of for chronic care for with 2 a critical Comparative Effectiveness Trial to and of Comparative Effectiveness for and to into a and implementation study Clinical Research on Health of the Health Risk and Use with for a a to a patient the research to improve community in a for developing and in a Research for quality of progression in with cancer primary or Randomized Trials in Health A Systematic and of research requires a of research of and under uncertainty on and comparative effectiveness study of for of for Treatment in patients with a comparative effectiveness to the and important health outcomes for comparative effectiveness research: An systematic and of low and Effectiveness Research in Respiratory and effectiveness clinical trials to of and health in with and their study for Trial , a our approach to clinical research in uptake of comparative effectiveness and outcomes research from characteristics of the of Health and the and Health in Research in the in of Clinical Trials and influence the clinical performance of A systematic and comparative effectiveness research cannot a approach for health data and study the Childhood Arthritis Rheumatology Research Alliance Juvenile Idiopathic Arthritis Consensus Treatment Effectiveness Research in and Effectiveness of in With in outcomes research for effectiveness of in patients with A of and in implications of using real-world evidence in comparative effectiveness research: from and its impact on and of With to and in and An Medicine to in of and in Observational With Assessing a clinical An analysis of trials in Clinical Trial and use of comparative effectiveness research findings in care results from a national and on engagement in comparative effectiveness on comparative effectiveness research and research stakeholders to inform comparative effectiveness synthesis for the of trial evidence when is and of comparative effectiveness research has not in