The Dutch Institute for Clinical Auditing

For the medical community, information on care processes and outcomes of their daily clinical practice is often lacking. An important tool in gaining insight and improving health care quality is clinical auditing, defined as the systematic analysis of processes and outcomes of medical care with the ultimate aim of improvement. The concept was introduced over a century ago by Dr. Ernest Amory Codman.1 His “end-result theory” states: “that every hospital should follow every patient it treats, long enough to determine whether or not the treatment has been successful, (…) with a view to preventing similar failures in the future.” In the Netherlands, one of the leading organizations that facilitates clinical auditing is the Dutch Institute for Clinical Auditing (DICA). DICA was founded in 2010, at a time when the demand for transparent, hospital-specific performance information was growing. Simultaneously, professional organizations wanted to redirect the performance discussion from merely procedural volume to a broader outcome-based evaluation.2 The main goal of DICA is to gain better outcomes for patients by measuring quality of care, giving benchmarked feedback to clinicians, stimulating short-cycled improvement initiatives, enabling external transparency, and reducing healthcare costs. This article provides insight into how Codman's clinical auditing concept has been implemented on a nationwide scale in the Dutch healthcare system. DEVELOPMENT Origins and Organization In the Netherlands, the Dutch Colo-Rectal Audit (DCRA) was the first nationwide professional-driven initiative to provide medical teams with benchmarked, hospital-specific performance information, offering new opportunities to improve their care.3 It was launched in 2009 as an initiative of the Association of Surgeons of the Netherlands. The DCRA served as a blueprint for subsequent DICA audits. Figure 1 visualizes DICA's governance structure. A key feature is the leading role of those “personally engaged in the activity concerned”: the clinicians. The Scientific Committee (SC) determines audit objectives and dataset content, takes the lead in interpreting data and functions as a link with other clinicians in the professional associations. DICA's scientific bureau provides methodological and operational support and is backed by a methodological advisory committee and a privacy committee.FIGURE 1: Organizational structure of the DICA.DICA is a nonprofit organization. Since 2016, audits are structurally financed by an umbrella organization of health care insurance companies (ZN). They consider DICA audits as an important source of reliable, independent hospital-specific information. Although ZN participates in the establishment of the transparent indicator sets, it does not influence or have access to audit content or data analyses. Dataset Development and Quality Measurement The SC composes the dataset using (inter)national evidence-based guidelines, taking into account what is meaningful and actionable information for clinicians. Quality indicators meet the requirements of relevance, validity, reliability, and feasibility. In accordance with the Donabdian model, indicator sets consist of structure, process, and outcome indicators.4 Indicators are primarily of use in quality assurance and improvement initiatives by participating hospitals (local quality cycle), although data are also used to evaluate performance at national level (national quality cycle). To ensure their continuing value, the audits’ focuses and quality indicators are critically evaluated on a yearly basis. Data Entry, Storage, and Quality Assurance Depending on the indicators defined, datasets contain information on hospital structure variables, care processes, and patient outcomes. Baseline patient characteristics are included to enable risk adjustment. For data collection, encryption, storage, and processing there is close cooperation with a certified data processor: Medical Research Data Management (MRDM). Participating hospitals retain ownership of their data. All data is subjected to several validation processes: in the web-based registration system, by means of an electronic error report and by in-hospital verification of registered data by an independent third party. Internal Feedback and External Transparency Through a weekly updated, secure, online environment called “MyDICA” participating physicians are provided with hospital-specific feedback, including information on patient, disease, and treatment characteristics. Quality-indicator results are presented in funnel-plots with 95% confidence intervals around the national average or a defined norm, anonymous with regard to other hospitals. If indicators are found relevant and valid, indicator scores can be used as public information, case-mix adjusted when applicable. Hospital-specific information becomes externally available in a stepwise process agreed on by all stakeholders—patients, professionals, payers and government organizations, collaborating in a biannual meeting to define transparent indicator sets. Hospitals authorize the sharing of their indicator scores through a DICA-facilitated web-portal, being unable to change these scores. Quality Improvement at National Level The SC plays a major role in evaluating and interpreting audit data. Between-hospital variation is assessed to identify opportunities for quality improvement, for example, by learning from best practices, or potential controversies guiding new research or guideline improvement. The medical community is informed through an annual report, conferences, and scientific articles. Professional organizations use audit results in their integrated quality policy, for example, to verify adherence to guidelines and quality standards, and to catalyze quality improvement at national level. Outcomes Research Detailed population-based audit data become available for research provided they are complete and verified. Research applications are assessed for relevance, methodology, and availability of data. Types of research questions that have arisen thus far include: evaluation of clinical practice patterns for diagnostics and treatments, reports on the introduction of new techniques, mechanisms behind hospital variation, identification of best practices, and methodological research developing new (composite) measures or risk stratification models. The next section highlights the most important accomplishments. All publications using DICA-data up to 2018 are included in Supplemental Table 1, https://links.lww.com/SLA/B809. ACCOMPLISHMENTS Expansion of the Audits DICA was founded in 2011. Up to December 2017, 21 nationwide audits have been initiated, resulting in the registration of >700,000 patients (Fig. 2). Initially, audits were monodisciplinary and treatment-specific, mainly focusing on cancer surgery. Over time, this has expanded to include nonmalignant diseases, nonsurgical treatments, and evaluation of the entire multidisciplinary care pathway. Audits with additional functionalities include the Dutch Melanoma Treatment Registry, used to study cost-effectiveness and real-world performance of newly developed treatments (immune and targeted therapies) at population level, and the Dutch Head and Neck Cancer Audit additionally evaluating paramedical care, like swallowing and speech therapy.FIGURE 2: The evolution of DICA-facilitated audits: type of audits, number of registered patients, and publications per year.The number of medical associations involved increased from 5 (2011) to 17 (2017). In parallel, the number of clinicians actively involved in the SCs and CABs rose from 32 to 243. The number of transparent indicators calculated from DICA audits rose from 6 (2012) to 161 (2017). National improvements were observed together with a decline in between-hospital variance. DICA data provided a reliable source to ascertain compliance with the volume standards and to study the volume–outcome relationship in several diseases.5,6 Developments in Quality Evaluation Since single-measurement indicators can be less suitable for hospital comparison, new composite measures were developed, such as “failure to rescue” and “textbook outcome” for colorectal cancer, esophagogastric cancer, and elective aneurysm surgery.7 Risk adjustment models were developed to enable valid hospital comparisons. Insights Into National Clinical Practice Variation observed between caregivers, hospitals, or internationally is an important stimulus for in-depth investigation of underlying causes and improvement. For example, the relatively high use of neoadjuvant radiotherapy for rectal cancer in the Netherlands compared to other countries in 2011, led to guideline adjustment in 2014. Rapid implementation was observed, with a decrease in radiotherapy use from 84.2% to 64.4% in 2 years, without compromising oncologic outcomes.8 Other findings include the increased use of minimally invasive surgery at national level.9 Although learning curves were observed, minimally invasive procedures proved to be safely introduced and were considered to be important drivers for postoperative outcome improvement.9 Nationwide Quality Improvements Figure 3 shows 2 examples of process indicator improvements for lung cancer and breast cancer, with an increased national average and a decreased between-hospital variation. For example, the national percentage of patients undergoing lung cancer surgery with a preoperatively recorded clinical TNM stage increased from 75.4% to 98.3% (2012–2016) (Fig. 3A and B). Subsequently, variation decreased from 0 to 100% to 82% to 100%. Several process indicators regarding time to treatment also improved at national level with a decline in between-hospital variation. The number of patients treated within a certain time limit increased for the surgical treatment of carotid stenosis (63%–79%) and lung cancer (41%–71%), the radiotherapeutic treatment of lung cancer (60%–71%), and any treatment of ovarian cancer (77%–86%), rectal cancer (49%–56%), and esophageal cancer (31%–47%).FIGURE 3: Decrease in variation in process indicators for lung cancer (A, B) and breast cancer (C, D).Improvements on the outcome indicator “postoperative 30-day or in-hospital mortality” were seen in various DICA audits. For example, between 2012 and 2015, postoperative mortality decreased from 4.2% to 2.5% after resection for colon cancer and from 2.5% to 1.7% for rectal cancer, resulting in a reduction of >200 care-related deaths per year in the Netherlands.10 In surgical colorectal cancer, treatment improvements were also seen in severe complication rates, “failure to rescue” (the proportion of patients that die following severe complications), and oncologic resection quality. For carotid artery interventions, there was a decrease in “complicated course” from 4.3% to 2.7% (2013–2016). A decrease in hospitalization days was observed after resection for gastric cancer: from a median of 10 days in 2012 to 8 days in 2016. In colorectal cancer surgery, improvements in mortality and complication rates reduced the mean cost per patient from €14,237 to €13,145 (−7.7%).11 Extrapolating to national level, the potential additional savings could be >20 million Euros per 3 years if all hospitals perform as best practices. The observed improvements are likely to be multifactorial—with simultaneous centralization, specialization and introduction of new techniques—and not solely attributable to the audits. Nevertheless, the audits provided insights into performance of the national health care system and individual providers that were previously not available. Reliable, actionable data from the audits form the basis for improvement. PERSPECTIVES Worldwide many initiatives have been developed to monitor and improve healthcare quality by using data. What distinguishes DICA-facilitated audits is the central role of clinicians and their professional societies, close collaboration with other parties involved in health care provision, short-cycled benchmarked feedback, national coverage, and data verification processes to secure data quality. The leading role of clinicians and cooperation with other parties is essential to produce meaningful quality information. DICA data are simultaneously used to provide internal feedback to medical teams at hospital level and to calculate externally transparent indicators. Instead of a merely volume-based discussion, DICA's hospital-specific outcome information has led to a more solid quality of care discussion. Integrating audits into quality assurance policies, for example, via mandatory “participation indicators” and stepwise external transparency has stimulated nationwide participation, allowing better hospital comparisons and the provision of unbiased information, in contrast to registries of more voluntary nature. Today, there are some limitations to the current audits, whether or not to be resolved by DICA. One limitation is the administrative burden associated with data collection. A solution already being worked on is partly automated data collection from Electronic Patient Records. To achieve this, in-hospital workflow redesign will be necessary and close cooperation between doctors and hospital IT providers is indispensable. Increasingly stringent privacy legislation could be a barrier in linking various data sources for audit purposes. Second, finding a balance with demands on transparency is challenging. In DICA's view, caregivers should retain the option of evaluating data internally, allowing medical teams to act on their results in a safe environment. Third, a current barrier to achieving maximum benefit from clinical auditing is the fixed format in which feedback information is offered. More dynamic, interactive systems could optimize information provision for clinicians and stimulate data use for quality improvement cycles and data-driven discussions by medical teams. For that reason, DICA introduced exploratory “Codman dashboards” in 2019, in which clinicians can select certain patient groups and compare their results in these groups with a national benchmark. Generally, a potential flaw of transparent indicators is their potential influence on clinical decisions and risk-averse behavior. There are no indications in the current audits for the latter.12 A more “disease-focused” rather than treatment-specific quality evaluation could contribute to insights into risk-averse behavior. Future perspectives therefore focus on quality evaluation of the entire care spectrum per condition. Recapitalizing, the digital era has brought opportunities to realize Codman's dream, by implementing his clinical auditing model on a nationwide basis. This brings the insights and improvements in healthcare quality he intended 100 years ago. Although there are challenges to be overcome, the DICA example shows important principles for a successful introduction of these audits: clinicians in the lead, close collaboration with various stakeholders in healthcare, use of audit outcomes in improvement initiatives of professional associations and real-world data with timely actionable feedback information for clinicians. Clinical audits can catalyze internal quality improvement, ultimately leading to equal distribution of healthcare quality and accountability to all stakeholders.