Volume 21, Issue 5 , Pages 617-625, May 2010
Guidelines for Establishing a Quality Improvement Program in Interventional Radiology
Article Outline
- Preamble
- Methodology
- Introduction
- Quality in the United States Health Care System
- Quality Assurance Versus CQI
- Systematic Approach to Quality Improvement in Interventional Radiology
- Total Quality Management in Interventional Radiology: An Example
- QA
- CQI
- Conclusion
- Appendix A: Consensus Methodology
- Acknowledgments
- References
- References
- Copyright
Abbreviations: CQI, continuous quality improvement, PDSA, plan do study act, QA, quality assurance
Preamble
THE membership of the Society of Interventional Radiology (SIR) Standards of Practice Committee represents experts in a broad spectrum of interventional procedures from both the private and academic sectors of medicine. Generally Standards of Practice Committee members dedicate the vast majority of their professional time to performing interventional procedures; as such they represent a valid broad expert constituency of the subject matter under consideration for standards production.
The clinical practice guidelines of the Society of Interventional Radiology attempt to define practice principles that generally should assist in producing high quality medical care. These guidelines are voluntary and are not rules. A physician may deviate from these guidelines, as necessitated by the individual patient and available resources. These practice guidelines should not be deemed inclusive of all proper methods of care or exclusive of other methods of care that are reasonably directed towards the same result. Other sources of information may be used in conjunction with these principles to produce a process leading to high quality medical care. The ultimate judgment regarding the conduct of any specific procedure or course of management must be made by the physician, who should consider all circumstances relevant to the individual clinical situation. Adherence to the SIR Quality Improvement Program will not assure a successful outcome in every situation. It is prudent to document the rationale for any deviation from the suggested practice guidelines in the department policies and procedure manual or in the patient's medical record.
Technical documents specifying the exact consensus and literature review methodologies as well as the institutional affiliations and professional credentials of the authors of this document are available upon request from SIR, 3975 Fair Ridge Dr., Suite 400 N., Fairfax, VA 22033.
Methodology
SIR produces its Standards of Practice documents using the following process. Standards documents of relevance and timeliness are conceptualized by the Standards of Practice Committee members. A recognized expert is identified to serve as the principal author for the standard. Additional authors may be assigned dependent upon the magnitude of the project.
An in-depth literature search is performed using electronic medical literature databases. Then a critical review of peer-reviewed articles is performed with regards to the study methodology, results, and conclusions. The qualitative weight of these articles is assembled into an evidence table, which is used to write the document such that it contains evidence-based data with respect to content, rates, and thresholds.
When the evidence of literature is weak, conflicting, or contradictory, consensus for the parameter is reached by a minimum of 12 Standards of Practice Committee members using a modified Delphi consensus method (Appendix). For purposes of these documents consensus is defined as 80% Delphi participant agreement on a value or parameter.
The draft document is critically reviewed by the members of the Standards of Practice Committee, either by telephone conference calling or face-to-face meeting. The finalized draft from the Committee is sent to the SIR membership for further input/criticism during a 30-day comment period. These comments are discussed by the Subcommittee, and appropriate revisions made to create the finished standards document. Prior to its publication the document is endorsed by the SIR Executive Council.
Introduction
“[Quality is] a never-ending cycle of continuous improvement.”
—W. Edwards Deming, father of the Toyota Production System (1)
Quality is not a static goal but a progressively improving state, and interventional radiology is a rapidly moving, technology-driven subspecialty in which high-quality patient care should be the norm. The health care we deliver next year must be better than the health care we deliver today. In order to attain such essential goals, interventional radiology must initiate specialty wide continuous quality improvement (CQI) programs. Ensuring high-quality patient care in interventional radiology is a primary goal and responsibility of the Society of Interventional Radiology (SIR).
Quality in the United States Health Care System
Several published reports have outlined major problems with quality and patient safety in the United States health care system (2, 3, 4, 5, 6). These reports described how fragmented care and lack of reliable information with which to support clinical decisions and evaluate actual performance resulted in direct patient harm (2), massive variation in quality and access (3), and inconsistent care delivery. High-cost procedures are often performed with little or no evidence to demonstrate their long-term benefits (4). Treatment regimens often vary so much among providers and institutions that useful comparisons are difficult to obtain (5). Waste abounds in the health care system because of duplication of service lines, inefficient health care delivery, and massive variation in care delivered (3, 6).
Multiple national and local initiatives have been launched to address these system-wide problems. For example, the Centers for Medicare and Medicaid Services has initiated the Pay for Performance and Physician Quality Reporting Initiative programs, and private payers have incorporated utilization, quality, and outcomes data measurement (7).
Some physician specialty groups have also proactively designed extensive quality improvement programs. The Society of Thoracic Surgeons, for instance, has established standardized metrics and provided members comparative data for more than a decade (8). As interventional radiologists continue to increase their clinical presence, we too must design and implement practical, effective specialty-wide quality improvement programs. Establishment of standardized metrics will allow compilation of comparative data. Analysis of these comparative data will permit identification of best practices and lead to the creation of clinical care pathways that deliver more efficient, higher-quality care. Incremental change and regular data analysis will drive constant improvement of care delivery. Such a cycle is commonly used in industry, where it has led to many of the technologic advances from which we benefit daily. Furthermore, in the fields of aviation and nuclear power, human-factors engineering has repeatedly demonstrated that standard processes improve quality while decreasing the impact of inevitable human error (9).
Quality Assurance Versus CQI
The definition of quality varies widely and depends on the product being evaluated and its use. In industry, the factors contributing to overall quality have been referred to as the “dimensions of quality” and include performance, reliability, durability, serviceability, aesthetics, features, perceived quality, and conformance to standards (10). In health care, quality has been described as having three parts: the content (ie, the care delivered and its resultant medical outcome), the delivery (ie, “service” and patient satisfaction with the health care experience), and the cost (11). Because of the subjective nature of health care delivery, patients' value systems and expectations can significantly alter their perception of the quality of the care they receive. A technically successful procedure does not result in high-quality care if it is performed by individuals who are rude to the patient.
Two major approaches have been used to try to ensure high quality in health care in the United States: quality assurance (QA) and CQI. The traditional approach to ensuring high-quality health care, QA, uses thresholds to identify low-quality events. Energy is focused on identifying poor individual outcomes or poorly performing practitioners; no significant effort is focused on continuous improvement of technique, process, or patient care. In order to ensure a quality baseline, hospital credentials committees regularly review events considered below the standard of care (ie, complications) and frequently require that practitioners have successfully performed a minimum number of cases before they are granted specific privileges. These methods, however, only identify negative outliers on a bell curve and thereby create a “statistical tail” (Fig 1a). The concept of a minimal standard of performance is reinforced, and a sense of “good enough” prevails (12). This atmosphere allows below-average practitioners to continue practicing without making improvements as long as their complication rate remains below a locally accepted threshold. This “good enough” mentality is a disservice to patients in an era of ever-advancing health care.
Figure 1.
(a) Diagram of changes associated with a quality assurance program wherein a threshold (quality tail) is used to identify outliers. In this situation there is not significant change in the mean performance. (b) Changes associated with a quality improvement program wherein the emphasis is on a significant shift in mean performance (quality shift).
Whereas QA reacts to individual problem events or problem providers, CQI attempts to anticipate problems and improve processes (13). CQI was initially described by American physicist and engineer Walter Shewhart (14). While working at Bell Laboratories, Shewhart realized that reducing process variation, a methodology currently known as industrial quality control, significantly improved the quality of the final product (14). Shewhart's writings influenced W. Edwards Deming of Toyota fame. The research of Shewhart and Deming ultimately launched the process improvement concepts of Six Sigma (15) and lean manufacturing (16). Six Sigma attempts to decrease long-term defect levels by decreasing variation and employing standard processes. State-of-the-art manufacturing processes now achieve stunningly low failure rates, below 3.4 defects per million opportunities (a commonly used measure of industrial process performance) (15). Lean manufacturing, also known as “lean production,” is a practice that seeks to eliminate steps in a process which add no value (ie, waste)—ie, to achieve more value with less work. This process management philosophy was derived primarily from the Toyota Production System (16).
In health care delivery, we are faced with process challenges similar to those in industrial production: the care we provide is the culmination of many distinct, individual processes—eg, scheduling, transportation, performance of the procedure, recovery, education, discharge, and billing. Therefore, improving individual care delivery processes through CQI can lead to greater cumulative improvement than only addressing individual events through QA (Fig 1b).
For a CQI program to be successful, it must (i) decrease waste and (ii) limit variation. In medicine, two types of waste exist, quality waste and production waste. Quality waste occurs when resources are used and the effort fails to produce the desired outcome (eg, performance of an inappropriate imaging examination provides no additional clinical benefit). Productivity waste occurs when more resources than necessary are expended to achieve a given clinical outcome (eg, as a result of unique practice styles). When total health care resources are limited, waste and variation do harm. For instance, in the care of a patient with an acutely cold leg and loss of arterial pulses, when emergent angiography would be appropriate, performing a lower-extremity radiography series would be an example of quality waste; and performing lysis using a combination of mechanical and chemical approaches when either approach alone would produce equivalent clinical outcomes would be an example of productivity waste.
In health care, one common method to decrease variation, waste, and cost while increasing personal efficiency and clinical effectiveness is the use of treatment pathways or clinical care algorithms. These tools are dynamic, created and regularly updated on the basis of the best evidence available. Regular review of the pathways in light of recent research and institutional outcomes encourages beneficial pathway modifications and continuous improvement. Clinical care algorithms provide the cornerstone of a practical CQI program and facilitate comparative research because a standardized practice enhances the generation of useful, consistent data that are easily collected and analyzed. Small changes in practice standards can be implemented and their effects easily studied. If the effects are positive, the practice standard is modified; if not, the modification is discarded. Such strategies have been well documented to improve overall health care quality (17).
Systematic Approach to Quality Improvement in Interventional Radiology
QA and CQI programs should reflect the underlying dynamics of a radiology department. Different hospitals cater to different patient populations with different needs. While all interventional radiology quality programs will share certain critical concerns, including patient safety, process improvement, and technical and professional expertise, specific programs will vary among institutions. However, a successful interventional radiology quality program will usually have three parts: a CQI program that measures a broad spectrum of performance indicators within a department, a QA program focusing on individual events and providers, and ongoing small-scale quality improvement projects. Suggested steps in establishing a quality improvement program in an interventional radiology department are outlined here.
Form a Radiology Quality Committee
The first step in establishing a successful interventional radiology quality improvement program is to form a quality committee. The committee must include both those who deliver the care and those with the authority to make change, and the membership must be broad enough to reflect all segments involved in the patient health care experience. Important points to consider include the following:
Identify QA Standards
Professional QA has been practiced within interventional radiology for years. Examples of QA programs are hospital-based peer review, continuing medical education, and reaccreditation. SIR has published extensive QA documents among its clinical practice guidelines; these documents outline recommended standards of care for individual procedures. Evidence-based standards such as these in combination with internal benchmarks can provide the structure necessary to establish a robust QA program.
Understand the System and Define its Processes
Health care delivery is a complex system containing multiple distinct processes. In order to measure, analyze, and ultimately improve care, a QA committee must dissect the system into its separate processes, which are then individually addressed. Flowcharts, cause-and-effect diagrams, and Pareto charts can be very helpful by showing critical steps, problems, and potential bottlenecks, all areas of significant potential improvement. A flow chart is a common type of chart in which each step in a process resides within a shape, connected in order by arrows. Flowcharts are used to reduce and visually analyze a process into discrete steps that can individually be measured and managed. Cause-and-effect diagrams (also called fishbone diagrams or Ishikawa diagrams) are tools used to show the potential causes of an event. A Pareto chart is a special type of bar chart in which the values being plotted are arranged in descending order. The graph is accompanied by a line graph that shows the cumulative totals of each category, left to right. One example of a complex system is emergent computed tomographic (CT) imaging and intervention in stroke patients. As timeliness directly affects outcomes, accurate information must be obtained and proper triage performed very quickly. Small delays in multiple processes can accumulate and result in significant patient harm. Critical steps can be chosen, and potential bottlenecks labeled on a flow diagram, to demonstrate patient movement through the system and delineate the various processes involved (Fig 2).
Figure 2.
Flowchart of movement of an emergency room patient with a possible stroke through initial clinical and imaging evaluation. Four subprocesses are identified, and four corresponding metrics are created. Because avoidance of delays is critical to good outcome of patients with stroke, the metrics were defined as time required to complete each process. (Available in color online at www.jvir.org.)
Establish Metrics and Set Threshold Values
After identifying critical departmental processes and the various steps in these processes, the quality committee must establish metrics that will provide the most useful data. The importance of metric selection must be underscored. Good metrics provide reliable, timely, and reproducible data that accurately represent the process in question. Flowcharts are often helpful in determining the critical areas on which metrics should focus. In our emergent stroke imaging example (Fig 2), four critical subprocesses were identified and the corresponding metrics created. National guidelines such as those issued by the Joint Commission, National Quality Forum best practices, and Center for Medicare and Medicaid Services Physician Quality Reporting Initiative can also serve as good sources of metrics (18). Although current systems provide little financial incentive for providers to report data (Center for Medicare and Medicaid Services has a 2% of estimated total allowable charges incentive payment [19]), required external reporting is only expected to become more common in the future.
The quality committee should consider establishing threshold values for certain metrics, such that if the value of a metric is beyond the acceptable threshold, not only is the value recorded but a cascade of further investigation is triggered. The threshold values can be based on external benchmarks (eg, those of SIR) or internal benchmarks and will vary depending on the process in question. For example, although a threshold reporting rate of 100% for adverse events and critical results is reasonable; a similar threshold for successful image-guided percutaneous biopsy would not be.
Collect and Analyze Data
Data-based decision making is a core concept within CQI. Unfortunately, though, data collection often is challenging: for example, the institutional electronic medical record may have limitations, as may the radiology information system. Assistance from the institution's information technology professionals may be required to efficiently access and compile an institution's collected data. Furthermore, although manual data collection is less efficient and more costly than electronic data collection, and has increased potential for human error, it is sufficient to start a CQI program. Simply graphing data over time and creating an annotated run chart can be remarkably valuable. Returning to our example of emergent CT imaging for stroke, a run chart could be created to evaluate transport time. Baseline data could be collected for metric 2—the time from when the transporter was called to when the patient arrived at the CT scanner—and presented graphically using a basic annotated run chart (Fig 3).
Figure 3.
Run chart showing data from a continuous quality improvement study of the transport time for patients from the emergency room to the CT imaging unit (metric 2 in Fig 2). (Available in color online at www.jvir.org.)
Initiate a Quality Improvement Project
A good strategy for quality improvement in health care is rapid cycle improvement. Rapid cycle improvement is simple, requires no special training, is easily implemented, and uses the scientific method. Initially described by Shewhart (14) and later modified by Langley (20), rapid cycle improvement consists of ongoing “plan, do, study, act” (PDSA) cycles. The rapid cycle PDSA concept provides a framework for incremental data-driven system change. Returning to our example of emergency imaging for stroke patients, here is how a PDSA cycle might play out: Run chart showing data from a CQI study of the transport time for patients from the emergency room to the CT imaging unit (metric 2 in Fig 2). Addition of a second transporter on day 14 resulted in decreased transport time. (Available in color online at www.jvir.org.)
Figure 4.
One can easily appreciate how the PDSA method provides a simple way to continuously improve quality.
PDSA-based quality improvement strategies are not restricted to clinical operations—they can also be used to improve the performance of individual health professionals. A large interventional radiology practice will often have much variation in how individual physicians perform procedures. In such cases, data analysis leading to incremental changes can result in improvement in processes such as interpreting images or performing procedures. For example, a PDSA study could be constructed to evaluate the entire process of uterine artery embolization for symptomatic leiomyomas. A complete analysis would extend from initial patient contact through to clinical outcome. Such a study would be somewhat more complex, encompassing operational and clinical metrics. Initially, a flowchart would be created and metrics chosen that best reflect subprocesses of greatest interest and importance. Examples would include:
Process Metrics
Physician-based Metrics
Data could be collected to determine operational and professional proficiency. Individual physicians could be compared with colleagues and national benchmarks. Analysis of these data may reveal certain patterns of practice that are advantageous. These could then be implemented by all physicians in hopes of decreasing variation and improving patient outcomes. Additional data could be collected to verify whether such improvement in fact occurred.
Establish a Framework of Ongoing CQI
CQI is a fluid science. Metrics and focus will change as improvements are realized and required reporting standards shift. Therefore, a system that is flexible and easy to use is critical. Many excellent articles describe the creation of CQI programs within academic imaging departments and may serve as sources of ideas and templates (22). Successful quality improvement projects can be implemented through standing order sets and best-practice guidelines. Some initial resistance may be encountered by those who believe algorithms and shared clinical pathways are “cookbook” medicine, stifling innovation and creativity. In fact, though, the opposite is true. Once metrics are in place and rapid cycle improvement (the PDSA process) is initiated, innovation is more effective and more easily measured. Data accurately express the impact and usefulness of change, often revealing logistical, manpower, distribution, and performance problems within the department.
Total Quality Management in Interventional Radiology: An Example
Implementing CQI practices in a functioning interventional radiology department can be challenging. In general, though, a good framework for a thorough quality management program is one that includes (i) QA, (ii) ongoing focused PDSA cycles, and (iii) a global CQI program. QA focuses on individual events and providers, PDSA focuses on specific processes, and CQI deals with the entire system or department and the complete cycle of patient care—before, during, and after the procedure. An example is presented here:
QA
Ongoing PDSA Projects
CQI
The basic structure described here can be used as a template on the basis of which to create a quality program tailored to an individual institution. Additional factors that must be taken into account when the initial proposal is crafted for a department-wide quality improvement effort include institution-specific reporting requirements and particular departmental needs.
Conclusion
Providing excellent patient care is important to all interventional radiologists. In the coming era of quality-driven health care, those who can prove their expertise will be rewarded with patient referrals and third-party reimbursements. To successfully practice quality-driven health care, physicians must understand and thrive in an environment of process improvement and outcomes metrics. They must share individual, group, department, and hospital data to demonstrate increased value for patients. This requires robust and flexible systems to collect, analyze, and process data. Physicians must also make continuous improvements and track their impact in the relentless pursuit of perfection. The era of quality-driven health care provides tremendous opportunities for interventional radiologists to showcase the field's value, build credibility, and ensure the survival and growth of the specialty. To those who remain skeptical, consider another quote commonly attributed to W. Edwards Deming, a man ignored in Detroit but embraced in Tokyo: “It is not necessary to change. Survival is not mandatory.”
Appendix A: Consensus Methodology
Reported complication-specific rates in some cases reflect the aggregate of major and minor complications. Thresholds are derived from critical evaluation of the literature, evaluation of empirical data from Standards of Practice Committee members' practices, and, when available, the SIR HI-IQ System national database.
Consensus on statements in this document was obtained utilizing a modified Delphi technique (1, 2).
Acknowledgments
Joseph R. Steele, MD, authored the first draft of this document and served as topic leader during the subsequent revisions of the draft. Sanjoy Kundu, MD, FRCPC, is chair of the SIR Standards of Practice Committee. John F. Cardella, MD, is Councilor of the SIR Standards Division. Other members of the Standards of Practice Committee and SIR who participated in the development of this clinical practice guideline are (listed alphabetically): John F. Angle, MD, Ganesh Annamalai, MD, Stephen Balter, PhD, Daniel B. Brown, MD, Danny Chan, MD, Christine P. Chao, MD, Timothy W.I. Clark, MD, MSc, Horacio D'Agostino, MD, Brian D. Davison, MD, B. Janne D'Othee, MD, Peter Drescher, MD, Debra Ann Gervais, MD, S. Nahum Goldberg, MD, Eric J. Hohenwalter, MD, Maxim Itkin, MD, Sanjeeva P. Kalva, MD, Arshad Ahmed Khan, MD, Neil M. Khilnani, MD, Venkataramu Krishnamurthy, MD, Patrick C. Malloy, MD, J. Kevin McGraw, MD, Tim McSorley, BS, RN, CRN, Philip M. Meyers, MD, Steven F. Millward, MD, Charles A. Owens, MD, Darren Postoak, MD, Dheeraj K. Rajan, MD, Anne C. Roberts, MD, Tarun Sabharwal, MD, Cindy Kaiser Saiter, NP, Marc S. Schwartzberg, MD, Nasir H. Siddiqi, MD, LeAnn Stokes, MD, Timothy L. Swan, MD, Raymond H. Thornton, MD, Patricia E. Thorpe, MD, Richard Towbin, MD, Aradhana Venkatesan, MD, Thomas Gregory Walker, MD, Joan Wojak, MD, and Darryl A. Zuckerman, MD.
References
- . Out of the crisis. 1st ed.. Cambridge, MA: MIT Press; 2000;
- . To err is human: building a safer health system. Washington, DC: National Academy Press; 2000;
- . Dartmouth atlas of healthcare 1999. Chicago: American Hospital Association; 1999;
- . Applying evidence-based imaging to policy: the Washington State experience. J Am Coll Radiol. 2009;6:366–371
- Transarterial therapy for hepatocellular carcinoma: which technique is more effective? (A systematic review of cohort and randomized studies). Cardiovasc Intervent Radiol. 2007;30:6–25
- . Medicare: focus on physician practice patterns can lead to greater program efficiency. Washington, DC: GAO; 2007;
- . Physician Quality Reporting Initiative (PQRI) 2009. www.cms.hhs.gov/PQRI/Downloads/2009_PQRI_MeasuresList_030409.pdfAccessed April 14, 2009
- . STS Database. www.sts.org/section/stsdatabaseAccessed April 13, 2009
- . Strategies for improving safety and quality in interventional radiology. J Vasc Interv Radiol. 2008;19:3–7
- . Competing on the eight dimensions of quality. Harvard Business Review. 1987;65:101–109
- . An introduction to quality assurance in health care. New York: Oxford University Press; 2003;
- . The role of quality assurance versus continuous quality improvement. J Vasc Surg. 1998;28:378–380
- . Continuous quality improvement for radiologists. Acad Radiol. 2004;11:155–161
- . Economic control of quality of manufactured product. Milwaukee, WI: American Society for Quality Control; 1980;
- . The Six Sigma black belt handbook. New York: McGraw-Hill; 2005;
- . The machine that changed the world: how Japan's secret weapon in the global auto wars will revolutionize western industry. 1st ed.. New York: HarperPerennial; 1991;
- . Improving quality and patient safety by minimizing unnecessary variation. J Vasc Interv Radiol. 2009;20:157–163
- . National Patient Safety Goals (Accreditation Program: Hospitals). www.jointcommission.org/NR/rdonlyres/31666E86-E7F4-423E-9BE8-F05BD1CB0AA8/0/HAP_NPSG.pdfAccessed June 24, 2009
- . 2009 PQRI–Incentives Payments. http://www.cms.hhs.gov/PQRI/25_AnalysisAndPayment.aspJune 30, 2009
- . The improvement guide: a practical approach to enhancing organizational performance. 2nd edition. San Francisco: Jossey-Bass; 2009;
- . Improving healthcare with control charts: basic and advanced SPC methods and case studies. Milwaukee, WI: ASQ Quality Press; 2003;
- . Strategies for establishing a comprehensive quality and performance improvement program in a radiology department. Radiographics. 2009;29:315–329
- . General Radiology Improvement Database. https://nrdr.acr.org/portal/GRID/Main/page.aspxAccessed January 4, 2010
References
None of the authors have identified a conflict of interest.
PII: S1051-0443(10)00089-8
doi:10.1016/j.jvir.2010.01.010
© 2010 SIR. Published by Elsevier Inc. All rights reserved.
Volume 21, Issue 5 , Pages 617-625, May 2010