Volume 20, Issue 2 , Pages 157-163, February 2009
Improving Quality and Patient Safety by Minimizing Unnecessary Variation
Article Outline
Quality and safety in health care have proven difficult to precisely define and measure. In other fields, quality is defined as the absence of unnecessary variation and process improvement efforts are gauged by their ability to reduce variation. This article explores how this definition can be applied to various attributes of image-guided procedures.
DESPITE continual increases in the resources devoted to health care, outcomes remain far less than optimal. Although perfection is an unattainable goal, there is every reason to believe there is substantial room for improvement. The first article in this series (1) reviewed process improvement strategies and illustrated how every data-driven strategy is based on the scientific method. The second article (2) offered a systematic approach to choosing process improvement projects. The third article (3) illustrated how system performance in health care remains tightly linked to human performance and how human performance can be measured, and introduced the topic of variation. This final article will examine how excessive variation diminishes quality and patient safety.
Quality and Safety Are Linked
The frequency, severity, and cost of medical errors have made patient safety a high-profile issue (4, 5). Quality and patient safety are clearly linked, as quality errors lead to unsafe practices and procedural complications are often linked to quality lapses. Although process improvement will never yield systems that are completely error-free, we can still strive to decrease the frequency, minimize the burden, and improve the early detection of errors (6).
High-reliability industries such as air transport, nuclear power, and electronics manufacturing have long understood how safety and quality add value (7). The previous articles in this series (1, 2, 3) advocated studying these industries and adapting their methods to medicine in general and interventional radiology in particular (8, 9, 10). We suggest that the principles that guide the quality control programs in these and other high-reliability industries can be applied to image-guided procedures.
The First Step Is to Define Quality in Measurable Terms
The medical literature contains numerous articles describing the need to develop measures of quality and how such metrics might improve quality (11, 12, 13). As shown in Table 1, quality involves multiple attributes of a product or process. Although one can consider numerous attributes, it is typical to focus on a few attributes that are considered critical to quality. High-reliability industries use operational definitions of these attributes to measure progress toward quality goals (19, 20, 21). The operational definitions focus on observable characteristics and include a description of how the characteristic is to be measured. For example, based on the knowledge that the performance of dialysis and other central venous catheter–related tasks depends on the position of the catheter tips relative to the right atrium (22, 23), one might define catheter tip position relative to the cavoatrial junction as a metric that is critical to quality. Recent work by Baskin et al (23) illustrates how an operational definition of optimal catheter tip position was developed by examining the distance between the catheter tip and carina on supine radiographs.
Table 1. Dimensions of Quality for Tunneled Dialysis Catheter Placement (14, 15, 16, 17, 18)⁎
| Dimension | Process | Product | ||
|---|---|---|---|---|
| Query | Definition | Query | Definition | |
| Performance | Will the process perform as planned? | Tunneled catheter placed in time for next dialysis appointment | Will the product perform as planned? | Catheter provides flow rates >300 mL/min |
| Reliability | How often does the process fail? | Unable to place tunneled catheter in time for dialysis appointment | How often does the product fail? | Frequency that flow rate is <300 mL/min |
| Conformance to standards | Does the process proceed as planned? | Catheter placed via jugular rather than subclavian vein | Is the product made as planned? | Catheter tip 2 cm below cavoatrial junction† |
⁎Adapted from Montgomery et al (15). That text includes other dimensions of quality such as durability, serviceability, aesthetics, features, and perceived quality. |
†The specification that the catheter tip should lie 2 cm below the cavoatrial junction was chosen to approximate the observed practice of many interventional radiologists. It is also congruent with the Kidney and Dialysis Outcomes Quality Initiative recommendation that the distal tip of the catheter should be within the right atrium (14). Catheter tip position is a controversial topic and some agencies recommend that no catheter tip should be positioned below the superior margin of the pericardium to minimize the risk of cardiac tamponade caused by the catheter tip eroding through the vessel wall (16, 17, 18). |
It is crucial to realize that such metrics are at best faint reflections of quality. The metric is not quality per se, but rather a means of assessing the attribute considered critical to quality. It is clear that catheter tip position varies with respiration and patient position (22). It is also obvious that measuring the vertical distance between the carina and the catheter tip provides only indirect evidence of the relationship between the catheter tip and the cavoatrial junction (24). Such problems are an inherent issue with any measurement, and indeed measurement theory states that it is impossible to measure any parameter with absolute precision (25). We should recognize that our measurements are merely an attempt to infer quality. Problems occur when one begins to believe that the metric directly reflects quality. Such mistaken beliefs will cause one to focus on the metric itself and lead to instances in which one aspect of quality is improved but other facets suffer (19, 20). For example, one could use digital subtraction angiography or computed tomography to better assess how the central venous catheter tip is positioned relative to the cavoatrial junction in every patient, but these would clearly subject the patient to additional risk such as that incurred from exposure to more ionizing radiation.
Metrics and Tolerance Bands Are Frequently Used to Assess Quality
To illustrate these points, we ponder how a future pay-for-performance program (26) might use such critical-to-quality metrics to identify and reward exemplary performance. This hypothetical pay-for-performance program would assess catheter tip position with the aim of having dialysis catheter tips positioned 2 cm below the cavoatrial junction. The program designers would recognize that variation is part of any process and thus allow a ±2 cm tolerance band around that target on the postprocedural supine radiograph. Equipped with this operational definition, the inspector arrives and audits your practice. The inspector collects radiographs from your procedures, extracts data, and compares his measurements to the pay-for-performance standard. Based on his measurements, the inspector makes a decision regarding the quality of each catheter placement. If the inspector judges the catheter tip to be within the specified tolerance band, the catheter passes inspection. If the audit reveals that you meet the quality target in a specified percentage of cases, you earn a bonus. If the inspector's measurement falls outside this tolerance band, the catheter fails inspection, and if this occurs frequently, you lose the bonus. This is a typical step-function approach to quality (Fig 1a).
Figure 1.
Illustration of the use of tolerance bands and value curves to estimate production costs. (a) Typical approach to quality control. A target value of 2 cm below the cavoatrial junction is set and a tolerance band of ±2 cm is used to determine which items should pass inspection. In this model, it costs $100 to place the catheter. Catheters that fail inspection generate costs of $275, as it costs $100 to place a new catheter and $75 to remove that catheter. The lower and upper specification limits are marked by dotted lines labeled LSL and USL. (b) Illustration of how revision of marginal items might impact costs. In this model, catheters that are just outside the tolerance band can be reworked at a cost of $50. (c) Illustration of a Taguchi value function (27) in which item cost was calculated for each data point. The result was converted to a score by using a Taguchi value function: cost = kD2 + c, Where D is the difference between the target and observed catheter tip positions, k is 0.25, and c is $100. This function yields values of $100 when the length is at the target value and $200 when the catheter tip is positioned at the edge of the tolerance band.
Because you accept the notion that incorrectly positioned catheters can cause complications and you also wish to be identified as an exemplary supplier in the pay-for-performance program, you use the fluoroscopic image and redouble your efforts to insure that each catheter will pass inspection. If, toward the end of the procedure, it appears the catheter tip might fall outside the tolerance band, you are faced with a decision. You might decide to revise the catheter and monitor progress via fluoroscopy. Revision is associated with a small added cost attributable to the time and radiation dose needed to make the revision. For catheters that appear to fail by a large margin, you might decide to completely replace the catheter. Replacement is associated with a greater additional cost. The different costs of revision and replacement create a series of intermediate steps at the edges of the tolerance band (Fig 1b).
Errors in measurement introduce additional possibilities. The auditor may decide to define the carina or the catheter tip position slightly differently than you do. As a result, the auditor's measurements differ from yours by 10%–20%. You anticipate this potential problem and decide to improve your odds of passing inspection by revising catheters that appear to deviate from the pay-for-performance standard by 1.5 cm. Such problems with measurement and decision-making are common in nearly every human endeavor. This problem led Taguchi (27) to reconsider the traditional step-function approach to quality. He proposed the use of a U-shaped curve to best model how costs increase as one moves away from the target value (Fig 1c). This continuous value function suggests two strategies for maximizing the value of your efforts (Fig 2). First, you could strive to identify the ideal location for the catheter tip and adjust your process so that results are centered around that target. In the current example, the pay-for-performance program has defined that target point, but such occurrences are rare in interventional radiology. Still, one typically has a goal for every step of a procedure and aims to meet every one of these intermediary goals. Second, you could strive to reduce variation because, even if your process is centered at the target, excessive variation will markedly degrade value (Fig 2). Quality improvement programs stress that the key to improving quality is reducing variability (7). The rationale is that quality is tied to consistent performance in two areas: (i) providing the optimal balance between results obtained and costs incurred and (ii) meeting the customer's expectations. Delivering quality is far easier if you are able to bring your process into a state of statistical control (7, 15, 19, 20). High-reliability manufacturers and service providers continually assess their processes, and because the ideal target value is rarely known, they typically focus on decreasing variation.
Figure 2.
Illustration of how central tendency and variability can influence the costs for dialysis catheter placement. Cost was modeled using the Taguchi value function (27) defined in Figure 1. Other examples in which costs will likely increase geometrically with positioning errors include IVC filters and vascular stents. (a) Taguchi value function. (b–e) Normal distribution was used to generate 300 data points. The central tendency and variability for each distribution were defined. Histograms are shown to illustrate how changing the mean and standard deviation of each data set distorts how that data set overlaps with the target value, as well as upper and lower specification limits. The individual data points were then used to calculate the cost of each data point with the Taguchi value function. The cost for the 300 items were then summed to determine how changing the central tendency and variability would influence overall costs. Although it is common to focus on improving the mean value of a process, these examples indicate that increases in variability can also lead to substantial increases in cost. Indeed, when comparing b and e, the increasing costs associated with a shift in the process mean were nearly offset by decreasing variation.
Applying These Concepts to a Broader Array of Image-guided Procedures
We contend that value curves and minimizing variability can be applied to a wide variety of image-guided procedures. Before any procedure, you formulate a plan and use the image as a feedback device to monitor whether the procedure is proceeding according to plan. The plan includes target positions for the needle, guide wire, catheter, or other devices, as well as tolerance bands surrounding those positions. The image is monitored and used as an information source to drive a steady stream of decisions. Decisions can be considered as a nested series of binary events. The first decision is whether the result is acceptable; if so, you move to the next segment of the overall plan. If the result is considered unacceptable, you are faced with another decision. You must decide to bring the result in line with the predetermined plan. You might also decide that, even though the observed result might previously have been considered unsatisfactory, in light of current conditions, it is acceptable. In essence, you changed your plan and subsequent steps toward the next goal should account for this change. In most cases, the position of the target, the size of the surrounding tolerance band, and the decision whether to proceed or revise are made at a subconscious level. When conscious decisions are made, the process resembles the steps used to construct a Taguchi value curve (27). For these conscious decisions, you view the image and extract data, which are weighed against the risks and benefits of, and alternatives to, the current position. It is clearly impractical to derive a complete Taguchi value curve for each decision point because that would require gathering data on numerous factors such as the costs of all complications (both immediate and delayed) along with their relative probabilities. Instead you use the information from the image along with your experience and clinical judgment to estimate the target point and slopes of the U-shaped function.
The process of planning, assessing concordance with the plan, and adjusting the system is commonly known as a feedback control loop. Feedback control loops can be explicitly analyzed by using control theory, and that analysis illustrates how imaging guidance can improve value during central venous catheter placement. Every process starts with a goal. In this case, it is the catheter tip position. Images obtained near the end of the procedure are used to assess conformance with that goal. If one interprets the image and decides that the catheter tip falls outside the target zone, a decision is made to adjust the catheter, and the central nervous system sends a signal to motor neurons to push in or pull back the catheter. A new image is obtained and interpreted. The sequence ends when the catheter tip is determined to be within the target zone. This is an example of a closed-loop feedback system in which the controller (ie, the physician) is able to adjust system output in near-real time (Fig 3). Bedside catheter placement without fluoroscopy typically operates as an open-loop system whereby the physician must wait until the procedure has ended and the chest radiograph has been obtained to assess catheter tip position. For bedside catheter placement, adjusting the catheter tip position requires a second, albeit truncated, procedure. Properly constructed closed-loop control systems are far more effective in reducing variation than open-loop systems (28). For dialysis catheter placement, the added value of the closed-loop system comes at a cost. This cost includes the resources required to transport the patient to the fluoroscopy suite and operate the fluoroscopy equipment. Costs also include the risks associated with the ionizing radiation used to create the feedback loop.
Figure 3.
Closed feedback loop model for catheter tip positioning during an image-guided procedure. The control unit uses the radiographic image to compare the current catheter tip position to the desired catheter tip position. The control unit also contains a decision-making algorithm that determines whether to attempt to move the catheter tip. That result is transmitted to the plant, where a motor control program converts the decision into a series of commands that are transmitted to motor neurons (31). The plant includes the external portion of the catheter that is advanced or retracted to reposition the catheter tip. The catheter tip position is then reassessed and the cycle repeated if necessary.
Improving Quality by Reducing Variation in Planning and Execution
The Institute of Medicine defines quality as “increasing the likelihood of desired health outcomes” (29), and therefore quality depends on careful planning and skillful execution. Pay for performance, Six Sigma (7), and other process improvement strategies require a series of quality metrics, which in turn require a series of operational definitions. To determine whether any quality/safety improvement program is making progress toward its goals, we suggest assessing planning and execution skills separately.
Planning skills for central venous catheter placement can be assessed by having the physician explicitly state his/her target point for catheter tip position. In the absence of external influences, individual physicians will almost certainly have different target points and acceptable zones. However, external forces such as the medical literature and pay-for-performance programs will undoubtedly create standards. The first step in assessment might use a task model that requires the physician to apply that standard to an actual or simulated patient (Fig 4). The evidence model would compare the observed physician performance with the standard to generate a “planning subscore.” The size of the target zone could also be assessed by providing radiographs of varying catheter tip positions and asking the physician to categorize them as acceptable or unacceptable. Finally, one could use multiple choice questions to determine whether the physician understands the consequences of leaving the catheter tip outside the target zone. Quality begins with a uniform and explicit plan. In addition, it is impossible to measure variation in execution without previous agreement on the plan.
Figure 4.
Task and evidence models for catheter tip planning. (a) In the task model, individuals would be asked to evaluate an image and explicitly report their desired catheter tip position. (b) A single potential response: a black cross is positioned approximately two vertebral bodies below the carina. The x–y position of the cross's center would be recorded and compared with the desired value, where the y value is of greater interest than the x value. (c–e) Illustration of how repeated measures from the same or different individuals might reveal a tightly control process (c) or a planning process that is variable and skewed (e).
Quality during task execution would be measured separately. The task model would require the physician to place a catheter in actual or simulated patients and the evidence model would measure the distance between the observed and optimal catheter tip positions. As outlined earlier, closed-loop feedback systems such as those found in image-guided procedures are expected to yield far less variation than open-loop systems.
Some might argue that any systematic attempt to reduce variability during medical procedures might hinder innovation. Reducing variability not only reduces errors and waste, it also improves the ability to decide whether an improved outcome can be attributed to process changes (3). Inquisitive minds will continue searching for ways that processes and products might be improved. Reducing variation simply improves the probability that small, deliberate changes will be recognized as improvements rather than simply “noise” in the process (30). Process improvement must be governed by data and it will fail if the metrics are poorly conceived or mindlessly followed.
Summary
Throughout this series of articles, we have argued that process improvement follows the scientific method whereby one the existing system and makes predictions about how the current process might be improved. Data are collected and compared with those predictions. These data allow one to make an informed decision about whether the evidence supports the prediction or refutes it. Although it may seem cumbersome to collect and analyze data, the alternative is to allow conjecture and emotion to drive the process. Identifying and reducing the causes of excessive variation in any process will make it easier to observe the benefits of small, incremental improvements. We suggest that the pace of innovation in image-guided procedures will accelerate if we adapt and adopt these proven methods.
References
- . Strategies for improving safety and quality in interventional radiology. J Vasc Interv Radiol. 2008;19:3–7
- . Strategies for choosing process improvement projects. J Vasc Interv Radiol. 2008;19:471–477
- . Assessing system performance. J Vasc Interv Radiol. 2008;19:987–994
- . Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001;
- . To err is human: building a safer health system. Washington, DC: National Academy Press; 2000;
- . Failure mode and effect analysis: FMEA from theory to execution. second ed.. Milwaukee: American Society for Quality; 2003;
- . The Six Sigma handbook. New York: McGraw-Hill; 2003;
- . Is health care ready for Six Sigma quality?. Milbank Q. 1998;76:565–591
- . Design of high reliability organizations in health care. Qual Saf Health Care. 2006;15(suppl 1):i4–i9
- . Applying the lessons of high risk industries to health care. Qual Saf Health Care. 2003;12(suppl 1):i7–i12
- . National Quality Measures Clearinghouse [online database]. www.qualitymeasures.ahrq.govAccessed June 19, 2007
- . Quality-a radiology imperative: report of the 2006 intersociety conference. J Am Coll Radiol. 2007;4:156–161
- . Quality-a radiology imperative: interpretation accuracy and pertinence. J Am Coll Radiol. 2007;4:162–165
- . K/DOQI clinical practice guidelines for vascular access. Am J Kidney Dis. 2001;37(suppl 1):S137–S181
- . Introduction to statistical quality control. fifth ed.. Hoboken, NJ: John Wiley & Sons; 2005;
- . Precautions necessary with central venous catheters. FDA Drug Bulletin. 1989;15–16
- . Central venous catheterization: better and worse. J Intens Care Med. 2006;21:51–53
- . Safe placement of central venous catheters: where should the tip of the catheter lie?. Br J Anaesth. 2000;85:188–191
- . The new economics for industry, government, education. second ed.. Cambridge, MA: MIT Press; 2000;
- . Out of the crisis. Cambridge, MA: MIT Press; 2000;
- . Course No. 617. Scientific methods in administration and management. Washington, DC: George Washington University; 1980;
- . Central venous catheter tip position: a continuing controversy. J Vasc Interv Radiol. 2003;14:527–534
- . Cavoatrial junction and central venous anatomy: implications for central venous access tip position. J Vasc Interv Radiol. 2008;19:359–365
- . The carina as a landmark in central venous catheter placement. Br J Anaesth. 2000;85:192–194
- . Validity (Educational measurement). In: Linn RL editors. Phoenix, AZ: Oryx Press; 1993;p. 13–104
- . Rewarding provider performance: aligning incentives in medicare (pathway to quality health care series). Washington, DC: National Academies Press; 2007;
- . Introduction to quality engineering: designing quality into products and processes. Tokyo: Asian Productivity Organization; 1986;
- . Modern control system theory and design. second ed.. New York: John Wiley & Sons; 1998;
- . Medicare: a strategy for quality assurance. Washington, DC: National Academies Press; 1990;
- . Improving almost anything: ideas and essays. Hoboken, NJ: Wiley-Interscience; 2006;
- . Motor control and learning: a behavioral emphasis. fourth ed.. Champaign, IL: Human Kinetics; 2005;
This work was supported in part by the Society of Interventional Radiology (SIR) Foundation, Barnes Jewish Hospital Foundation (St. Louis, Missouri), and Siemens Medical Systems (Erlangen, Germany). Neither of the authors has identified a conflict of interest.
PII: S1051-0443(08)00998-6
doi:10.1016/j.jvir.2008.10.031
© 2009 SIR. Published by Elsevier Inc. All rights reserved.
Volume 20, Issue 2 , Pages 157-163, February 2009
