Journal of Vascular and Interventional Radiology
Volume 14, Issue 9, Part 2 , Pages S395-S404, September 2003

Reporting Standards for Clinical Evaluation of New Peripheral Arterial Revascularization Devices

  • David Sacks, MD (Chairman)

      Affiliations

    • Corresponding Author InformationAddress correspondence to SIR, 10201 Lee Hwy, Suite 500, Fairfax, VA 22030
    • ,
  • David L. Marinelli, MD
    • ,
  • Louis G. Martin, MD
    • ,
  • James B. Spies, MD
    • ,
  • The members of the Society of Interventional Radiology Technology Assessment Committee

      Affiliations

    • Curtis W. Bakal, MD, Gary J. Becker, MD, Dana R. Burke, MD, Patricia E. Cole, MD, Michael D. Dake, MD, Richard J. Gray, MD, Margaret E. Hansen, MD, Ziv J. Haskal, MD, Robert W. Holden, MD, Michael D. Katz, MD, Lindsay S. Machan, MD, Nilesh H.Patel, MD, and Richard Shlansky-Goldberg, MD

Technology Assessment Committee of the SIR, Department of Radiology, The Reading Hospital and Medical Center, 6th and Spruce Sts, West Reading, PA 19603

Article Outline

Abbreviations:  ABI, ankle/brachial index , PTA, percutaneous transluminal angioplasty , SIP, Sickness Impact Profile , SVS/ISCVS, Society of Vascular Surgery/International Society of Cardiovascular Surgery

 

Endoluminal revascularization devices (balloon angioplasty, atherectomy, laser recanalization, stents) have typically been evaluated in small, poorly controlled series. This has led to overly slow acceptance of some techniques (1) because of doubts by the medical community regarding long-term durability of balloon percutaneous transluminal angioplasty (PTA) versus surgery, and overly rapid acceptance of some techniques because of the excitement of new technology without sufficient proof of efficacy (2, 3). Most of the new peripheral arterial devices are far more expensive than balloon PTA and may be associated with increased complications (4, 5), but there is no reliable documentation of cost-effectiveness of these devices. There is increasing need for proof of efficacy of new devices prior to reimbursement as well as proof of cost-effectiveness in comparison to current technology (6). This article proposes clinical methods of evaluating new revascularization devices so that these issues can be addressed prospectively and rapidly. This article does not address biocompatibility or physical testing of new devices (7). This article relies on the discussion of general principles of evaluation in the preceding article from the Technology Assessment Committee of the Society of Interventional Radiology (SIR) and discusses specific applications of these principles to investigations of new peripheral arterial revascularization devices. While these principles are necessary to evaluate new devices, they also represent an optimal way to evaluate all devices with the goal of creating reliable evidence of device effectiveness (8, 9, 10, 11, 12, 13). Many of the standards that follow are based on previously published SIR and Society of Vascular Surgery/International Society of Cardiovascular Surgery (SVS/ISCVS) standards and this information is summarized in tables.

Back to Article Outline

REPORTING STANDARDS 

Define Patient Selection and Outcomes of Therapy 

It is essential to be clear regarding which patients are being selected for therapy and what a successful outcome of therapy is. A successful outcome might be improvement in claudication, relief of rest pain, healing of gangrene, avoidance of amputation, return to work, improvement in lifestyle, and so forth. For example, if patients are being selected to be treated for claudication, it would be inappropriate to evaluate the success of the procedure by only measuring amputation rates (14, 15, 16). Outcome measures should be appropriate to the goals of therapy. An explicit hypothesis for the study will guide the choice of outcome measures. Success can be measured clinically, hemodynamically, or anatomically, as discussed later.

Pretreatment Evaluation 

This should include objective and quantitative measures of disease severity, measurement of functional status, and description of risk factors for treatment failure or progressive disease.

Disease severity is measured anatomically, hemodynamically, and clinically. Anatomic factors include the severity and location of stenosis, presence of occlusion, ulceration, length of the lesion treated, multiplicity, calcification, eccentricity (defined as lesion lumen in the outer one-quarter diameter of the apparent normal lumen) (7), and quality of the runoff.

The more commonly used method of grading runoff as poor (none or one vessel) or good (two or three vessels) does not describe how to deal with patent but diseased runoff, and ignores the very favorable prognosis of single-vessel straight-line runoff to the foot (17). The grading method of SVS/ISCVS (Table 1) is detailed, but unfortunately cumbersome, and has not been widely accepted (18). This method has been further refined so that it correlates better with actual measurement of runoff resistance, but this refinement is even more awkward to use (19). Grading schemes have been proposed that attempt to combine anatomic criteria into four categories of disease severity for peripheral endovascular procedures (20, 21). These two schemes are similar and can be used in analysis of data. However, as technology improves, the relevance of these schemes may change. In view of the imperfections of all of the categorization schemes, complete anatomic data should be acquired with subsequent stratification of data as appropriate. For the sake of consistency with previous literature, the more thorough method of categorizing the quality of runoff developed by SVS/ISCVS is advocated for the reporting of data. This method may be revised by that organization to make it less cumbersome to employ and to include data on the patency of the pedal arch (22).

Table 1. Weighting of Runoff Arteries
No. of Units Assigned*
Site of Intervention3 21
CIA EIAIIA
EIACFAorSFAPROF
CFA SFAPROF
SFA, POP AT, PT, PER
AT Distal tibialPedal arch
PT Distal tibialPedal arch
PER Pedal archCollaterals to AT and PT
DPPedal arch

Note.—Modified with permission from reference 18.

CIA = common iliac artery, EIA = external iliac artery, IIA = internal iliac artery, CFA = common femoral artery, SFA = superficial femoral artery, POP = popliteal, PROF = profunda femoral artery, AT = anterior tibial artery, PT = posterior tibial artery, PER = peroneal artery, DP = dorsalis pedis artery.

* Points assigned for degree of occlusion: 3 = occluded throughout most of its length; 2.5 = occluded for less than half its length with visible collaterals; 2 = 50%–99% stenosed; 1 = 20%–49% stenosed; 0 = normal or minimal evidence of disease (ie, less than 20% narrowing).

Hemodynamic measures of disease severity rely most heavily on resting and postexercise ankle/brachial index (ABI) and pulse volume recordings (Table 2, Table 3). These are the most objective and reproducible measures, particularly with regard to follow-up studies. These tables may undergo some minor modifications by the SVS (22), in which case the modified tables are expected to be incorporated into this document as well.

Table 3. Clinical Categories of Chronic Limb Ischemia
GradeCategoryClinical DescriptionObjective Criteria
00Asymptomatic, no hemodynamically significant occlusive diseaseNormal results of treadmill*/stress test
I1Mild claudicationTreadmill exercise completed, postexercise AP is greater than 50 mm Hg but more than 25 mm Hg less than normal
2Moderate claudicationSymptoms between those of categories 1 and 3
3Severe claudicationTreadmill exercise cannot be completed, postexercise Ap is less than 50 mm Hg
II4Ishemic rest painResting AP of 40 mm Hg or less, flat or barely pulsatile ankle or metatarsal plethysmographic tracing, toe pressure less than 30 mm Hg
III5Minor tissue loss, nonhealing ulcer, focal gangrene with diffuse pedal ischemiaResting AP of 60 mm Hg or less, ankle or metatarsal plethysmographic tracing flat or barely pulsatile, toe pressure less than 40 mm Hg
6Major tissue loss, extending above transmetatarsal level, functional foot no longer salvageableSame as for category 5

Reproduced with permission (23).

Note.—AP = ankle pressure.

* Five minutes at 2 mph on a 12° incline.

Table 2. Clinical Categories of Acute Limb Ischemia
Doppler Signals
CategoryDescriptionCapillary ReturnMuscle WeaknessSensory LossArterialVenous
ViableNot immediately threatenedIntactNoneNoneAudible AP more than 30 mm HgAudible
ThreatenedSalvageable if promptly treatedIntact, slowMild, partialMild, incompleteInaudibleAudible
IrreversibleMajor tissue loss, amputation required regardless of treatmentAbsent (marbling)Profound, paralysis (rigor)Profound, anestheticInaudibleInaudible

Note.—AP = ankle pressure.

Reproduced with permission (23).

Clinical evaluation of disease severity is performed using the Rutherford criteria (18, 23) for acute and chronic limb ischemia (Table 2, Table 3). For those patients treated for claudication (Rutherford categories 1, 2, and 3) use of a graded treadmill protocol helps to quantify objectively the severity of the claudication (24). The graded treadmill protocol is more effective than a standard constant grade protocol in distinguishing the severity of the claudication in a reproducible manner as well as providing a tool to analyze the effect of interventions (24, 25, 26).

Clinical evaluation can also be performed by means of measures of functional status and quality of life. Functional status is defined as the patients' perception of their ability to walk, their overall physical activity, and their physical function in the community and is directly affected by the severity of peripheral arterial disease. These measures are distinct from objective measures of anatomic or hemodynamic status, which do not necessarily correlate with exercise performance or quality of life (27, 28, 29, 30). General quality of life measures also include physical health, mental health, social functioning, role functioning, general health perceptions, and perhaps level of pain (31) and are measured with a patient questionnaire (24). These questionnaires attempt to measure the overall quality of life of patients and can be particularly useful in assessing the effect of complications, prolonged hospitalizations, and wound healing on patient well being (32). Such measures have been used by Wolf et al in a randomized trial of balloon angioplasty and surgical bypass (33) using the Sickness Impact Profile (SIP) (34).

Ideally, these questionnaires should include evaluation of the broad categories noted above. The measures should be both specific to the disease being treated and general. An efficient and effective approach is to use a general health measure, such as the SIP or the Medical Outcomes Study (24, 35), to supplement a problem-specific questionnaire (31) for peripheral arterial disease that correlates with performance on a graded exercise protocol such as the Walking Impairment Questionnaire (Appendix 1) and the Peripheral Arterial Disease Physical Activity Recall Questionnaire (24, 36). The tests used to evaluate functional status should be the same for all treatment and control groups of comparison trials.

Risk factors for treatment failure or progressive disease are numerous. Some factors may be device specific. General risk factors include smoking, diabetes, age, gender, hypertension, renal failure, hyperlipidemia, previous intervention in the same vessel or in more proximal or distal ipsilateral vessels, use of antiplatelet or anticoagulant medications, and exercise status.

Some factors, such as cardiac and pulmonary status increase the morbidity and mortality of procedures rather than influence patency. Ideally, these risks should be identified and graded (13, 18, 37) so that morbidity and mortality can be compared between endovascular and surgical procedures for patients with comparable risks.

Treatment Description 

Details of balloon pressure or duration of inflation are most appropriate for registry data. It is difficult to dictate which treatment data need to be reported. At a minimum, the data provided (such as the choice of balloon size with respect to vessel size) should be adequate for the study to be duplicated by others.

Post-treatment Evaluation 

Follow-up periods are defined as immediate (1–30 days after the interventional procedure), short-term (30 days–12 months after the procedure), and long-term (greater than 12 months after the procedure) (Table 4). Success is evaluated with use of anatomic, hemodynamic, and clinical criteria (Table 5) (18, 21, 23). Although what we are usually interested in is clinical success (ie, did the patient get better), clinical success can be highly subjective (38). Therefore, measures of anatomic and hemodynamic success are used as objective but indirect measures of procedure success, which is defined as an improvement in patient outcome. Improvement in patient outcome is evaluated with postprocedure comparisons of functional status and exercise performance (24, 27).

Table 4. Follow-up Periods
Immediate0–1 month
Short term1–12 months
Long term>12 months
Table 5. Definitions of Success
I.Technical: Meets the criteria for both anatomic and hemodynamic success in the immediate postprocedure period
A.Anatomic: <30% final residual stenosis measured at the narrowest point of the vascular lumen

Continued anatomic: < 50% recurrent stenosis

B.Hemodynamic: ABI or thigh/brachial index improved by 1.0- or greater above baseline and not deteriorated by >0.15 from the maximum early postprocedure level, or pulse volume recording distal to the reconstruction maintained at 5 mm above the preoperative tracing (only for patients with incompressible vessels)


II.Clinical: Immediate improvement by at least 1 clinical category

Sustained improvement by at least 1 clinical category

Patients with tissue loss (categories 5 and 6) must move up at least 2 categories and reach the level of claudication to be considered improved (18)

The choice of which measure of procedure success to use depends on the outcomes criteria defined in the study design. For some patients, limb salvage is more important than anatomic or hemodynamic definitions of success. Data on patency, improvement, and continued anatomic and hemodynamic success are presented in life tables (18). Note that tables are constructed on the basis of intention to treat, and technical failures must be included in the table. Intention to treat is particularly important for randomized clinical trials. Intervals of assessment need to be frequent initially (1 month, 3 months, 6 months, 12 months, yearly). In the first few postoperative weeks, there may be a high occlusion rate due to inappropriate patient selection or technical error. It is important to estimate the effect of this phenomenon so that long-term patency can be viewed with the effect of these early occlusions taken into account. For this it is helpful to have shorter review intervals in the early stages (39).

The use of binary categories of success and failure is necessary because of the traditional use of these categories in the literature. However, continuous data loses information when forced into discontinuous categories (40). Therefore, it is necessary that data also be reported as average percentage stenosis, ABI, or pressure gradient before and after intervention. In view of results from coronary revascularization studies correlating the ability of a larger postprocedure lumen to reduce the probability of restenosis (40), measurements of immediate postprocedure luminal diameter, acute gain in luminal diameter, and late loss of luminal diameter are highly recommended. This will require a method to account for radiographic magnification. The use of continuous data in addition to binary data is necessary for immediate, short-term, and long-term results.

Because of the importance of those risk factors affected by patient compliance (taking medications, continuing exercise, stopping smoking), evaluation of patient compliance should also be part of the post-treatment evaluation. For example, patients treated with surgery may be more motivated to quit smoking than patients treated with angioplasty. This might improve the long-term success of surgery and confuse a comparison of the inherent efficacy of the two procedures.

Anatomic Success 

The most widely used anatomic measure of success is improvement in the angiographic appearance. The definition of success recommended is a less than 30% final residual luminal stenosis measured at the narrowest point of the vascular lumen (modified from reference 41). Continued anatomic success is defined as the lack of a recurrent stenosis greater than 50% diameter (21, 40). In the presence of an angiographically visible dissection at the treatment site, the residual lumen is measured from the widest opacified lumen regardless of cracks, knowing that the true lumen is difficult to measure accurately in this situation. Although Capek et al (42) reports that a residual stenosis of 30% or greater predisposes to recurrent stenosis, the ability to measure this accurately on postangioplasty arteriograms is doubtful. Other means of measurement including intravascular ultrasound (US), duplex and color US, spiral computed tomography angiography, magnetic resonance angiography, and angioscopy are of uncertain utility. Residual lumen measured with these techniques may not correspond to the lumen measured with angiography (43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53). Although angiography may or may not be the most accurate measure of residual lumen, it is the most widely used and widely available. A follow-up arteriogram 6–12 months after intervention is strongly recommended. If nonangiographic modalities are used for follow-up, they must be compared to the same modality over time and not to other modalities. The same modality must be used for measurement in the treatment and control arms of comparison trials. Measurement of interobserver differences in assessing residual stenosis should be provided.

Hemodynamic Success 

Hemodynamic measures of success should include objective evidence of improved flow across the treated area. Palpation of pedal pulses is adequate for routine clinical assessment but, as noted by Rutherford (18), “accurate patency data are so crucial to comparisons of arterial reconstructive techniques that discriminating methods deserve to be used.” ABI (or thigh/brachial index to evaluate proximal revascularization in the presence of untreated distal disease) should be improved by 0.1 or greater above the baseline value and not deteriorated by more than 0.15 from the maximum early postprocedure level (Table 5) (23). The criteria of Ahn et al (21), which differ slightly from those of Rutherford/Becker in that they require an immediate improvement by 0.15 or more maintained over time, do not give a significantly different estimate of patency (54). For purposes of consistency, the Rutherford/Becker criteria are used.

ABI and thigh/brachial index are easily reproducible indirect measures that roughly correlate with clinical improvement. Direct intravascular pressures are considered the gold standard of hemodynamic assessment but are unavailable as a noninvasive follow-up method. For this reason, intravascular pressures should not be used as a substitute for ankle and thigh pressures but can provide useful adjunctive data. Peak systolic pressure measurements across the lesion should be obtained immediately after treatment. These measurements may be augmented by exercise, reactive hyperemia, or pharmacologically induced vasodilation. Ideally, the proximal and distal pressures should be obtained simultaneously and in such a manner that a catheter is not placed across the treated area. Often, however, sequential pullback pressures must be obtained by a catheter that crosses the treated area. Intravascular pressure criteria for defining a significant vascular stenosis are inconsistent (55, 56, 57, 58, 59). In the absence of consensus in the literature or scientific evidence supporting any one method of measuring intravascular pressures, we recommend that intravascular pressures be reported as continuous data rather than as hemodynamic success or failure. Peak systolic pressures should be used for these measurements. If the pressures are pharmacologically enhanced, this must be stated and all patients in the study must be treated similarly.

Clinical Success 

For a procedure to be clinically successful, relief or improvement of presenting symptoms must be demonstrated (ie, relief of pain, increased exercise tolerance, and so forth). For peripheral revascularizations, the Rutherford clinical criteria for acute or chronic limb ischemia adopted by the SVS/ISCVS must be improved by at least one category immediately after the intervention and be sustained during follow-up by one category above the pretreatment clinical level. Patients with tissue loss must improve by at least two categories and reach the level of claudication to be considered improved (18). The use of graded treadmill exercise protocols helps to quantify the degree of improvement in claudication (24). Formal tests of functional status are highly recommended as a means of more completely assessing effects on patient quality of life. For instance, a patient may be free of claudication because he has become an invalid. Functional status will measure how far the patient is walking and how successful he is performing activities of daily life (13, 36).

Functional status data are usually treated as a continuous variable (average change in score before and after intervention and between interventions, changes in score over time, and so forth). If the functional status data are condensed into a binary result (improved versus not improved), these data could be displayed in lifetable format. This format may be helpful in answering the question “What percent of treated patients are alive at 1 year with a functional limb?” These kinds of data can be obscured by a life table of success alone because patients withdrawn because of death are displayed in the same manner as patients withdrawn due to loss to follow-up. For example, if one treatment has a higher mortality rate from myocardial infarction but patients die with the vessel free of restenosis, a life table of functional status will display this information.

Technical Success 

To be considered technically successful in the immediate postprocedure period the intervention must meet the criteria for both anatomic and hemodynamic success as determined from the outcomes of therapy initially defined in the study design. A procedure may be technically successful and clinically unsuccessful (ie, if there is severe untreated disease elsewhere in the leg or if the patient's symptoms were not due to ischemia). It is recognized that some technologies may produce a better anatomic result (eg, atherectomy vs balloon angioplasty) without necessarily improving hemodynamics. Therefore, anatomic success may not be as important as hemodynamic success. If anatomic criteria are used, the criteria described above are required.

Improvement 

Improvement combines criteria for anatomic, hemodynamic, and clinical success. Ahn and Johnston (21, 60) called this “continued success” or “success” and defined it as sustained improvement of at least one objective hemodynamic or imaging test in addition to the clinical evaluation. The definition of improvement used by Rutherford (18, 23) includes clinical and hemodynamic measures only and is the definition used in this document (Table 6). This definition depends on perfusion of the entire limb and can lead to false conclusions. For example, in a patient treated with balloon angioplasty in the iliac artery, subsequent progressive disease in the superficial femoral artery can cause a deterioration in ABI and clinical symptoms that would falsely label the angioplasty site as failed. For this reason, in addition to the routine follow-up tests, when there has been hemodynamic deterioration, an imaging test is required to determine if this deterioration is due to recurrent or new disease. The imaging test used should be consistent between treatment groups.

Table 6. Definitions of Improvement
GradeDefinition
+3Markedly improved. Symptoms are gone or markedly improved. ABI increased to >0.90.
+2Moderately improved. Still symptomatic but with improvement in lesion category.* ABI increased by >0.10 but not normalized.
+1Minimally improved. Categorical improvement in symptoms without significant ABI increase (0.10 or less) or vice versa (but not both).
0No change. No categorical shift and less than 0.10 change in ABI.
−1Mildly worse. Either worsening of symptoms or decrease in ABI of >0.10.
−2Moderate worsening. Deterioration of the patient's condition by one category or unexpected minor amputation.
−3Marked worsening. Deterioration of the patient's condition by more than one category or major amputation.

Source.—Reference 23.

Categories refer to clinical categories in Table 2, Table 3.

Patency 

Patency was historically defined for use in evaluating bypass grafts rather than angioplasty sites (13). It is defined as continued flow through the treated segment, not the absence of recurrent stenosis (18). A treated segment may be considered patent when any one of the five criteria in Table 7 are met (18, 23). Vascular imaging or direct visualization should be used to determine patency whenever possible because of the potential inaccuracy of segmental pressures, plethysmographic traces, and pulse palpation, which may not distinguish stenosis from occlusion or disease in the treated segment from disease in adjacent segments. If imaging studies and direct visualization of the segment are not available, one should use the pressure index from the next level beyond the segment. Patency can be described as primary or secondary patency depending on whether a subsequent procedure has been performed in the treated segment. To allow a better comparison with surgical procedures, patency of the treated segment (aorto-iliac, femoro-popliteal, infrapopliteal) should be reported. Patency of the treated site may be reported separately.

Table 7. Definitions of Patency
Patency: Lack of occlusion of the treated segment determined by the presence of any one or more of the following criteria:
1.Demonstrably open with vascular imaging (arteriography, duplex Doppler, or magnetic resonance imaging).

2.Maintenance of achieved improvement in the appropriate segmental limb pressure index (ie, ABI or thigh/brachial index must have increased by more than 0.10 initially and not deteriorated by more than 0.15 from the maximum early postprocedure level.

3.Pulse volume recording distal to the treated segment maintained at

5 mm above preoperative tracing (only for diabetic patients with incompressible arteries).
4.Palpable pulse or biphasic or triphasic Doppler waveform measured at two points directly over a superficially placed graft.

5.Direct observation at operation or post mortem.

Primary patency: Uninterrupted patency with no procedures performed on or at the margins of the treated segment. Only procedures performed proximal or distal to the initially treated segment to treat progression of disease in an ad-jacent native vessel are exempted.
Assisted primary patency: Any procedure performed in the treated segment before thrombosis that might prevent eventual failure.
Secondary patency: Any procedure that restores patency after thrombosis.

Source.—18, 23.

Patency refers to continued flow and therefore differs from continued anatomic success or improvement. Because of the limited usefulness of this definition of patency, the use of continued anatomic success and gradation of improvement is recommended in evaluating the effectiveness of a procedure (13). Results of interventions should be reported as primary, assisted primary, and secondary continued anatomic success or improvement rather than patency. Patency may be reported but should be limited to occlusion of the treated site or segment.

Complications 

Complications need to be both listed (local infection, arterial wall injury, dissection, thromboemboli, fistulas, hematoma, acute occlusion, perforation, vasospasm, renal failure, stroke, myocardial infarct, and so forth) and graded so that comparisons can be made. Individual and classes of complications should be consistent with the SIR categories (Appendix 2). Grading of complications is necessary to quantify the overall morbidity produced by a procedure (18). Complications are graded according to the definitions of the SIR (Table 8). It is generally accepted that any complication within 24 hours after a procedure is considered procedure related (18). To avoid confusion as to whether a later complication is related to the procedure or disease, all complications and deaths within 30 days or within the same hospitalization are considered procedure related. This differs from previous recommendations that beyond 24 hours only complications specific to the procedure or the underlying disease should be reported (18). Complications and deaths should be recorded for the duration of the study, not just 30 days, because late effects on morbidity and mortality are possible (61, 62). For example, Ouriel et al found 62% lower mortality at 12 months in patients randomized to lytic therapy compared with surgery for treatment of acute limb ischemia, even though immediate mortality was comparable (61).

Table 8. Definitions of Complications
Minor Complications
A.No therapy, no consequence

B.Nominal therapy, no consequence, includes overnight admission for observation only

Major Complications
C.Require therapy, minor hospitalization (<48 hours)

D.Require major therapy, unplanned increase in level of care, prolonged hospitalization (>48 hours)

E.Permanent adverse sequelae

F.Death

Costs 

Analysis of expenses has traditionally been based on charges rather than costs to the hospital. Unfortunately, charges are arbitrary and true costs are frequently impossible to obtain (63). A rigorous analysis of costs should include the cost of complications, discounting of future costs to present values, and should specify the perspective of the cost analysis (ie, from the perspective of society, thirdparty payer, provider, or patient) because costs can vary depending on the party incurring them (6, 64, 65). Whatever method is used to calculate costs should be the same for all treatment groups. Costs have also traditionally been measured only until the time of discharge (66). Cumulative costs per patient to maintain improvement should be measured over several years. For example, if a procedure is initially inexpensive but has poor durability, costs over time increase from repeat procedures to treat restenosis. In the long run, a more expensive but more durable procedure may be cheaper. A calculation should also be made of the marginal cost/benefit of the new technique. For example, if intravascular stents have a 10% lower restenosis rate compared with balloon angioplasty after 5 years, what is the increased cost per additional improved extremity if all lesions are treated with stents?

For those studies analyzing cost-effectiveness, the concept of quality adjusted life years (QALY) is important. Quality of life is assessed by tests of functional status. Quality adjusted life years combines measures of quality and quantity of life to express health care benefits in terms of equivalents of well-years of life (67, 68, 69, 70). This has not often been evaluated in radiology articles (33) but will be important in justifying the benefits of revascularization beyond that of avoiding amputation (14, 15, 16).

Comparisons between Treatment Groups 

Comparisons could be made to the natural history of untreated disease, conservatively treated disease (exercise, diet, and so forth), placebotreated disease, or disease treated with another revascularization technique. Lack of comparison to the appropriate control group places the clinical trial more in the realm of a feasibility study, regardless of the number of patients treated. Comparing a revascularization technique to conservative treatment is not unreasonable for patients with claudication. Only 25% of claudicants will progress to severe ischemia in 4–7 years, with only 4%–8% requiring amputation. Three to 6 months of an intense walking program can quadruple normal walking distance and double the distance walked on a treadmill (71, 72). A randomized trial of exercise versus PTA as treatment for claudication found that PTA improved the ABI but did not improve maximum walking distance. An exercise program did not increase the ABI but there was a marked and progressive increase in maximum walking distance (73).

The studies could be performed with use of historical controls or randomized controlled trials. Most comparisons of revascularization techniques have been based on historical controls. For example, results of a study of balloon PTA are compared with another study of results from surgery, or results of a study of atherectomy are compared to a previous study of results from balloon PTA. Historical controls can be useful when the study endpoints are objectively measured, the disease is predictable and consistent, and the influence of baseline variables is minimal compared to the treatment effect. Unfortunately, this is rarely the case when studying the treatment of vascular disease. It is impossible to have all of the relevant information on inclusion and exclusion criteria; the available methods of diagnosis and treatment have usually changed; and the baseline characteristics of the historical group and the current study group are different (7). Comparisons need to be made between groups of patients with similar severity of disease, indications for treatment, and risk factors who are treated by physicians with similar abilities using similar equipment. Until recently, these data have not been provided in most articles. The more rigorous reporting requirements of Rutherford and Becker (18, 23) attempt to address this problem.

Registry data can sometimes be useful for historical controls. A registry of peripheral vascular interventions is underway (70), but it is not yet published and it is unknown if those data can be used as a reference against which new devices can be compared. In general, because registries lack a comparison group, they lack the scientific rigor of a randomized controlled trial that is necessary to compare the success and complication rates of different devices (41).

Randomized controlled trials are less likely to be biased and are considered statistically more trustworthy, but usually they require a greater number of patients and are therefore more expensive and time consuming. However, if one looks at the time and expense spent on therapies that are later demonstrated to be ineffective or no more effective than current therapies, the extra work and expense of a randomized trial initially is frequently a wiser choice for society (33, 75). In the evaluation of new revascularization devices, it is the opinion of the Technology Assessment Committee that randomized controlled trials are necessary for accurate assessment of the effectiveness and cost effectiveness of these devices. Comparison should be made to the appropriate control group (conservative therapy, surgery, or other endovascular technique).

Back to Article Outline

RECOMMENDATIONS AND REQUIREMENTS 

For a study to be used to make clinical decisions regarding the efficacy of a revascularization device compared with other devices, the rigorous requirements of a scientific study must be met. These include the above descriptions of patient selection, pretreatment evaluation, treatment description, post-treatment evaluation, complications, and statistically valid comparisons to the appropriate control group, which in most cases should mean a randomized controlled study. The Technology Assessment Committee of the SIR has therefore adopted standards for the evaluation of new peripheral arterial revascularization devices.

The following elements are required. The definitions of disease severity, success, and complications used in this document must be applied. Binary and continuous data with life-table analyses are required. Success data must be presented separately for anatomic, hemodynamic, and clinical criteria so that failures due to recurrent and new disease can be distinguished. Treadmill exercise testing is necessary in evaluating interventions for claudication.

The following elements are highly recommended. The use of graded rather than constant load treadmill exercise is highly recommended. Assessment of functional status is highly recommended and, for the sake of comparability, the use of the Walking Impairment Questionnaire (24) is recommended (Appendix 1). Assessments of quality of life are additionally highly recommended for trials comparing surgery and endovascular therapy. Measurement of acute gain and late loss of luminal diameter differs from current reporting practice for peripheral interventions but is highly recommended based on results from coronary interventions. Follow-up arteriography is highly recommended for evaluating anatomic success. Evaluation of patient compliance is urged. The inclusion of basic data on costs is highly recommended but is not a requirement of a scientific study. Given the importance of cost as well as effectiveness in the construction of clinical algorithms (76, 77), the acquisition of this information during an effectiveness study renders the study far more relevant.

The following elements are recommended. Identification of degree of eccentricity of treated lesions may establish subsets of lesions better treated with particular devices. Detailed analysis of costs is recommended.

Requirements and recommendations are summarized in Table 9.

Table 9. Reporting Requirements and Recommendations
DataRequiredHighly RecommendedRecommended
Pretreatment evaluation*
Risk factors/ComorbiditiesX
Measures of disease severity
Stenosis of treated siteX
Runoff gradeX
Eccentricity X
Noninvasive indices (ABI, TBI, PVR)X
Treadmill (claudicants)X
graded treadmill X
Functional status X
Quality of life X
Treatment descriptionX
Posttreatment evaluation*
Follow-up angiogram X
Technical success
Anatomic
StenosisX
Luminal gain X
Hemodynamic
Noninvasive (ABI, TBI, PVR)X
intravascular pressures X
Clinical success
Improvement categoryX
Functional status X
Quality of life X
Treadmill (claudicants)X
Graded treadmill (claudicants) X
Separate life-tables for anatomic, hemodynamic, and clinical dataX
ComplicationsX
Compliance X
Costs
Gross estimate (equipment, length of stay, ICU days, # encounters) X
Detailed X

Note.—TBI = thigh/brachial index, PVR = pulse volume recording, ICU = intensive care unit.

* Both binary and continuous data are required when appropriate.

Back to Article Outline

CONCLUSION 

In the excitement of new technology, interventional physicians have pursued multiple new methods of revascularization. Early reports of new technology have invariably been glowing, with more sober evaluations coming with maturity. The intention of a scientific clinical study is not to suppress innovation but to provide trustworthy information, unbiased by our excitement, with which to treat patients. It is the intention of this article to combine the recommendations of multiple sources into a single reporting standard for new peripheral arterial revascularization devices. Our recommendations are certainly not the only way to report data and, despite the best of intentions, they may not be the best way. But if studies can be reported in a uniform, consistent manner we will have the power to make accurate assessments of the effectiveness of our technology and give our patients the best possible care.

Back to Article Outline

APPENDIX. PERIPHERAL ARTERIAL DISEASE QUESTIONNAIRES 

Walking Impairment Questionnaire 

This questionnaire is designed to assess the degree of impairment experienced by the patient with claudication during daily activities. Question 1 is divided into two parts. Section A is specific for calf or buttocks claudication and is used to create a summary score for analysis. Section B is used to evaluate other symptoms that may limit walking ability. If the patient ranks a symptom from Section B as more severe than claudication pain (from Section A) then claudication is not the limiting symptom, and responses from questions 2–4 may not be valid for assessing claudication severity. A summary score is not created for Section B.

Questions 1A, 2, 3, and 4 are expressed on a scale of 0% (unable to perform because of severe claudication) to 100% (no impairment). For questions 2–4, each individual response is multiplied by its respective weight to create an individual score. All individual scores for a question are added and then divided by the maximal possible score to create the % score which is used for analysis.

Back to Article Outline

References 

  1. Doubilet P , Abrams HL . The cost of underutilization: percutaneous transluminal angioplasty for peripheral vascular disease . N Engl J Med . 1984;310:95–102
  2. Porter JM . Endovascular arterial intervention: expression of concern . J Vasc Surg . 1995;21:995–997
  3. Strandness DE , Barnes RW , Katzen B , Ring EJ . Indiscriminate use of laser angioplasty . Radiology . 1989;172:945–946
  4. Kotb MM , Kadir S , Bennett JD , Beam CA . Aortoiliac angioplasty: is there a need for other types of percutaneous intervention? . J Vasc Interv Radiol . 1992;3:67–71
  5. Popma JJ , Satler LF , Pichard AD , et al.   Vascular complications after balloon and new device angioplasty . Circulation . 1993;88(part 1):1569–1578
  6. Yin D , Forman H . Health care costbenefit and cost-effectiveness analysis: an overview . J Vasc Interv Radiol . 1995;6:311–320
  7. Interventional Cardiology Devices Branch, Division of Cardiovascular, Respiratory and Neurology Devices, Office of Device Evaluation of the Food and Drug Administration. Guidance for the submission of research and marketing applications for interventional cardiology devices. May 1993.
  8. Oxman AD , Sackett DL , Guyatt GH . Users' guide to the medical literature. I. How to get started . JAMA . 1993;270:2093–2095
  9. Guyatt GH , Sackett DL , Cook DJ . User's guides to the medical literature II: How to use an article about therapy or prevention. A. Are the results of the study valid? . JAMA . 1993;270:2598–2602
  10. Guyatt GH , Sackett DL , Cook DJ . Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? . JAMA . 1994;271:59–63
  11. Woolf SH . Practice guidelines, a new reality in medicine. II. Methods of developing guidelines . Arch Intern Med . 1992;152:946–952
  12. Cook DJ , Guyatt GH , Laupacis A , Sackett DL . Rules of evidence and clinical recommendations on the use of antithrombotic agents . Chest . 1992;102(suppl):305S–311S
  13. Rutherford RB . Presidential address: vascular surgery— comparing outcomes . J Vasc Surg . 1996;23:5–17
  14. Tunis SR , Bass EB , Steinberg EP . The use of angioplasty, bypass surgery, and amputation in the management of peripheral vascular disease . N Engl J Med . 1991;325:556–562
  15. Becker GJ , Ferguson JG , Bakal CW , et al.   Angioplasty, bypass surgery, and amputation for lower extremity peripheral arterial disease in Maryland: a closer look . Radiology . 1993;186:635–663
  16. Steinberg EP , Bass EB , Tunis SR . Interventional management of peripheral vascular disease: what did we learn in Maryland and where do we go from here? . Radiology . 1993;186:639–642
  17. Bakal CW , Sprayregen S , Scheinbaum K , Cynamon J , Veith FJ . Percutaneous transluminal angioplasty of the infrapopliteal arteries: results in 53 patients . AJR . 1990;154:171–174
  18. Rutherford RB , Flanigan DP , Gupta SK , et al.   Suggested standards for reports dealing with lower extremity ischemia . J Vasc Surg . 1986;4:80–94
  19. Peterkin GA , Manabe S , LaMorte WW , Menzoian JO . Evaluation of a proposed standard reporting system for preoperative angiograms in infrainguinal bypass procedures: angiographic correlates of measured runoff resistance . J Vasc Surg . 1988;7:379–385
  20. Standards of Practice Committee of the Society of Cardiovascular & Interventional Radiology . Guidelines for percutaneous transluminal angioplasty . Radiology . 1990;177:619–626
  21. Ahn SS , Rutherford RB , Becker GJ , et al.   Reporting standards for lower extremity arterial endovascular procedures . J Vasc Surg . 1993;17:1103–1107
  22. Rutherford RB, Johnston KW, Ernst CB, Baker JD, Porter JM. Suggested standards for reports dealing with lower extremity ischemia: revised version. J Vasc Surg (pending).
  23. Rutherford RB , Becker GJ . Standards for evaluating and reporting the results of surgical and percutaneous therapy for peripheral arterial disease . J Vasc Interv Radiol . 1991;2:169–174
  24. Hiatt WR , Hirsch AT , Regensteiner JG , Brass EP  and the Vascular Clinical Trialists . Clinical trials for claudication: assessment of exercise performance, functional status, and clinical endpoints . Circulation . 1995;92:614–621
  25. Hiatt WR , Regensteiner JG , Hargarten ME , Wolfel EE , Brass EP . Benefit of exercise conditioning for patients with peripheral arterial disease . Circulation . 1990;81:602–609
  26. Hiatt WR , Nawaz D , Regensteiner JF , Hossack DF . The evaluation of exercise performance in patients with peripheral vascular disease . J Cardiopulmonary Rehabil . 1988;12:525–532
  27. Regensteiner JG , Hiatt WR . Medical management of peripheral arterial disease . J Vasc Interv Radiol . 1994;5:669–677
  28. McDaniel MD, MacDonald PD, Mangione TW, et al. Health-related quality of life in intermittent claudication: final report of AHCPR Grant RO3 HS06675-01, 1992.
  29. Currie IC , Wilson YG , Baird RN , Lamont PM . Treatment of intermittent claudication: the impact on quality of life . Eur J Vasc Endovasc Surg . 1995;10:356–361
  30. Feinglass J , McCarthy WJ , Slavensly R , Manheim LM , Martin GJ  and the Chicago Claudication Outcomes Research Group . Effect of lower extremity blood pressure on physical functioning in patients who have intermittent claudication . J Vasc Surg . 1996;24:503–512
  31. Lohr KN . Conceptual background and issues in quality of life . In:  Mosteller F ,  Falotico-Taylor J editor. Quality of life and technology assessment: monograph of the council on health care technology. Institute of Medicine . Washington, DC: National Academy Press; 1989;p. 1–7
  32. Gibbons GW , Burgess AM , Guadagnoli E , et al.   Return to well-being and function after infrainguinal revascularization . J Vasc Surg . 1995;21:35–45
  33. Wolf GL , Wilson SE , Cross AP , Deupree RH , Stason WB  and the principle investigators and their associates of the Veterans Administration Cooperative Study Number 199 . Surgery or balloon angioplasty for peripheral vascular disease: a randomized clinical trial . J Vasc Interv Radiol . 1993;4:639–648
  34. Bergner M , Bobbitt RA , Carter WB , Gibson BS . The sickness impact profile: development and final revision of a health status measure . Med Care . 1981;19:787–805
  35. Ware JE , Sherbourne CD . The MOS 36-item short-form health survey (SF- 36). I. Conceptual framework and item selection . Med Care . 1992;30:473–483
  36. Regensteiner JG , Steiner JF , Panzer RJ , Hiatt WR . Evaluation of walking impairment by questionnaire in patients with peripheral arterial disease . J Vasc Med Biol . 1990;2:142–152
  37. Goldman L , Nussbaum SR , Southwick FS , et al.   Multifactorial index of risk in noncardiac surgical patients . N Engl J Med . 1977;297:846–850
  38. Cobb LA , Thomas GI , Dillard DH , Merendino KA , Bruce RA . An evaluation of internal-mammary-artery ligation by a double-blind technique . N Engl J Med . 1959;260:1115–1118
  39. Underwood CJ , Faragher EB , Charlesworth D . The uses and abuses of lifetable methods in vascular surgery . Br J Surg . 1984;71:495–498
  40. Kuntz RE , Safian RD , Carrozza JP , Fishman RF , Mansour M , Daim DS . The importance of acute luminal diameter in determining restenosis after coronary atherectomy or stenting . Circulation . 1992;86:1827–1835
  41. Baim DS , Dent KM , King SB , et al.   Evaluating new devices: acute (In-hospital) results from the new approaches to coronary intervention registry . Circulation . 1994;89:471–481
  42. Capek P , McLean GK , Berkowitz HD . Femoropopliteal angioplasty: factors influencing long-term success . Circulation . 1991;83(suppl I):I70–I80
  43. Mewissen MW , Kinney EV , Bandyk DF , et al.   The role of duplex scanning versus angiography in predicting outcome after balloon angioplasty in the femoropopliteal artery . J Vasc Surg . 1992;15:860–866
  44. Sacks D , Robinson ML , Marinelli DL , Perlmutter GS . Evaluation of the peripheral arteries with duplex US after angioplasty . Radiology . 1990;176:39–44
  45. Spijkerboer AM , Nass PC , de Valois JC , et al.   Iliac artery stenoses after percutaneous transluminal angioplasty: follow-up with duplex ultrasonography . J Vasc Surg . 1996;23:691–697
  46. Isner JM , Rosenfield K , Losordo DW , et al.   Percutaneous intravascular US as adjunct to catheter-based interventions: preliminary experience in patients with peripheral vascular disease . Radiology . 1990;175:61–70
  47. Sheikh KH , Davidson CJ , Kisslo KB , et al.   Comparison of intravascular ultrasound, external ultrasound and digital angiography for evaluation of peripheral artery dimensions and morphology . Am J Cardiol . 1991;67:817–822
  48. Miller BV , Sharp WJ , Shamma AR , Kresowik TF , Petrone S , Corson JD . Surveillance for recurrent stenosis after endovascular procedures . Arch Surg . 1991;126:867; 862
  49. Vroegindeweij D , Tielbeek AV , Buth J , van Kints MJ , Landman GHM , Mali WPTM . Recanalization of femoropopliteal occlusive lesions: a comparison of long-term clinical, color duplex US, and arteriographic follow-up . J Vasc Interv Radiol . 1995;6:331–337
  50. Snidow JJ , Harris VJ , Trerotola SO , et al.   Interpretations and treatment decisions based on MR angiography versus conventional arteriography in symptomatic lower extremity ischemia . J Vasc Interv Radiol . 1995;6:595–603
  51. Mistretta CA . Relative characteristics of MR angiography and competing vascular imaging modalities . JMRI . 1993;3:685–698
  52. Rieker O , Duber C , Schmiedt W , von Zitzewitz H , Schweden F , Thelen M . Prospective comparison of CT angiography with intraarterial digital subtraction angiography . AJR . 1996;166:269–276
  53. Tielbeck AV , Vroegindeweij D , Buth J , Schol FPG , Mali WPTM . Comparison of intravascular ultrasonography and intraarterial digital subtraction angiography after directional atherectomy of short lesions in femoropopliteal arteries . J Vasc Surg . 1996;23:436–445
  54. Matsi PJ , Manninen HI . Impact of different patency criteria on long-term results of femoropopliteal angioplasty: analysis of 106 consecutive patients with claudication . J Vasc Interv Radiol . 1995;6:159–163
  55. Palmaz JC , Garcia OJ , Schatz RA , et al.   Placement of balloon-expandable intraluminal stents in iliac arteries: first 171 procedures . Radiology . 1990;174:969–975
  56. Udoff EJ , Barth KH , Harrington DP , Kaufman SL , White RI . Hemodynamic significance of iliac artery stenosis: pressure measurements during angiography . Radiology . 1979;132:289–293
  57. Kaufman SL , Barth KH , Kadir S , et al.   Hemodynamic measurements in the evaluation and follow-up of transluminal angioplasty of the iliac and femoral arteries . Radiology . 1982;142:329–336
  58. Flanigan DP , Williams LR , Schwartz JA , Schuler JJ , Gray B . Hemodynamic evaluation of the aortoiliac system based on pharmacologic vasodilatation . Surgery . 1983;93:709–714
  59. Bonn J . Percutaneous vascular intervention: value of hemodynamic measurements . Radiology . 1996;201:18–20
  60. Johnston KW , Rae M , Hogg-Johnston SA , et al.   5-year results of a prospective study of percutaneous transluminal angioplasty . Ann Surg . 1987;206:403–413
  61. Ouriel K , Shortell CK , DeWeese JA , et al.   A comparison of thrombolytic therapy with operative revascularization in the initial treatment of acute peripheral arterial ischemia . J Vasc Surg . 1994;19:1021–1030
  62. Bergan JJ , Wilson SE , Wolf G , Deupree RH . Unexpected, late cardiovascular effects of surgery for peripheral artery disease . Arch Surg . 1992;127:1119–1124
  63. Finkler SA . The distinction between cost and charges . Ann Intern Med . 1982;96:102–109
  64. Eisenberg JM . Clinical economics: a guide to the economics of clinical practices . JAMA . 1989;262:2878–2886
  65. Powe NR . Economic and cost-effectiveness investigations of radiologic practices . Radiology . 1994;192:11–18
  66. Hunink MGM , Cullen KA , Donaldson MC . Hospital costs of revascularization procedures for femoropopliteal arterial disease . J Vasc Surg . 1994;19:632–641
  67. Yin D , Forman HP , Langlotz CP . Evaluating health services: the importance of patients' preferences and quality of life . AJR . 1995;165:1323–1358
  68. Russell LB , Gold MR , Siegel JE , Daniels N , Weinstein MC  for the Panel on Cost Effectiveness in Health and Medicine . The role of cost effectiveness analysis in health and medicine . JAMA . 1996;276:1172–1177
  69. Weinstein MC , Siegel JE , Gold MR , Karniet MS , Russell LB  for the Panel on Cost Effectiveness in Health and Medicine . Recommendations of the panel on cost-effectiveness in health and medicine . JAMA . 1996;276:1253–1258
  70. Seigel JA , Weinstein MC , Russell LB , Gold MR  for the Panel on Cost Effectiveness in Health and Medicine . Recommendations for reporting cost effectiveness analyses . JAMA . 1996;276:1339–1341
  71. Hertzer NR . The natural history of peripheral vascular disease: implications for its management . Circulation . 1991;83(suppl I):I12–I19
  72. Regensteiner JG , Steiner JF , Hiatt WR . Exercise training improves functional status in patients with peripheral arterial disease . J Vasc Surg . 1996;23:104–115
  73. Creasy TS , McMillan PJ , Fletcher EWL , Collin J , Morris PJ . Is percutaneous transluminal angioplasty better than exercise for claudication? Preliminary results from a prospective randomized trial . Eur J Vasc Surg . 1990;4:135–140
  74. Tielbeek AV , Vroegindeweij D , Buth J , Landman GHM . Comparison of balloon angioplasty and Simpson atherectomy for lesions in the femoropopliteal artery: angiographic and clinical results of a prospective randomized trial . J Vasc Interv Radiol . 1996;7:837–844
  75. Guyatt G , Drummond M , Feeny D , et al.   Guidelines for the clinical and economic evaluation of health care technologies . Soc Sci Med . 1986;22:393–408
  76. Detsky AS , Naglie IG . A clinician's guide to cost-effectiveness analysis . Ann Intern Med . 1990;113:147–154

Back to Article Outline

Uncited reference 

  1. Gardiner GA , Levin DC , Meranze SG , et al.   Initial experiences of a multicenter transluminal revascularization registry (abstr) . Radiology . 1990;177(P):103

 This article first appeared in J Vasc Interv Radiol 1997; 8:137-149, with an erratum in J Vasc Interv Radiol 8:492-493

PII: S1051-0443(07)61257-3

doi:10.1097/01.RVI.0000094613.61428.a9

Journal of Vascular and Interventional Radiology
Volume 14, Issue 9, Part 2 , Pages S395-S404, September 2003

Article Tools