| | Role of Virtual Reality Simulation in Teaching and Assessing Technical Skills in Endovascular InterventionReceived 13 May 2009; received in revised form 13 September 2009; accepted 16 September 2009. Training in endovascular intervention ultimately aims to produce interventionalists who demonstrate competence in technical skills. Herein, the authors investigate the rationale for simulation-based training by providing an overview of the psychological theories underpinning acquisition of technical skills, training and assessment history, recent advances in simulation technology, and a critical appraisal of their role in training and assessment in endovascular intervention. Simulators have potential for training and assessment and promise solution to many shortcomings of traditional ‘apprenticeship’ training models. Before inclusion into the curriculum, further work is needed regarding fidelity, validity, reliability, and design of simulators to ensure accurate transfer of acquired endovascular skills from simulator to patient. ENDOVASCULAR intervention, in the wake of continuously developing technology and interdisciplinary collaboration, has entered into an era where technical skills training and competence assessment are equally important. Training ultimately aims to produce interventionalists who demonstrate competence in both technical and nontechnical skills (decision making, communication, and teamwork). At present, training is carried out mainly on patients, sometimes at the expense of patient safety (1), which raises several legal and ethical concerns. In addition, there is an increased cost and intricacy of endovascular procedures, a decrease in simpler diagnostic angiography due to replacement by noninvasive imaging, and residents with capped working hours accompanied by an increasing population of patients with serious and complex problems needing expert input from the beginning (1, 2). Moreover, subjective assessment by mentors in the current system, although valuable, remains unreliable and does not fulfill the criteria for the process of summative assessment (3). Therefore the existing master-apprentice model is unlikely to be acceptable. Continuing advances in virtual reality (VR) simulator technology seem to be promising regarding their utility in medical training and assessment. The challenge is, however, to judiciously employ the contemporary advances in technology in the context of a structured and efficient training system with objective competence evaluation underpinned with evidence-based components and a sound theoretical basis. The implementation of training programs that employ simulators at a certain level must be justified with a corresponding level of evidence. Herein, we present an overview of the history of training and assessment, the psychological theories behind the acquisition of technical skills, and recent advances in simulation technology for training and assessment. In addition, we perform a critical appraisal of the tools in terms of feasibility, validity, and utility of simulation in endovascular intervention. Psychological Basis of Skill Acquisition  Technical proficiency has been considered to be one of the most important skills of an interventionalist (4). Various theories of cognitive motor learning to enable skill acquisition have been described in the literature, including Kopta's theory, Schmidt's schema theory, the cognitive apprenticeship model, and Ericsson's rationale for the acquisition of expertise (5). Kopta's theory highlights the importance of observation followed by practice. It involves three phases of acquisition of motor skills. The cognitive phase is observing new procedures. This is followed by the integrative phase, which involves feedback during practice. This information is put together toward appropriate motor responses, resulting in less erratic movements. Finally, there is the autonomous phase, where continued practice results in efficient performance of the task, without cognitive input resulting in automatic movement (6). Schmidt's theory looks more into how a motor skill develops. As trainees practice a maneuver, they learn the relationship between the movement parameters of the maneuver and the outcome (7). They then improve their understanding of the relationship between a maneuver outcome and their control of the movement's parameters. An important prediction of the theory is that people will learn more quickly the relationship between manipulating parameters and achieving a desired outcome if they practice a task in a wide variety of situations and experience errors in the process. Practice that lacks variety, but is instead precise or repetitious, will not (from Schmidt's perspective) provide enough information for a learner to fathom the rules that underlie the generalized motor program. This is very important when considering VR training. The cognitive apprenticeship model decontextualizes the knowledge so that a trainee can apply new skills in more than one setting. The cognitive apprenticeship model has three components performed by the mentor (modeling, coaching, and scaffolding) and three components performed by the student (articulating, reflecting, and exploring). This model of cognitive motor learning emphasizes the early stages of learning (8). Ericsson's model of skill acquisition of expertise emphasizes the importance of focused attention and practice in acquiring expert skill. Ericsson found that time of day was an important factor in success, with the morning being the best time to practice because this was when the ability to perform complex cognitive activities was the highest (9). The core concepts of these models of cognitive learning must all be considered when designing and implementing an effective simulator training curriculum. Trainees require focused attention, which translates into protected uninterrupted time to observe, practice, obtain trainer feedback, experience procedure variety, and reflect on the acquisition of a new endovascular skill. History of Training and Assessment of Technical Skills in Craft Specialities When considering the history of methods of medical training and assessment of technical skills in craft specialties such as endovascular intervention, authors begin with Sushutra from ancient India and Halstead from the early 19th century (10, 11), who described the concept of clinical training based on the practice of inanimate objects—the so-called “see one, do one, teach one approach.” Today, Sacks (12) eloquently describes the vision for simulators as “practice a whole bunch on a simulator and get good, then see a few, do a few, teach a few.” The modern history of medical simulation began in the 1950s with the pioneering work in cardiopulmonary resuscitation (13), followed by the introduction of computer-controlled simulation by Denson and Abrahamson (14). Simulators have been developed recently that are capable of training with catheter and guide wire techniques with real-time simulated fluoroscopic imaging (15, 16, 17). Evaluation of technical skills was an essential component of the examination for the fellowship to the Royal College of Surgeons of England (18), but cadaver availability led to its discontinuation in the 1940s (19). Aptitude testing dates back to 1949, when all American dental schools implemented the Chalk Carving Test to assess the psychomotor ability of the candidates. This test was so simple and basic that a student who performed very poorly on the test did not usually improve with repeated testing. However, the test proved to be expensive to administer and was not as discriminatory as compared to the Paper-Pencil Examination, which replaced it in 1973 (20). Other mechanical aptitude and space relations tests correlated significantly with technical skills of dental students (21, 22). In 1871, Kopta (6) outlined the scheme for technical skills development featuring three major steps that included perception, integration, and automatization. Lippert et al (23) described acquisition of skills as to be multidimensional and explained that the process of achieving automatization requires theoretical knowledge and appropriate skills in a specific domain as well as psychomotor distinction. One of the first skills workshops for practitioners was held at the Royal College of Surgeons of England in 1977 (24). Since then, the intercollegiate basic surgical skills course has become a mandatory part of surgical training in the United Kingdom. Within the United Kingdom, the need for personality assessment techniques and aptitude testing in the selection of surgical trainees was raised during an Anglo-Dutch symposium held in 1987 (25), although no practical developments have ensued since. Recent interest in the selection of surgical trainees has been directed toward the use of aptitude tests (26, 27, 28). In the United States, the need for psychomotor skills training in orthopedic surgery was recognized in the mid-1960s, and in Canada courses in surgical techniques were held as early as 1962 (23, 29). In 1975, these were followed up with the first 18-hour course on motor skills for residents held at the University of Washington's School of Medicine (23). The structured approach to assess clinical examinations (Objective Structured Clinical Examination) was first introduced by Harden et al (30) in 1979 and followed the Objective Structured Practical Examination as described by the same authors (30). The main features include (a) separate assessment of process and product through observation of performance and assessment of end result, (b) adequate sampling of skills and content to be tested, (c) an analytical approach to the assessment, (d) objectivity, and (e) feedback to the teacher and student (31). In the 1990s, an increase in the need for assessing the technical skills of doctors in craft specialties led researchers in Canada to develop a new tool named Objective Structured Assessment of Technical Skill. Objective Structured Assessment of Technical Skill, a derivative of the Objective Structured Clinical Examination introduced in 1997, is a performance-based assessment exercise designed to assess the technical skills of surgical trainees (32). Trainees rotate through different stations within a predetermined time. Six-station examination of technical skills includes excision of skin lesion, hand-sewn bowel anastomosis, stapled bowel anastomosis, insertion of a T-tube, abdominal wall closure, and control of inferior vena caval hemorrhage. The assessment of technical proficiency is based on a global rating scale and a checklist specific to the operation or task, thus making this process more objective, reliable, and valid. The other assessment method introduced during the 1990s was the modified version of Human Reliability Assessment techniques (33). Human Reliability Assessment was adapted for medical practice to categorize and record errors during laparoscopic surgery. Subsequently, the applicability, feasibility, and validity of this observational methodology in the assessment of technical skills in otolaryngology and ophthalmology were confirmed by other groups (34, 35, 36). These tools have not been validated for workplace for assessment of interventionalists. Recently, a number of endovascular simulators have been developed that allow practice with the manipulation of catheters and guide wires, contrast media injection, and real-time fluoroscopy (37). Simulators also can provide feedback as to operator performance regarding correct order of procedure steps, procedure time, fluoroscopy time, and specific procedure outcome measures. The field of interventional radiologic procedural simulation is still in the early stages of development. There is a dearth of evidence to support recognition for simulation of endovascular interventions, although there is a role for training any steps where content is shown to be valid. There are more limited roles where a simulation may be suitable for training tasks such as procedure sequencing and use of devices (38). Also, in combination with a dedicated clinical and cognitive curriculum, simulators may have a role in the training, continuous assessment, and re-certification of interventional radiologists (39). Role of Simulation in Skills Training Simulators are instruments that reproduce, under artificial conditions, components of clinical tasks that are likely to occur under normal circumstances (40). Simulation, originally developed in aviation, allows training and practice of complex procedures to a proved proficiency before performance at the workplace. With further advances in simulation technology, simulation is likely to shift training away from patients to a low-risk environment (41). Currently, there is an increasing trend toward adoption of VR endovascular interventional simulations for training. The types of simulation models include cadaver, animal, bench, and computer software–based simulators. We have looked into the utility, feasibility, and limitations of each modality of simulation for endovascular intervention (Table 1). | | |  | Simulation | Advantages | Disadvantages | |
|---|
| Synthetic models | Cheap Reusable portable Minimal risk | Low face validity Basic generic tasks Low fidelity | | | Animal models | High fidelity Full procedure simulation Assessment potential | Cost Ethical issues Special facilities required Anatomic difference | | | Human cadaver (48) | High fidelity High face validity Full procedure simulation Assessment potential | | | | VR simulation | Reusable Data capture Minimal setup time Assessment potential High degree of realism | Cost Maintenance Acceptability Non-availability of three-dimensional models | | | | |
Synthetic Models Synthetic models are relatively simple and cost effective. These models can replicate the beating heart and dynamic behavior of human arterial circulation to some extent (42, 43). The only problem that limits advanced simulation models is the effect of friction during passage of a real device through the simulated blood vessels (44). Guide wire insertion and balloon inflation can be taught by using the real devices and instruments on synthetic models (45). The feasibility and validity of these training models in the context of endovascular training remains unproved. Animal Models Animal models have a relatively higher degree of face validity, with an added advantage being that both open and minimally invasive procedures can be performed (46, 47). The use of animal models is limited by cost, ethics, animal licenses, and lack of pathology. However, many atherosclerotic models can now be created. Moreover, animals require a great amount of logistics such as a dedicated animal facility, trained personnel (including an anesthetist), and an operating room. Human Cadavers The human cadaveric circulation model offers the most lifelike conditions possible for training endovascular skills consisting of many elements of the normal and abnormal human circulation. The benefits include a better-quality model for research and development of endovascular technology and a widely applicable training tool for endovascular techniques (48). Catheter manipulation and balloon angioplasty can be performed repeatedly; however, once stents or stent-grafts are deployed within the arteries, they cannot easily be retrieved, thus limiting that arterial segment for further device implantation. Cadavers can be an effective way of teaching technical procedural skills because the procedure can be performed repeatedly to some degree, but the effectiveness in endovascular intervention is not established. Moreover, limited availability and high cost related to transfer and preservation limits the use of human cadavers for endovascular training (49). Virtual Reality VR is a communication interface based on interactive three-dimensional visualization that allows the user to interact with and integrate different sensory inputs that simulate important aspects of real-world experience (50). In the current settings, computer-based simulation seems to be the most practical option for endovascular training (51, 52). Despite reduced tactile feedback (haptics), lack of case-by-case variety, and lack of exact replication of fine motor skills with VR, an early endovascular training experience is possible with the advantage of objective assessment of performance. With the added benefit of both audio and video recordings of the trainee's VR performance, expert reviewers can assess each step of the simulation at a remote time and location (53, 54). In addition, VR provides a potential opportunity to enable continuous assessment of trained endovascular specialists and a method to examine individual suitability for specialist recertification. Available endovascular simulators include Procedicus VIST (Mentice, Gothenburg, Sweden), Angiomentor (Simbionix, Cleveland, Ohio), Simsuite (Medical Simulation, Denver, Colorado), and Endovascular Accutouch (Immersion Medical, Gaithersburg, Maryland). These simulators include guide wire and catheter insertion; angioplasty and stent placement modules of carotid, renal, iliac, and coronary vessels; neuro-interventions; and closure of patent foramen ovale. VR simulation has been shown to be an effective way of teaching minimally invasive procedural skills in certain craft specialties (55). Improvement of endovascular technical skills of trainees following simulator-based training has been demonstrated by various authors (15, 16, 17, 56). Transferability of Skills From VR Models to Real Patients Transferability of endovascular skills acquired from VR to animals and human patients has been demonstrated by various authors. Transfer of skills to a pig model from VIST has been shown by Berry et al (51) using the observational parameters (global assessment by an assessor) of assessment. Chaer et al (37) performed a randomized trial examining transfer of VR endovascular training (iliofemoral angioplasty) to the human model. In that study, training (VR and didactic) compared with the control group (didactic only) significantly improved performance that was rated with use of a procedure-specific checklist and a global rating scale (37). Simulator-derived performance reporting can also be used to track a trainee's learning curve in endovascular interventions (57). Furthermore, VR can potentially be useful for the development of a proficiency-based training curriculum, with progress determined by the demonstration of skills performance to a predetermined benchmark level (52). Attainment of this benchmark could possibly then serve as a prerequisite to supervised procedure performance in the real patient setting, with failing to acquire this benchmark necessitating further VR training (58). A few limitations of these studies include validation through a straightforward iliac task along with an unvalidated, observer-based assessment tool that failed to measure dexterity—which is crucial to the safe performance of some key procedural steps in patients. Moreover, the simulator-trained group received additional cognitive training that was not delivered to the apprenticeship-trained group (37). Further research is needed to establish the transferability of skills acquired from the VR models to real human patients. Assessment of Technical Skills Assessment can be defined as the process of documenting, usually in measurable terms, knowledge, skills, attitudes, and beliefs (59). In 1990, psychologist George Miller proposed a pyramidal framework for the assessment of clinical competence. At the bottom of the pyramid is knowledge (“knows”), followed by competence (“knows how”), performance (“shows how”), and action (“does”). “Action” in this pyramid focuses on what happens in real life (60). The target of assessment in healthcare is this top level of the Miller pyramid to obtain evidence about doctor's performance (Fig 1). The assessment tools can broadly be classified into observational and nonobservational tools. For assessment tools to be effective and acceptable, they should be feasible, reliable, and valid (Table 2). | Feasibility: | | |  Measure of whether something is capable of being done or carried out. | | | Validity: | | | Face validity is the extent to which the examination resembles the situation in the real world. | | | Content validity is the extent to which the intended content domain is being measured by the assessment exercise—for example, while trying to assess technical skills we may actually be testing knowledge. | | | Construct validity is the extent to which a test measures the trait that it purports to measure. One inference of construct validity is the extent to which a test discriminates between various levels of expertise. | | | Concurrent validity is the extent to which the results of the test correlate with the gold standard tests known to measure the same domain. | | | Predictive validity is the extent to which this assessment will predict future performance. | | | Reliability: | | | Reliability is a measure of a test to generate similar results when applied at two different points (test-retest). When assessments are performed by more than one observer, another type of reliability test is applicable that is referred to as inter-rater reliability, which measures the extent of agreement between two or more observers. | | | Fidelity: | | | Fidelity refers to the degree to which a simulation reproduces the state and behavior of a real world object, feature, or condition, in this case an endovascular intervention. Fidelity is therefore a measure of the realism of a model or simulation. Simulation fidelity has also been described in the past as the degree of similarity. | | | | |
Observational Tools for Assessment Recently, a number of authors have successfully applied observational tools (rating scales) to assess the performance on VR simulators. Observational rating scales have been used in parallel to objective measures of VR simulators (47, 61). Global assessment of technical skills is based on either procedure-specific or generic rating scales. Generic global rating scales have established validity and reliability in surgery and allied specialties. The components of generic rating scales include respect for tissue, time and motion, instrument handling, knowledge of instruments, use of assistants, flow of operation and forward planning, and knowledge of the specific procedure steps (32). In addition to global scales, procedure-specific checklists based on accurate performance of procedural steps and scoring based on error execution have also been used by various authors (62). Global and task-specific checklists have been validated on models such as vascular anastomosis (arterial anastomosis, arteriovenous anastomosis, insertion of vein patch and graft to arterial anastomosis), sapheno-femoral disconnection, femoral triangle dissection, carotid endarterectomy, and creation of a forearm arteriovenous bridge graft (62, 63, 64, 65, 66). An observational scale based on global rating has been shown to demonstrate a significant level of construct validity with transfer of endovascular skills from VR to an animal model when scored by independent assessors (47). Nonobservational Tools Nonobservational tools—VR simulators and motion analysis devices—are based on an automated assessment process using computer-based measurements of specific procedural events. Performance scoring is not carried out by individual assessors. The VR simulator provides objective evidence of performance at the end of each task, whereas motion analysis performance is determined by graphical representation of movements and time. VR simulators instantly provide a nonobservational objective report of the performance. Scoring is based mainly on errors enacted, economy of movements, and time taken to complete a task. Observational rating scales have also been validated in combination with nonobservational VR scoring (67, 68). VR simulators have been shown to differentiate between level of experience (17, 69, 70). A recent study by Kundhal and Grantcharov (71) demonstrated significant correlations between performance in the operating room (assessed with a well-validated rating scale) and that in a virtual environment (assessed with a computer simulator), thus validating the simulator system as an objective tool for assessing minimally invasive psychomotor skills. The VIST simulator has been used widely by various researchers. VIST VR simulator produces an objective metrics at the end of a procedure that includes static (procedure time, fluoroscopy time, contrast medium volume, cine-loops) and dynamic simulator metrics (stent or balloon/vessel ratio, coverage percentage, placement accuracy, residual stenosis, and lesion coverage). Efficiency and economy of movements are known to be the key discriminators of technical skill in craft specialties (72). The Imperial College Surgical Assessment Device tracks hand movement in three dimensions by using electromagnetic sensors, with a composite score based on economy of movement and qualitative analysis (73). The Imperial College Surgical Assessment Device has been used in conjunction with checklists to assess technical skills of vascular and cardiac surgical trainees, both in laboratory and real settings (72, 74). As yet, there is no motion analysis–based assessment study available for endovascular skill evaluation. Discussion Training effectiveness of simulators requires validity of the content for all possible procedural steps and that these are replicated with suitable fidelity (realism) and in an environment with appropriate face validity. For simulation-based assessment, steps that are considered crucial for the safe completion of the target procedure must be present (a part of face and content validation) and must also be correctly assessed with a weighting as to their critical nature. To date, no ideal tool for training and assessment is available (75). Herein, we presented an overview of the advances in knowledge within the constantly evolving field of training and assessment of endovascular technical skills (Table 3). Furthermore, we have pointed out the deficiencies in the literature along with a brief note about fundamental inconsistencies between the results of the relevant studies. | | | | Study | Level Participants | Specialty | | Assessment Method | Results/Conclusion | |
|---|
| Procedicus VIST | | | | | | | | Berry et al, 2008 (76) | 16 interventional radiology experts and 16 medical students | Interventional radiologists and medical students | CAS | SM, DM, VAS | Procedure and fluoroscopy times showed construct validity of the Procedicus VIST, other metrics did not. VR was valued more by novices. | | | Klass et al, 2008 (57) | 12 new intake | Radiology registrars | 5 left RAS in 6 mo | SM, DM, MP | Procedure and fluoroscopy times improved, DM fluctuated. Procedure time as an endpoint. | | | Tedesco et al, 2008 (61) | 17 4th- and 5th-year residents | General surgery | RAS | SM, DM, GRS (2- expert), VAS | Procedure time, fluoroscopy time, contrast medium volume, coverage percentage, placement accuracy, residual stenosis, and cine-loops were similar between the two groups. Structured endovascular skills assessment based on a checklist carried out by a blinded assessor correlated well with prior procedural experience within a high-fidelity simulation environment. | | | Glaiberman et al, 2008 (77) | Interventional radiology fellows | Radiology fellowship | RAS (3 levels of difficulty) | SM, DM, GRS (2- expert) | SM and DM can help differentiate levels of skills. | | | Van Herzeele et al, 2007 (78) | 45 experts with varying experience | Cardiologists, radiologists, vascular surgeons | CAS | SM,DM, VAS | Procedure and fluoroscopy times helped differentiate between levels of CAS experience in experienced interventionalists. DM scoring is currently not a valid mode of assessment and needs refinement. SM can differentiate expert interventional radiologists. | | | Aggarwal et al, 2006 (70) | 20 (>50 endovascular procedures as primary operator [n = 8], <10 procedures [n = 12]) | Vascular surgeons | RAS | SM | Procedure and fluoroscopy times improved. SM at previous study showed that novices can be better than experts based on procedure and fluoroscopy times (88). | | | Patel et al, 2006 (68) | 20 experienced interventionalists without experience in carotid artery angiography | Interventional cardiologists | Carotid angiograms; 5 test-trials | SM, DM | Procedure time, fluoroscopy time, and CE improved. Internal consistency is high for CE. CE has highest test-retest reliability. Contrast medium volume showed high enough test-retest reliability. This measure was low for procedure time, fluoroscopy time, and number of cine-loops. DM is a more reliable metric than procedure time, fluoroscopy time, and contrast use. Procedure time, fluoroscopy time and contrast medium volume are a crude assessment of the technical performance of the operator. | | | Berry et al, 2006 (91) | 8 expert interventional radiologists, 8 medical students | Interventional radiologists and medical students | RAS, 6 repetitions | SM, DM, VAS | No difference in residual stenosis, placement accuracy, procedure time, lesion coverage, or TVR. The fluoroscopy time was greater for the novice group (P < .01). Experts rated 6 of the 8 subjective parameters favorably, whereas the novice group approved of 7. With the exception of fluoroscopy time, SM failed to stratify performances based on experience level. | | | Dayal et al, 2004 (17) | 21 physicians (5 of them were experts) | Physicians | CAS | SM, GRS, TSC | Procedure and fluoroscopy times improved significantly for novices. No statistically significant difference in score, procedure time, or fluoroscopy time was noted for experts. Improvement was noted in guide wire and catheter manipulation skills in novices. Novices derived the greatest benefit from simulator training in a mentored program, whereas experienced interventionalists did not seem to derive significant benefit. | | | Procedicus-VIST and cognitive training | | | | | | | | Van Herzeele et al, 2008 (56) | 20 novice endovascular experience in 2 groups (groups A1, A2) and 27 expert endovascular physicians (group B) | Surgical trainees/physicians | IAS | SM, DM | With SM: A1 worse than A2, A1 worse than B, A2 better than B. With DM: A1 better than A2, A1 equal to B, A2 worse than B. Cognitive skills training significantly improved the quality of the end product on a VR endovascular simulator and is fundamental before the assessment of inexperienced subjects. Error module was not used in the study. | | | Procedicus VIST and cognitive, compared with pigs | | | | | | | | Berry et al, 2007 (47) | 12 vascular surgeons and interventional radiologists | Vascular surgeons and interventional radiologists | IAS | TSC, GRS, task-specific checklist | Endovascular skills learned in the virtual environment may be transferable to the real catheterization laboratory as modelled in the Porcine Laboratory. Skills acquired by using pigs were not transferable when using VR. SM and DM were not used. | | | AngioMentor angiographic simulator | | | | | | | | Duncan et al, 2007 (5) | 5 experienced angiographers | Angiographers | RAS | SM, DM (from videos), and aggregated together | Task analysis facilitated both protocol development and data analysis. Efficiency metrics were readily extracted from recordings of simulated procedures. Aggregating the metrics and dividing the procedure into segments revealed potential insights that could be easily overlooked because the simulator currently does not attempt to aggregate the metrics and only provides data derived from the entire procedure. The data indicate that analysis of simulated angiographic procedures will be a powerful method of assessing performance in interventional radiology. | | | Simulator model | | | | | | | | Hsu et al, 2004 (69) | 29 subjects (16 untrained, 13 advanced) | Neurosurgeons | CAS | Procedure time | Performance on the carotid stent placement simulator correlated with previous endovascular experience. Although both novice and advanced groups improved their procedure time after a 30–60-minute proctored training session, improvement in the novice group was greater than that in the advanced group, which suggests that novices may benefit disproportionately from this type of training. (continued) | | | Dawson et al, 2007 (15) | Vascular surgery residents; type of simulator: SimSuite | Vascular surgery trainees | Arteriography and intervention for treatment of aortoiliac, renal, and carotid artery disease | Procedure time, Contrast medium volume | Procedure time and contrast medium volume improved. Selection of angioplasty balloon catheters and stents was improved. | | | | |
| ⁎ CAS = carotid artery stenosis angioplasty procedure, IAS = iliac artery stenosis angioplasty procedure, RAS = renal artery stenosis angioplasty procedure. |
Most of the available studies with regard to endovascular intervention are based on VR simulation. Wide variability exists in results from the studies carried out to date in terms of establishing the validity of static and dynamic parameters (Table 4) of VR simulators. Although performance and fluoroscopy times are used by some investigators to establish construct validity of the VIST simulator (17, 70, 57), others considered these parameters as not showing construct validity (76). According to Van Herzeele et al (56), cognitive training improved the end product assessment but slowed the performance and fluoroscopy times. In addition, the performance and fluoroscopy times of novices was better than those of experts (56). Unlike others, who used performance and fluoroscopy times to assess performance improvement, this study assesses performance improvement on the end product assessment parameters (70). Another study (77) concludes that both static and dynamic parameters can be used to help differentiate between middle grade level participants. Van Herzeele et al (78) showed that performance and fluoroscopy times, components of static metrics, can be reliable in differentiating between experts and that dynamic metrics are unreliable, whereas Patel and Gould (79) have drawn the opposite results from their study. In another study (17), performance and fluoroscopy times were unable to differentiate the level of experienced performers; opposite results were demonstrated by another group (69). | | | | Parameter | Description | Construct Validity (Significant Results) | Intertest Reliability | |
|---|
| Procedure time | Minutes needed to complete the entire procedure | Berry et al 2008 (51), Klass et al (57), Dawson 2007 (15), Van Herzeele et al 2007 (78), Aggarwal et al 2006 (70), Patel et al 2006 (68) | ... | | | Fluoroscopy time | Duration of fluoroscopy during the procedure | Berry et al 2006 (91), Berry et al 2008 (51), Van Herzeele et al 2007 (78), Aggarwal et al 2006 (70), Patel et al (68) | ... | | | Contrast medium | Total amount of contrast medium used, in milliliters | Dawson et al 2007 (15), Patel et al 2006 (68) | Patel et al 2006 (68) | | | Cine-loops | No. recorded during procedure | ... | ... | | | Lesion coverage | Percentage of lesion covered by selected tool | ... | ... | | | Tool:lesion ratio | Inflated tool's diameter to the lesion diameter | ... | ... | | | Placement accuracy | Longitudinal distance in millimeters between the tool's center and the lesion's center | ... | ... | | | Residual stenosis | Percentage stenosis after dilation or stent deployment | ... | ... | | | | |
Error scoring has been shown to demonstrate significantly the level of experience in studies based on the Minimally Invasive Surgical Trainer–Virtual Reality (MIST-VR; Mentice Medical Simulation, Gothenburg, Sweden) simulator (80). However, error scorings based on assessment through VIST simulators are unable to demonstrate significant trends (57). Error analysis is a well-established method of scoring the performance, and the reasons why errors are unable to demonstrate the difference could be due to the fact that task analysis is not adequate. Due to this wide level of discrepancy among the results, individual tasks in a procedure must be analyzed by the experts to define metrics and critical performance indicators. Test validation should include content, construct, concurrent, and predictive validation with the objective of demonstrating transfer of VR-trained skills to procedures on patients. Feedback must be analyzed in the context of the educational effect and acceptability to both the trainers and the trainees, as a system not acceptable to either will not last for long (81, 82). Furthermore, many studies on training and transfer have focused on time to completion as the major endpoint (70, 83, 84). Experts can perform a procedure in a given time, but for a trainee or novice time to completion should not be considered as an indicator of technical skill. Quantitative measurements such as procedure and fluoroscopic times may measure efficiency but are inadequate for assessing quality. The important parameters must be analysis of errors and the quality of the end product (33, 85). Advances in simulator technology have presented a great opportunity for endovascular training and assessment. However, key questions about the exact role of the existing simulators in training programs must still be resolved. It is evident that the current generation of simulators, regardless of the degree of fidelity, will only function as an adjunct to, not a replacement for, traditional teaching in the clinical setting. In existing form, simulators are unable to provide a substitute for the skills and experience gained by a trainee while under the close supervision of a mentor in real settings. Simulators are more approved by novices than by experts. This shows that simulators have an evident role in familiarizing the novices with the technique of a certain procedure: the tools, the procedure steps, the basic target, the possible errors, and the basic movements. Once these components of a procedure are already known to an operator—as in the case of experts—the value of available simulators becomes limited. This might be explained by Schmidt's theory, as noted above, that the limited variety of situations provided by the simulator for a specific procedure is in fact a limitation in the training capability of these tools. In addition, the reduced tactile feedback and lack of exact replication of fine motor skills are limitations. A great deal of work is already in progress to further develop and design high-fidelity endovascular simulators to more closely mirror the “real” interventional procedural experience and to improve VR validation and the ability for skill transfer. Depending on continued development, improved design, and further validation, it is likely that simulation-based training and assessment will be included in the curriculum in the near future. Moreover, progress is being made toward the approximation of educational and professional experts with a common goal on the horizon (ie, structured training with objective assessment). The Society of Interventional Radiology, the Cardiovascular and Interventional Radiological Society of Europe, and the Radiological Society of North America have established a medical simulation task force that is also supported by the British Society of Interventional Radiologists (86). The common objective is to establish recommendations for the validation of simulator models, use of simulators for training and certification in interventional radiology (38, 87). Simulation models (Fig 2) may be suitable for certain aspects of procedural experience, such as learning the sequence of procedural steps and appropriate tool selection, thereby reducing the procedure learning curve. Patient-specific procedural simulation is another promising utility of simulators that can help interventional radiologists maintain their proficiency and skills (88, 90). There is no counterpart to the quality of clinical workplace-based training and assessment. Simulators cannot be the replacement for training in a real environment because of the level of decision making, complexity, and interactions involved there. Therefore, training on simulators can only serve as an augmentation to the quality of training in the real setting. In view of continuing advancement in technology, simulation training may become a prerequisite for certification or credentialing; in its present form, however, it is insufficient for either (38). In conclusion, simulation training provides a safe platform for training outside the clinical workplace. Before widespread inclusion into practice, performance assessment parameters of the VR simulators must be rationalized with subsequent validation. Further research regarding human factors–based simulator development and design is required. The issues with transferability of various training models, particularly if they are to become a part of training and assessment for certification or licensure, must be addressed. Development of a model is needed to simulate an entire interventional radiologic procedure. Furthermore, trainer observational assessment tool validation for marking trainees and specialists is still pending. References 1. 1Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177:28–32. Abstract | Full Text |
Full-Text PDF (124 KB)
|
CrossRef
2. 2Gates EA. New surgical procedures: can our patients benefit while we learn?. Am J Obstet Gynecol. 1997;176:1293–1298. Abstract | Full Text |
Full-Text PDF (808 KB)
|
CrossRef
3. 3Warf BC, Donnelly MB, Schwartz RW, Sloan DA. Interpreting the judgment of surgical faculty regarding resident competence. J Surg Res. 1999;86:29–35. Abstract |
Full-Text PDF (126 KB)
|
CrossRef
4. 4Dawson S, Gould DA. Procedural simulation's developing role in medicine. Lancet. 2007;369:1671–1673. Full Text |
Full-Text PDF (46 KB)
|
CrossRef
5. 5Wong JA, Matsumoto ED. Primer: cognitive motor learning for teaching surgical skill—how are surgical skills taught and assessed?. Nat Clin Pract Urol. 2008;5:47–54.
CrossRef
6. 6Kopta JA. The development of motor skills in orthopaedic education. Clin Orthop Relat Res. 1971;75:80–85.
CrossRef
7. 7Schmidt RA. A schema theory of discrete motor skill learning. Psychol Rev. 1975;82:225–260. 8. 8Collins A, Brown JS, Newman SE. Cognitive apprenticeship: teaching the crafts of reading, writing, and mathematics. In: Resnick LB editors. Knowing, learning, and instruction: essays in honor of Robert Glaser. Hillsdale, NJ: Lawrence Erlbaum Associates; 1989;p. 453–494. 9. 9Ericsson KA. The acquisition of expert performance: an introduction to some of the issues. In: Ericsson KA editors. The road to excellence: the acquisition of expert performance in the arts and sciences, sports, and games. Mahwah, NJ: Lawrence Erlbaum; 1996;p. 1–50. 10. 10Halsted WS. The training of the surgeon. Bull Johns Hopkins Hosp. 1904;15:267–275. 11. 11Iram R. The Far East. In: Iram R editors. Surgery. an illustrated history. Philadelphia: Mosby; 1993;p. 65–69. 12. 12Sacks D. President's address at the 2008 SIR annual members' business meeting. J Vasc Interv Radiol. 2009;20:303–305. Full Text |
Full-Text PDF (53 KB)
|
CrossRef
13. 13Safar P. Ventilatory efficacy of mouth-to-mouth artificial respiration; airway obstruction during manual and mouth-to-mouth artificial respiration. J Am Med Assoc. 1958;167:335–341. 14. 14Denson JS, Abrahamson S. A computer-controlled patient simulator. JAMA. 1969;208:504–508. MEDLINE 15. 15Dawson DL, Meyer J, Lee ES, Pevec WC. Training with simulation improves residents' endovascular procedure skills. J Vasc Surg. 2007;45:149–154. Abstract | Full Text |
Full-Text PDF (231 KB)
|
CrossRef
16. 16Wang T, Darzi A, Foale R, Schilling RJ. Virtual reality permanent pacing: validation of a novel computerized permanent pacemaker implantation simulator. J Am Coll Cardiol. 2001;37:493A–494A. 17. 17Dayal R, Faries PL, Lin SC, et al. Computer simulation as a component of catheter-based training. J Vasc Surg. 2004;40:1112–1117. Abstract | Full Text |
Full-Text PDF (198 KB)
|
CrossRef
18. 18Regulations relating to the examinations for the diploma of fellow. In: Minutes of Council 1941–1943. London, England: Royal College of Surgeons of England; 1943;p. 413–423. 19. 19Quarterly council, 1946. In: Minutes of Council 1945–1947. London, England: Royal College of Surgeons of England; 1947;p. 447–461. 20. 20Peterson S. The ADA chalk carving test. J Dent Educ. 1974;38:11–15. MEDLINE 21. 21Deubert LW, Smith MC, Jenkins CB, Berry DC. The selection of dental students (a pilot study of an assessment of potential manula ability by psychometric tests). Br Dent J. 1975;139:167–170. MEDLINE |
CrossRef
22. 22Wong AY, Watson JF, Thye RP. Evaluation of predictor variables for a self-instructional preclinical course. J Dent Educ. 1979;43:637–640. MEDLINE 23. 23Lippert FG, Spolek GA, Kirkpatrick GS, Briggs KA, Clawson DK. A psychomotor skills course for orthopaedic residents. J Med Educ. 1975;50:982–983. MEDLINE 24. 24Apley AG. Fixation of fractures: organising a course. Ann R Coll Surg Engl. 1980;62:219–222. MEDLINE 25. 25Gough MH, Holdsworth R, Bell JA, et al. Personality assessment techniques and aptitude testing as aids to the selection of surgical trainees. Symposium. England, 18 November 1987. Ann R Coll Surg Engl. 1988;70:265–279. MEDLINE 26. 26Gough M, Bell J. Introducing aptitude testing into medicine. Br Med J. 1989;298:975–976. 27. 27Grace DM. Aptitude testing in surgery. Can J Surg. 1989;32:396–397. 28. 28Graham KS, Deary IJ. A role for aptitude testing in surgery?. J R Coll Surg Edinb. 1991;36:70–74. MEDLINE 29. 29Couves CM. A course in surgical technique for medical students. Can J Surg. 1970;13:31–32. 30. 30Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J. 1975;1:447–451. MEDLINE 31. 31Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ. 1979;13:41–54. MEDLINE 32. 32Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84:273–278. MEDLINE |
CrossRef
33. 33Joice P, Hanna GB, Cuschieri A. Errors enacted during endoscopic surgery: –a human reliability analysis. Appl Ergon. 1998;29:409–414. MEDLINE |
CrossRef
34. 34Tang B, Hanna GB, Joice P, Cuschieri A. Identification and categorization of technical errors by observational clinical human reliability assessment (OCHRA) during laparoscopic cholecystectomy. Arch Surg. 2004;139:1215–1220. MEDLINE |
CrossRef
35. 35Malik R, White PS, Macewen CJ. Using human reliability analysis to detect surgical error in endoscopic DCR surgery. Clin Otolaryngol Allied Sci. 2003;28:456–460. MEDLINE |
CrossRef
36. 36Cox A, Dolan L, Macewen CJ. Human reliability analysis: a new method to quantify errors in cataract surgery. Eye. 2008;22:394–397.
CrossRef
37. 37Chaer RA, Derubertis BG, Lin SC, et al. Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg. 2006;244:343–352. MEDLINE 38. 38Gould DA, Reekers JA, Kessel DO, et al. Simulation devices in interventional radiology: caveat emptor. Cardiovasc Intervent Radiol. 2006;29:4–6. MEDLINE |
CrossRef
39. 39Gould DA. Interventional radiology simulation: prepare for a virtual revolution in training. J Vasc Interv Radiol. 2007;18:483–490. Abstract | Full Text |
Full-Text PDF (282 KB)
|
CrossRef
40. 40Krummel TM. Surgical simulation and virtual reality: the coming revolution. Ann Surg. 1998;228:635–637. MEDLINE |
CrossRef
41. 41Reznick RK, MacRae H. Teaching surgical skills: –changes in the wind. N Engl J Med. 2006;355:2664–2669.
CrossRef
42. 42Tokuda Y, Song MH. A simple training model for coronary artery anastomoses. Heart Surg Forum. 2006;9:E880–E882.
CrossRef
43. 43Stanbridge Rde L, O'Regan D, Cherian A, Ramanan R. Use of a pulsatile beating heart model for training surgeons in beating heart surgery. Heart Surg Forum. 1999;2:300–304. MEDLINE 44. 44Suzuki Y, Fujitsuka M, Chaloupka JC. Simulation of endovascular neurointervention using silicone models: imaging and manipulation. Neurol Med Chir (Tokyo). 2005;45:567–572. MEDLINE |
CrossRef
45. 45Schachner T, Bonaros N, Ruttmann E, et al. Training models for coronary surgery. Heart Surg Forum. 2007;10:E248–E250.
CrossRef
46. 46Donias HW, Schwartz T, Tang DG, et al. A porcine beating heart model for robotic coronary artery surgery. Heart Surg Forum. 2003;6:249–253. 47. 47Berry M, Lystig T, Beard J, Klingestierna H, Reznick R, Lonn L. Porcine transfer study: virtual reality simulator training compared with porcine training in endovascular novices. Cardiovasc Intervent Radiol. 2007;30:455–461. MEDLINE |
CrossRef
48. 48Garrett HE. A human cadaveric circulation model. J Vasc Surg. 2001;33:1128–1130. Abstract | Full Text |
Full-Text PDF (73 KB)
|
CrossRef
49. 49Satava RM. Virtual reality surgical simulator: the first steps. Surg Endosc. 1993;7:203–205. MEDLINE |
CrossRef
50. 50Riva G. Applications of virtual environments in medicine. Methods Inf Med. 2003;42:524–534. MEDLINE 51. 51Berry M, Hellstrom M, Gothlin J, Reznick R, Lonn L. Endovascular training with animals versus virtual reality systems: an economic analysis. J Vasc Interv Radiol. 2008;19:233–238. Abstract | Full Text |
Full-Text PDF (70 KB)
|
CrossRef
52. 52Neequaye SK, Aggarwal R, Van Herzeele I, Darzi A, Cheshire NJ. Endovascular skills training and assessment. J Vasc Surg. 2007;46:1055–1064. Abstract | Full Text |
Full-Text PDF (537 KB)
|
CrossRef
53. 53Duncan JR, Kline B, Glaiberman CB. Analysis of simulated angiographic procedures. II. Extracting efficiency data from audio and video recordings. J Vasc Interv Radiol. 2007;18:535–544. Abstract | Full Text |
Full-Text PDF (242 KB)
|
CrossRef
54. 54Duncan JR, Glaiberman CB. Analysis of simulated angiographic procedures. I.– Capture and presentation of audio and video recordings. J Vasc Interv Radiol. 2006;17:1979–1989. Abstract | Full Text |
Full-Text PDF (1191 KB)
55. 55Sutherland LM, Middleton PF, Anthony A, et al. Surgical simulation: a systematic review. Ann Surg. 2006;243:291–300. MEDLINE |
CrossRef
56. 56Van Herzeele I, Aggarwal R, Neequaye S, et al. Experienced endovascular interventionalists objectively improve their skills by attending carotid artery stent training courses. Eur J Vasc Endovasc Surg. 2008;35:541–550. Abstract | Full Text |
Full-Text PDF (389 KB)
|
CrossRef
57. 57Klass D, Tam MD, Cockburn J, Williams S, Toms AP. Training on a vascular interventional simulator: an observational study. Eur Radiol. 2008;18:2874–2878.
CrossRef
58. 58Grantcharov TP, Reznick RK. Teaching procedural skills. Br Med J. 2008;336:1129–1131. 59. 59Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287:226–235. MEDLINE |
CrossRef
60. 60Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65:S63–S67. MEDLINE |
CrossRef
61. 61Tedesco MM, Pak JJ, Harris EJ, Krummel TM, Dalman RL, Lee JT. Simulation-based endovascular skills assessment: the future of credentialing?. J Vasc Surg. 2008;47:. 62. 62Pandey VA, Wolfe JH, Liapis CD, Bergqvist D. The examination assessment of technical competence in vascular surgery. Br J Surg. 2006;93:1132–1138. MEDLINE |
CrossRef
63. 63Sidhu RS, Park J, Brydges R, MacRae HM, Dubrowski A. Laboratory-based vascular anastomosis training: a randomized controlled trial evaluating the effects of bench model fidelity and level of training on skill acquisition. J Vasc Surg. 2007;45:343–349. Abstract | Full Text |
Full-Text PDF (265 KB)
|
CrossRef
64. 64Datta V, Bann S, Aggarwal R, Mandalia M, Hance J, Darzi A. Technical skills examination for general surgical trainees. Br J Surg. 2006;93:1139–1146. MEDLINE |
CrossRef
65. 65Beard JD, Choksy S, Khan S. Assessment of operative competence during carotid endarterectomy. Br J Surg. 2007;94:726–730. MEDLINE |
CrossRef
66. 66Brydges R, Sidhu R, Park J, Dubrowski A. Construct validity of computer-assisted assessment: quantification of movement processes during a vascular anastomosis on a live porcine model. Am J Surg. 2007;193:523–529. Abstract | Full Text |
Full-Text PDF (519 KB)
|
CrossRef
67. 67Hislop SJ, Hsu JH, Narins CR, et al. Simulator assessment of innate endovascular aptitude versus empirically correct performance. J Vasc Surg. 2006;43:47–55. Abstract | Full Text |
Full-Text PDF (512 KB)
|
CrossRef
68. 68Patel AD, Gallagher AG, Nicholson WJ, Cates CU. Learning curves and reliability measures for virtual reality simulation in the performance assessment of carotid angiography. J Am Coll Cardiol. 2006;47:1796–1802. Abstract | Full Text |
Full-Text PDF (188 KB)
|
CrossRef
69. 69Hsu JH, Younan D, Pandalai S, et al. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40:1118–1125. Abstract | Full Text |
Full-Text PDF (216 KB)
|
CrossRef
70. 70Aggarwal R, Black SA, Hance JR, Darzi A, Cheshire NJ. Virtual reality simulation training can improve inexperienced surgeons' endovascular skills. Eur J Vasc Endovasc Surg. 2006;31:588–593. Abstract | Full Text |
Full-Text PDF (142 KB)
|
CrossRef
71. 71Kundhal PS, Grantcharov TP. Psychomotor performance measured in a virtual environment correlates with technical skills in the operating room. Surg Endosc. 2008;. 72. 72Bann SD, Khan MS, Darzi AW. Measurement of surgical dexterity using motion analysis of simple bench tasks. World J Surg. 2003;27:390–394. MEDLINE |
CrossRef
73. 73Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg. 2001;193:479–485. Abstract | Full Text |
Full-Text PDF (212 KB)
|
CrossRef
74. 74Hance J, Aggarwal R, Stanbridge R, et al. Objective assessment of technical skills in cardiac surgery. Eur J Cardiothorac Surg. 2005;28:157–162. Abstract | Full Text |
Full-Text PDF (215 KB)
|
CrossRef
75. 75Vleuten CPM. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Education. 1996;1:41. 76. 76Berry M, Reznick R, Lystig T, Lonn L. The use of virtual reality for training in carotid artery stenting: a construct validation study. Acta Radiol. 2008;49:801–805.
CrossRef
77. 77Glaiberman CB, Jacobs B, Street M, Duncan JR, Scerbo MW, Pilgrim TK. Simulation in training: one-year experience using an efficiency index to assess interventional radiology fellow training status. J Vasc Interv Radiol. 2008;19:1366–1371. Abstract | Full Text |
Full-Text PDF (297 KB)
|
CrossRef
78. 78Van Herzeele I, Aggarwal R, Choong A, Brightwell R, Vermassen FE, Cheshire NJ. Virtual reality simulation objectively differentiates level of carotid stent experience in experienced interventionalists. J Vasc Surg. 2007;46:855–863. Abstract | Full Text |
Full-Text PDF (520 KB)
|
CrossRef
79. 79Patel AA, Gould DA. Simulators in interventional radiology training and evaluation: a paradigm shift is on the horizon. J Vasc Interv Radiol. 2006;17:S163–S173. Full Text |
Full-Text PDF (896 KB)
80. 80Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl. 1999;81:281–286. MEDLINE 81. 81Ahmed K, Ashrafian H, Hanna GB, Darzi A, Athanasiou T. Assessment of specialists in cardiovascular practice. Nat Rev Cardiol. 2009;659–667. 82. 82Ahmed K, Nagpal K, Ashrafian H. Do we need to train assessors?. Med Educ. 2009;43:389.
CrossRef
83. 83Lehmann KS, Ritz JP, Maass H, et al. A prospective randomized study to test the transfer of basic psychomotor skills from virtual reality to physical reality in a comparable training setting. Ann Surg. 2005;241:442–449. MEDLINE |
CrossRef
84. 84Hamilton EC, Scott DJ, Fleming JB, et al. Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surg Endosc. 2002;16:406–411.
CrossRef
85. 85Szalay D, MacRae H, Regehr G, Reznick R. Using operative outcome to assess technical skill. Am J Surg. 2000;180:234–237. Abstract | Full Text |
Full-Text PDF (144 KB)
|
CrossRef
86. 86Gould D, Patel A, Becker G, et al. SIR/RSNA/CIRSE Joint Medical Simulation Task Force strategic plan executive summary. J Vasc Interv Radiol. 2007;18:953–955. Full Text |
Full-Text PDF (48 KB)
|
CrossRef
87. 87Gould DA, Reekers JA, Kessel DO, et al. Simulation devices in interventional radiology: validation pending. J Vasc Interv Radiol. 2006;17:215–216. Full Text |
Full-Text PDF (50 KB)
88. 88Dawson S. Procedural simulation: a primer. J Vasc Interv Radiol. 2006;17:205–213. Abstract | Full Text |
Full-Text PDF (476 KB)
89. 89Holdsworth RF. Objective assessment: the state of the art. Ann Royal Coll Surg Engl. 1988;70:266. 90. 90Van Herzeele I, Aggarwal R, Neequaye S, Darzi A, Vermassen F, Cheshire NJ. Cognitive training improves clinically relevant outcomes during simulated endovascular procedures. J Vasc Surg. 2008;1223–1230. 91. 91Berry M, Lystig T, Reznick R, Lonn L. Assessment of a virtual interventional simulator trainer. J Endovasc Ther. 2006;13:237–243. MEDLINE |
CrossRef
a Department of Biosurgery & Surgical Technology, Imperial College London, 10th Fl, QEQM Bldg, St. Mary's Hospital Campus, Praed St., London, W2 1NY, England b Department of Radiology, St Mary's Hospital, London, England c Department of Radiology, Northwestern Memorial Hospital, Chicago, Illinois  Address correspondence to K.A
None of the authors have identified a conflict of interest. PII: S1051-0443(09)00961-0 doi:10.1016/j.jvir.2009.09.019 © 2010 SIR. Published by Elsevier Inc. All rights reserved. | |
|
FULL TEXT
