PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Technol Cancer Res Treat. Author manuscript; available in PMC 2017 April 1.
Published in final edited form as:
PMCID: PMC4826053
NIHMSID: NIHMS736853

Simulation-Based Cryosurgery Intelligent Tutoring System (ITS) Prototype

Anjali Sehrawat, PhD,1 Robert Keelan, PhD,1 Kenji Shimada, Ph.D.,1 Dona M. Wilfong, DNP, RN,2 James T. McCormick, D.O.,3 and Yoed Rabin, D.Sc.1,2,4

Abstract

As a part of an ongoing effort to develop computerized training tools for cryosurgery, the current study presents a proof-of-concept for a computerized tool for cryosurgery tutoring. The tutoring system lists geometrical constraints of cryoprobes placement, simulates cryoprobe insertion, displays a rendered shape of the prostate, enables distance measurements, simulates the corresponding thermal history, and evaluates the mismatch between the target region shape and a pre-selected planning isotherm. The quality of trainee planning is measured in comparison with a computer-generated planning, created for each case study by previously developed planning algorithms. Two versions of the tutoring system have been tested in the current study: (i) an unguided version, where the trainee can practice cases in unstructured sessions, and (ii) an intelligent tutoring system (ITS), which forces the trainee to follow specific steps, believed by the authors to potentially shorten the learning curve. While the tutoring level in this study aims only at geometrical constraints on cryoprobe placement and the resulting thermal histories, it creates a unique opportunity to gain insight into the process outside of the operation room. Posttest results indicate that the ITS system maybe more beneficial than the non-ITS system, but the proof-of-concept is demonstrated with either system.

Keywords: Cryosurgery, Planning, Training, Simulation, Bioheat, Intelligent System

Introduction

Cryosurgery is the destruction of undesirable tissue by freezing. Minimally invasive cryosurgery is performed by strategically placing an array of cryoprobes within a target region in order to maximize internal freezing damage, while minimizing damage to the surrounding tissues. In the case of prostate cryosurgery, the target region for destruction may be all or a preselected portion of the prostate gland (1). A key to the successful cryosurgery outcome is the optimal selection of the number of cryoprobes, the cryoprobe layout, and cryoprobe thermal history (collectively termed cryosurgery parameters in the current study). An ideal cryoprobe plan involves the creation of a three-dimensional (3D) thermal field such that a pre-selected isotherm (the planning isotherm) matches perfectly to the outer surface of the target region. The selected isotherm can be: (a) the temperature at the onset of freezing, which is closely related to the visualized frozen region by means of medical imaging; (b) the lethal temperature—a temperature threshold below which maximum destruction is assumed; or, (c) a clinically relevant temperature selected based on the surgeon’s own preference (2).

Cryoprobe placement has yet to be standardized, where many cryosurgeons use techniques based on their own experience, recommendations made by cryodevices manufacturers, and accepted practices. Suboptimal cryoprobe layout may unintentionally leave untreated areas in the target region, lead to cryoinjury in the healthy surrounding tissues, require unnecessarily large numbers of cryoprobes, increase the duration of the surgical procedure, and increase the likelihood of post-cryosurgery complications, all of which affect the quality and cost of the medical treatment (3).

Concurrent training in minimally invasive cryosurgery follows the apprenticeship model, wherein a novice physician first reviews a set of guidelines believed to yield cryosurgery success, followed by observing surgical practice by an experienced physician. The surgical case may include computer-assisted planning, provided by the cryosurgery hardware manufacturer. Following the apprenticeship approach and relying on available clinical cases may be constrained by opportunity, availability of patients and instruction time, and may even conflict with clinical operations (4). Moreover, since cryoprobe placement is not standardized, the applied cryosurgery techniques and the supporting concepts are likely to vary among instructing cryosurgeon. While the apprenticeship approach may provide appropriate training for specific cases and environment of operation, it lacks the exposure to a wide range of cases and also to a wide base of knowledge, which would lead to the development of a broader skillset.

Computer-based training can potentially overcome some of the limitations inherent to traditional education methods, while offering a variety of potential benefits, such as reduced risk, increased cost-effectiveness, increased opportunities for demonstration and practice/exercise, improved means to assess knowledge and competences, and mitigation of ethical issues associated with training on patients (5). Computer-based training tools can be conveniently grouped into two categories: part-task trainers and screen-based systems. A part-task trainer includes a part- or a full-body mannequin capable of replicating normal patho-physiological vital signs. Screen-based training tools consist of multi-media applications and virtual-reality simulators, often paired with part-task trainers. A virtual reality system recreates the operating environment with high-fidelity and is typically accompanied by one or more haptic systems that replicate the kinesthetic and tactile perception of the task being trained (6). Virtual reality systems are widely used to train physicians on vascular access, endoscopic, arthroscopic and laparoscopic surgical techniques (79). Multi-media applications provide computer-assisted instruction (CAI) that may complement lectures or provide case-based instructions to hone decision-making and diagnostics skills (1013). Intelligent tutoring systems (ITSs)—the subject matter of the current study—represent a subset of CAI. An ITS integrates explicit programming of domain knowledge and pedagogic expertise, in order to provide individualized instruction and feedback throughout the student problem-solving process. The ITS presents a set of problems, each strategically broken down into multiple steps, allowing the system to intervene at every step as needed. By contrast, a traditional tutoring system does not intervene; it only provides feedback to the student based on the final solution. In essence, the ITS aims at emulating the teaching approach—much like one-on-one teacher-student interaction (14).

The earliest attempt to develop an ITS for medical applications was known as the GUIDON project in the early 1980s, which focused on the treatment of infectious meningitis, bacteremia, and causative organisms based upon individual patient presentation (15). In the mid-1990s, Elliot and co-workers developed the Cardiac Tutor (16), to teach resuscitation management of patients undergoing cardiac arrest. The Cardiac tutor integrated simulations of physiological and functional cardiac arrhythmias in response to various medications and interventions. In the late 1990s, parallel efforts focused on the development of two ITS prototypes to interpret and diagnose abnormalities in mammograms and neuroradiology imaging, termed the RadTutor and the MR Tutor, respectively (17,18). At about the same time, Crowley and co-workers developed the SlideTutor, an ITS prototype that teaches medical residents how to interpret pathology slides for diagnostic inference (19). In the following years, Evens developed the CIRCSIM, a cardiovascular physiology tutor emphasizing the causal relationships in circulatory physiology (20). Among the prototypes listed above, only the Cardiac Tutor (16) integrated the domain subject with simulation. The ITS approach for computer-based medical education presents a new and unique opportunity with fast growing potential, associated with rapid advancement in computation techniques, computer platforms, and communication technology.

As a part of an ongoing effort to develop computerized training tools for cryosurgery, the current study presents a proof-of-concept for a cryosurgery, simulation-based ITS. This ITS targets the acquisition of knowledge and the development of intuition necessary to create an optimal match between the frozen and the target regions, while using prostate cryosurgery as a developmental model. The current study combines a formative evaluation to determine the effectiveness of the ITS by measuring performance displayed by surgery residents. A reduced version of the simulator was used for control in this study, representing less intelligent training as a benchmark. To the best of the authors’ knowledge, this study represents the first attempt to develop a cryosurgery ITS and the first attempt to evaluate a computerized tutoring system for cryosurgery.

Cryosurgery ITS Development – Methodology & Framework

Challenges

ITS design is heavily influenced by the intrinsic nature of the domain (i.e., cryosurgery) and the transferability of its pedagogic strategy into computerized instruction. The two most common types of ITS are known as cognitive and constraint-based, differentiated by the specific learning theory used to model the domain. Cognitive modeling is based on Anderson’s ACT-R theory, which states that there are two types of knowledge: declarative and procedural. Initially declarative knowledge (i.e., fact-based knowledge) is learned, which is later translated into more efficient procedural knowledge (i.e., skills) (21). Here, the domain is modeled using procedural knowledge, represented in the form of rules. In general, cognitive tutors are considered superior for linear domains, where rules may be constructed in a way that force students to take a specific path to reach an established solution. Constraint-based modeling is rooted in Olsson’s theory of learning from performance errors, where mistakes result from declarative knowledge that has not been internalized in one’s procedural knowledge (22). Here, the domain is modeled using declarative knowledge represented in the form constraints. Constraint-based tutors allow students to reach a solution, or a solution-space through a somewhat self-selected path.

The case-based teaching approach, focusing on students’ successful integration of domain knowledge to solve a particular problem, is frequently employed in medical tutoring systems. Material presented in such tutors typically comes from an extensive database of annotated cases, in fields with fairly well-established, standardized practices. This allows the developer to represent the domain by a set of rules, consistent with linearity of standardized practice and problem-solving strategies demonstrated in the annotations. In cryosurgery however, where accepted practices and criteria for surgical success may vary among clinicians and practicing institutions, the selection of optimal cryosurgery parameters may be equivocal. Instead of having a single path to a unique cryosurgery solution, several acceptable solutions may be created, with each solution developed in a non-sequential iterative fashion. This observation calls for a problem-solving process that accommodates multiple solutions and strategies. Furthermore, many intuitive steps may be involved in cryosurgery practice, which are difficult to formalize into verbal instruction targeted towards learning. For example, a competent cryosurgeon must integrate concepts and knowledge from the following areas during surgery planning: patient history and evolution of the disease, geometry of the target region and surrounding tissues, cryosurgery hardware capabilities, knowledge and/or intuition about heat transfer during cryosurgery, and medical imaging interpretation.

The cryosurgeon must possess the ability to process the resulting data streams in order to create mentally reconstructed, three-dimensional images of the target region and the frozen region. The cryosurgeon must plan and operate the specific procedure with the goal of minimizing the mismatch between the above reconstructed images. The solution to this minimization process may come about by varying the cryosurgery parameters and by selecting the appropriate comparison metrics, which are often dictated by clinical and individual patient data. Some of the related surgeon’s abilities are intrinsic while others can only be developed by training. Either way, they can all improve with experience, whether in the operation room or by using a virtual tool such as the one developed in the current study.

The ITS development in the current study focuses on planning of the cryoprobe layout for a cryosurgical procedure, while placing an emphasis on thermal and geometrical considerations. For the current phase of development, it is assumed that the patient is a good candidate for the cryosurgery treatment, and that the trainee is competent with the relevant medical imaging method. Even with these assumptions in mind, it is quite challenging to encode some of the highly intuitive concepts listed earlier into computerized instruction for an ITS. The approach taken in this study combines trial-and-error interaction with customized ITS feedback. Hence, this paper presents a hybrid modeling approach, using both rules and constraints to represent the cryosurgery planning domain.

ITS Learning Objective

The learning objective in the current study is to develop or improve the ability to create an acceptable cryoprobe layout for cryosurgery. A necessary but not sufficient condition for achieving this objective is that each and every cryoprobe of the array satisfies a set of preselected procedural constraints. With this condition in mind, the learning objective is met if the trainee generates a similar or better-quality cryoprobe layout in comparison with a fully automated computer-generated layout, based on previously developed planning algorithms. The quality of planning is calculated as the overall volume mismatch between the planning isotherm and the target region contour.

Derived from common prostate-cryosurgery practices, specific criteria used in the current study, are: (1) any probe must not be closer than 3 mm from the prostate capsule, (2) any probe must not be closer than 3mm from urethra, and (3) the active cooling surface of a probe must be included within the target region. A tolerance of 1 mm is used as an acceptable match between the trainee probe placement and the computer-generated probe location, which is comparable with the certainty in distance measurements based on high-quality ultrasound imaging for prostate cryosurgery. The above criteria have been encoded in the form of constraints (geometric conditions) and rules (overall layout adequacy) that are used to evaluate the trainee solution. Note that the above rules and constraints do not define a planning solution or a strategy, where it is possible to find multiple cryoprobe layouts that fulfill all the above criteria. Furthermore, the computerized solution created by the trainer is not merely a correct solution but also a computer-optimized solution, generated by planning algorithms discussed in the ITS Architecture section below. Thus, it is used as a form of an expert solution in the current study. The learning objectives have been evaluated using typical thermal histories and cooling capabilities to Joule-Thomson based devices, as discussed in detail in (23). A representative prostate model volume of 35 ml was applied in all case studies.

ITS Architecture

With reference to Fig. 1, the cryosurgery trainer comprises of two components: ITS and computerized cryosurgery tools (CCT). The CCT is comprised of the cryosurgery simulator and the cryoprobe-layout planner. The simulator takes a specific cryoprobe layout as input and uses an efficient numerical technique to simulate the resulting bioheat transfer process in the treated tissue (23). The simulator’s output are: (i) the three-dimensional temperature field at the end of the simulated procedure, and (ii) the mismatch between the planning isotherm and target region shape, also defined as the defect value (24). The cryoprobe layout planner is based on the bubble-packing method (25,26), and is applied on the specific geometry of the target region (i.e., the prostate and the urethra) for a given number of cryoprobes. The ITS is comprised of three components: a domain module, a student module, and a tutor. The domain module contains a set of rules and constraints that govern the cryosurgery planning process, combined with means to evaluate the trainee solution against the computer generated solution. The student module records the trainee actions throughout the problem solving process, including possible violations of rules and constraints. The tutor can be viewed as the administrative portion of a physical instructor, as it presents the student with a problem, asks relevant questions, passes the trainee-input to the domain module for evaluation, and returns feedback to the student based on the evaluator output. The database holds all case-related information, including organ geometries, previously solved cases, and student track record.

Figure 1
Schematic illustration of the computerized cryosurgery trainer architecture

With reference to Fig. 2, the trainee-ITS interaction within each ITS case is comprised of five phases: (0) initial setup, (1) plan layout, (2) verify constraints, (3) cryosurgery simulation, and (4) case evaluation. The trainee can progress from one phase to the next only if all the rules and constraints are met. Otherwise, feedback containing the appropriate instructions are provided, to help correct the process. Phase 2 is the only phase where constraint violations do not impact the trainee’s ability to move on to the next phase. Here, the trainee is given direct feedback regarding probe-constraint violation and the trainee may move affected probes as desired. During phase 3, a defect minimization search is conducted using an optimization technique previously developed (23). The graphical user interface (GUI), shown in Figs. 3 and and4,4, provides the trainee with an interactive problem-solving environment. Prostate cryosurgery is typically performed under ultrasound guidance. In the current study, the added challenges associated with imaging analysis have not been integrated into the educational experience. Concurrent effort is now being devoted to integrate medical imaging analysis to cryosurgery education (27).

Figure 2
Schematic illustration if the sequence of operations during trainee-ITS interaction
Figure 3
A representative screen of the cryosurgery ITS used to evaluate rules and constrains in a tested cryoprobe layout: (A) instructions panel; (B) geometrical presentation of the prostate to be treated, where (1) is the prostate capsule, (2) is the urethral ...
Figure 4
A representative screen of the cryosurgery ITS to evaluate rules and constrains in a tested cryoprobe layout: (A) prostate contour with user-selected distance measurements; (B) temperature map presented at the end of a cryosurgery simulation, where pink ...

Study Design and Evaluation Methodology

It is reemphasized that the current study is not aimed at a comprehensive educational experience on how to perform cryosurgery, nor is it aimed at verifying simulation results with follow-up clinical procedures; those broader aims are left for follow-up investigations and extend well beyond a single report. The objective in the current study is to develop a proof-of-concept for a cryosurgery ITS trainer, to explore its effectiveness, and identify possible pathways for its future development.

It is quite difficult to study the effectiveness of the computerized training concept, as comparable data from current training methods (i.e., non-computerized) is virtually nonexistence. Therefore, the current study presents a novel benchmarking concept, where the ITS system performance is compared against the performance of a second but simplified computerized training system, containing no ITS components. The non-ITS trainer is essentially a reduced version of the same training code, where all functionalities are identical except for the features that provide feedback, guidance, and force the trainee to go through a sequence of instructional steps. In the non-ITS tutor, the trainee can study the 3D shape of the prostate, place cryoprobes, measure distances, run a bioheat simulation of the cryoprocedure, inspect the resulting 3D temperature field, and evaluate defect results relative to the computer-generated expert solution. While the non-ITS trainer provides the temperature field and defect value for a generated layout, it does not provide instructions on how to improve the layout using the resulting simulation data. Furthermore, the non-ITS tutor does not provide feedback on geometric constraints violations. In a sense, the non-ITS trainer facilitates intuition development in an unstructured manner.

The human subject study was approved by the Allegheny Singer Research Institute, Allegheny Health Network (AHN), Pittsburgh, PA (IRB Protocol #14-006). This study was performed at the Simulation, Teaching and Academic Research (STAR) Center of AHN. Twenty three subjects were recruited from the AHN General Surgical Residency Program, where 15 and 8 subjects participated in the ITS tutor and non-ITS evaluation, respectively. Of the 23 residents, 16 (70%) were male, 7 (30%) were female, not by choice but due to the current demographics of the surgical residency program at AHN. Five residents reported previous experience with medical software and 17 residents reported previous video gaming experience at variable level of competence.

For both the ITS and non-ITS groups, trainees were given a brief introduction to prostate cryosurgery and planning guidelines, followed by a demonstration on how to use the computer code. For both groups, each session combined six cases for training and three cases for post-training test. For each case, the trainee was presented with a prostate and urethra model and asked to use ten cryoprobes to plan a layout. Each case used a different geometrical model of the prostate and the urethra, which was generated by means of geometrical deformation, based on ultrasound-reconstructed organs (28). All cases were limited to a uniform insertion depth for cryoprobes (29), where the trainee can only select the x and y coordinates for each probe (follow-on studies are planned to address multi-depth configurations, which presents a higher level of planning complexity in 3D).

During training, the ITS trainees had to follow instructions provided by the system, resulting in a finite number of attempts until evaluation criteria was met. The non-ITS trainees could have as many attempts as they wish. During the post-training test, all trainees regardless of the group had only one attempt to solve each case. During these tests, all trainees had access to the distance-measuring tool but no additional means of feedback. After each planning case, the trainees were able to see the resulting defect values and bioheat simulation results. Trainees concluded the session by filling out a short questionnaire about their medical background, computation skills, and their thoughts about the tutor.

Results

All statistical analyses were performed using Statistical Package for the Social Sciences (SPSS), IBM Co. Comparison between the two tutor groups was performed using the Mann-Whitney U-test. Evaluation of the effectiveness of specific tutor features was performed using logistic regression and correlation between two variables was determined using Spearman's rho correlation coefficient.

For both tutor groups, the training session and posttest lasted on average 60 and 30 minutes, respectively. Figure 5 displays the mean time spent and number of attempts made on each case during the training portion of the session. Overall, the time required to complete a case tends to decrease with increasing number of cases solved (displayed in Fig. 5), with little difference between the two groups (U=50, p=0.52). The mean number of attempts per case remains in the same range throughout the training session for the ITS group, but varied considerably for the non-ITS group, with statistically significant difference between the groups (U=24.5, p=0.02).

Figure 5
(a) Mean time and (b) mean number of attempts per case number for ITS and non-ITS groups. Error bars represent standard error at 95% confidence interval.

For the ITS group, all cases are structured such that the solution must meet both defect and constraints criteria, resulting in solutions that are free of constraint violations. For the non-ITS group, the solution must only meet the defect criterion, although the trainee has the option (and was verbally encouraged during the introductory portion of the session) to continue modifying the layout as deemed necessary in order to meet all rules and constraints. Since no computerized mechanism prohibits the solution for a non-ITS case to be free of constraint violations, the trainee can use the distance measurements tool and must use their own judgment. In the lack of constraint violation enforcement, the number of solutions submitted by the non-ITS group that met the constraint criteria was remarkably low. The best performance in this group was 5 out of 6 cases that met the criteria, achieved only by one trainee. Due to technical problems, data was lost for the first case for one trainee; this trainee submitted 3 out of 5 passing cases. Out of the six cases, one trainee submitted a total of 2 passing cases, three trainees submitted only 1 passing case, and the remaining two trainees completed the training session with 0 cases that met the constraint criteria.

Both tutors had unlimited access to the distance measurement tool. Twelve out of 15 and 6 out of 7 trainees in the ITS and non-ITS groups, respectively, measured probe distances using the measurement tool in the first training case. The same measurement tool was used only once by five trainees (ITS: 1, non-ITS: 4) in one of their remaining five cases.

The effectiveness of the probe distance plots (labeled J on Fig. 3)—a feature only available for the ITS group—was evaluated using logistic regression. In this plot (labeled J), the red line represents the constraint, the blue line represents the actual distance between the probe and the respective organ contour along the active surface of the cryoprobe, the green bar represents a safe region, and a yellow bar represents the portion of the cryoprobe which violates the distance constraint. The constraints check table (labeled H on Fig. 3) is continuously displayed for review by the trainee. It is assumed for the current analysis that if the trainee did not use the probe distance plots (based on recorded keystrokes), the trainee must have used the constraints check table. Improvement in meeting the constraints criteria is defined as the reduction in number of probes violating a constraint during an attempt. A scenario in which the trainee had no constraint violation before and after using the same distance tool is also classified as an improvement, since it demonstrates that the trainee understands the geometric concepts while the operation is counted as a positive step towards meeting all constraints. Only a scenario in which the trainee moved a probe after checking constraints and before running a simulation were included in this analysis (n=101). Among the ITS group, no significant difference is observed between trainees who used the probe distance plots and those that did not (B= −0.816, SE=0.517, p=0.12).

The effectiveness of the temperature field plots was evaluated for the non-ITS tutor only, where the trainees could relocate probes with no restrictions. With reference to Fig. 4, label B points to the organ contour, label C points to the temperature-to-color scale bar, and black displays a defect region. Improvement is defined as decrease in the total defect value between two consecutive attempts, where the possible scenarios are listed in Table 1 (n=159). Logistic regression revealed that improvement in defect was not statistically significant with the use of temperature field plots (B=0.187, SE=0.357, p=0.60).

Table 1
Defect evaluation scheme to determine improvement between attempts in cryoprobe layout, where D is the trainee planning defect, i is the planning attempt counter, and G is the computer-generated planning defect.

To examine the tutors’ learning effect, the probe violation rate was calculated for each case as the ratio of the number of probes violating a constraint to the number of probes available to be moved at same stage, with the median error rate displayed in Fig. 6. Expectedly, each of the ITS cases eventually reaches a solution free of violations. By contrast, the non-ITS group displays no definitive pattern. The defect ratio (trainee defect value divided by the computer-generated defect value) and probe constraint violations after the first attempt for each case are plotted in Fig 7.

Figure 6
Median probe violation rate versus number of attempts, where Q1, Q2, and Q3 indicate quartiles of the total number of attempts by all trainees to complete the case.
Figure 7
The number of probes violating at least one constraint in the first planning attempt versus the defect ratio, defined as the ratio of trainee-planning defect to the computer-generated planning defect (optimized solution). Tabulated values display ratio ...

Post-training test results are displayed in Fig. 8. A defect ratio smaller than one indicates a superior trainee solution compared with the computer-generated solution. Results of Case 7 (the first posttest case) indicate an average trainee defect value close to the optimized computer-generated defect. Posttest results for the next two cases (#8 and #9) indicate trainee planning superior to computer generated planning, when constraint violations are not accounted for. In terms of the mean defect ratio alone (U=49, p=0.48) or in terms of the mean number of probes violations alone (U=52, p=0.60), there is no statistically significant difference between the ITS and the non-ITS groups. Bivariate correlation analysis was performed between trainees’ background, including self-rated computational skills (1 to 5 scale), years of video gaming experience, and year of residency and trainee’s posttest performance in terms of number of probes violating a constraint and defect ratio, but no statistically significant correlation was found.

Figure 8
The number of probes violating at least one constraint in the posttest versus the defect ratio, defined as the ratio of trainee-planning defect to the computer-generated planning (optimized solution) defect. A defect ratio smaller than one indicates a ...

Discussion

Inherent differences between trainees and varying complexity between prostate shapes are two major confounding effects in the current study. Given the total number of available trainees (medical residents) and volunteering time (up to two hours) for this proof-of-concept study, constructing a more elaborate study to filter trainee variability effects is deemed unwarranted. Additionally, given the typical volume and the variable shape of candidate prostate for cryosurgery, creating a larger selection of cases (nine cases used in the current study) may be warranted in future studies, where more flexible cryoprobe placement strategy will be tested (such as multi-depth insertion). Following this approach, further tools will have to be developed to control for the geometrical variations of the target region, an effort which extends beyond the proof-of-concept level of the current study.

Within the ITS group, using the probe distance plots (versus constraints check table) did not lead to statistically significant improvement in terms of probe placement violations. The frequency with which trainees used the distance plots decreased with increasing number of cases solved, which were used 47%, 44%, and 22% of the time during cases 1&2, 3&4, and 5&6, respectively. A plausible explanation for this behavior is that the trainee’s intuition about the layout quality may have increased, which is also consistent with the decreasing number of attempts as more cases are completed. It may also be that generating a higher incentive to use this tool may improve the training outcome.

The usage of the temperature-field plots also displayed a similar trend of deceased use as training progressed. In the ITS group, trainees used the plots 58% of the time in the first case and then their usage fluctuated between 12.5–38% (mean 25%) in the subsequent cases. The non-ITS group used temperature field plots over 70% of the time in the first two cases, which declined to the range of 37% to 58% of the time in subsequent cases. The feedback provided by the temperature-field plot is essential to layout-quality planning, but its interpretation requires analytical reasoning for which further step-by-step instructions may need to be integrated. Difficulties with this tool may be associated with scrolling through two-dimensional plot in order to create a mental image of a three-dimensional field. It appears that this difficulty can be overcome by at least one of the two alternatives: (i) pre-training in bioheat transfer effects, or (ii) integrating a tutor-generated three-dimensional illustration of the temperature field following (see (25) as an example). The latter alternative is more consistent with the approach presented in the current study, while the prior alternative may be more appropriate if training of the principles of cryosurgery is needed for cryosurgery hardware developers.

The ITS provides two additional forms of feedback during phases 3 and 1. In phase 3, if the layout has yet to meet the defect target, any probe that is placed within a predefined tolerance of its optimal location (according to the computer-generated optimized layout), is locked in place. With this hint, the trainee will then return to phase 1 and continue the session while focusing only on the remaining probes that are sub-optimally placed. In addition, the ITS will reveal the optimal location for an additional probe at the beginning of revisited phase 1—suggesting the least intuitive probe displacement for the trainee (i.e., the farthest required probe displacement). Once the trainee follows this move, the trainee has the freedom to keep moving the remaining probes in an attempt to improve the overall layout. With this scheme in mind, the training process will converge, where the maximum attempts for a cryoprobe layout is bounded by the number of cryoprobes. Average results displayed in Fig. 8 support this reasoning, indicating that indeed the ITS group displays superior performance in the posttest. Analysis of results distribution suggests that this scheme does not have a major impact on the trainee’s performance in terms of overall posttest results alone (Fig. 8). It may well be that this instruction scheme will yield a better return with a larger number of training cases or a longer time of ITS operation. However, this outcome calls for developing alternative and more efficient ITS schemes.

Regardless of the minimization process of the defect region, the ITS provides a unique opportunity to decrease constraint violations, as can be seen from Fig. 6, which is related to geometrical relationships rather than the thermal effects. In the first two cases in Fig. 6, the median probe violation rate is the same for both tutor groups during the first attempt. In the next three cases, the median rate for ITS is 0.1 below the non-ITS rate and in the last case, the median probe rate is 0 for both tutors. While the differences between the two tutors are small, both tutors appear to be effective in increasing the trainees’ understanding of geometric conditions. On the other hand, the shape of the overall curve of each tutor in Fig. 6 differs significantly. While the non-ITS curve displays marginal improvement with the increasing number of attempts, the ITS curve displays a significant effect on the quality of learning.

Figure 7 displays results of the first attempt for all the training cases, with the objective of evaluating the learning-curve trend during the training session. In terms of constraints violations, both tutors display improvement with increasing case number. In terms of overall performance, the ITS group performed better in 4 out of 6 cases than the non-ITS group. Nonetheless, compared with the first case, there is an improvement performance trend with increasing case number for both groups. With these observations in mind, it should be noted that six cases may be considered a small training set for trend evaluation as the geometric complexity of the prostate models may vary significantly.

While the data analysis relates primarily to how well the trainees followed the rules defined for training, some trainees had difficulties in reconciling those rules and their own clinical judgment about planning. These subjects were well aware of critical neighboring anatomical structures around the prostate and felt that, in some instances, it would be better to increase the local defect in favor of reducing risk of harming those structures. This led them to disagree with some of the tutor recommendations, which could have prolonged their session and led to increased error as interpreted by the tutor. With this observation in mind, no trainee had prior experience in cryosurgery, and it was well articulated that the current study is about how well the trainee can learn the computerized trainer rules. Ultimately, integrating individual clinical judgment will strengthen the educational experience. This goal will be achieved by live expert (human) mentorship present in the background and also, independently, by contextualizing the planning algorithms.

Summary

As a part of an ongoing effort to develop computerized training tools for cryosurgery, the current study presents a proof-of-concept for a computerized cryosurgery tutoring system. To the best of knowledge of the authors, this is the first-ever attempt to present and validate such a proof-of-concept. The tutoring system lists geometrical constraints of cryoprobes placement, simulates cryoprobe insertion, displays a rendered shape of the prostate, enables distance measurements, simulates the corresponding thermal history, and evaluate the mismatch between the target region shape and a pre-selected planning isotherm. The quality of trainee planning is measured in comparison with a computer-generated planning, created for each case study by previously developed planning algorithms. It is noted that the computer-generated solution is not unique, and alternative solutions can be found of similar quality; this observation is taken into account in the tutoring system feedback, as well as in data analysis in the current study. Two versions of the tutoring system have been used in the current study: (i) an unguided version, where the trainee can practice cases in unstructured sessions, and (ii) the ITS, where the system forces the trainee to follow specific steps, believed by the authors to potentially shorten the learning curve. Table 2 summaries system features and key conclusions for both training system versions.

Table 2
Summary of ITS versus non-ITS training systems and key conclusions, based on 23 subjects from the AHN surgical residency programs (ITS = 15, non-ITS = 8).

While the tutoring level in this study aims only at geometrical constraints on cryoprobe placement and the resulting thermal histories, it creates a unique opportunity to gain insight into the process outside of the operation room. It appears that the computerized trainer effectively illustrates several basic concepts: the geometry of the target region, how cryoprobe placement affects the resulting defect region, and how using two-dimensional cross-sections in cryoprobe placement (such as those created by the rectal ultrasound transducer during prostate cryosurgery) requires special attention to prevent damage in the third dimension. Posttest results indicate that the ITS system maybe more beneficial than the non-ITS system, but the proof-of-concept is demonstrated with either system. Based on the results displayed above, ongoing efforts are now devoted to formulate new methodologies, provide better feedback, and increase the incentive in learning, with the long-term goal of shortening the learning curve and improving its outcome.

Acknowledgement

This study was supported in parts by Award Number R01CA134261 from the National Cancer Institute. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health. This study has also been supported in part by the Simulation, Teaching and Academic Research (STAR) Center of the Allegheny Health Network, Pittsburgh, PA. Special thanks to the residents of the General Surgery Residency Program of the Allegheny Health Network for volunteering time and effort to support this study.

References

1. Babaian RJ, Donnelly B, Bahn D, et al. Best practice statement on cryosurgery for the treatment of localized prostate cancer. J Urol. 2008;180:1993–2004. [PubMed]
2. Gage AA, Baust J. Mechanisms of Tissue Injury in Cryosurgery. Cryobiology. 1998;37:171–186. [PubMed]
3. Rossi MR, Tanaka D, Shimada K, Rabin Y. Computerized Planning of Prostate Cryosurgery Using Variable Cryoprobe Insertion Depth. Cryobiology. 2010;60:71–79. [PMC free article] [PubMed]
4. Ziv A, Small SD, Wolpe PR. Patient safety and simulation-based medical education. Med Teach. 2000;22:489–495. [PubMed]
5. Scalese RJ, Obeso VT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education. J Gen Intern Med. 2008;23:46–49. [PMC free article] [PubMed]
6. Bradley P. The history of simulation in medical education and possible future directions. Med Educ. 2006;40:254–262. [PubMed]
7. Aggarwal R, Black SA, Hance JR, Darzi A, Cheshire NJW. Virtual Reality Simulation Training can Improve Inexperienced Surgeons’ Endovascular Skills. Eur J Vasc Endovasc Surg. 2006;31:588–593. [PubMed]
8. Larsen CR, Soerensen JL, Grantcharov TP, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ. 2009;338:b1802–b1802. [PMC free article] [PubMed]
9. Srivastava S, Youngblood PL, Rawn C, et al. Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity. J Shoulder Elb Surg Am Shoulder Elb Surg Al. 2004;13:196–205. [PubMed]
10. Jenkins S, Goel R, Morrell DS. Computer-assisted instruction versus traditional lecture for medical student teaching of dermatology morphology: A randomized control trial. J Am Acad Dermatol. 2008;59:255–259. [PubMed]
11. McNulty JA, Sonntag B, Sinacore JM. Evaluation of computer-aided instruction in a gross anatomy course: A six-year study. Anat Sci Educ. 2009;2:2–8. [PubMed]
12. Kopp V, Stark R, Fischer MR. Fostering diagnostic knowledge through computer-supported, case-based worked examples: effects of erroneous examples and feedback. Med Educ. 2008;42:823–829. [PubMed]
13. Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ. 2009;43:303–311. [PubMed]
14. Corbett A, Koedinger KR, Anderson J. In: Handbook of Human-Computer Interaction. Helander M, Landauer T, Prabhu P, editors. Elsevier Science BV; 1997.
15. Clancey W. The Handbook of Artificial Intelligence. William Kaufmann, Inc; 1982.
16. Eliot CR, Williams KA, Woolf BP. An intelligent learning environment for advanced cardiac life support. Proc AMIA Annu Fall Symp. 1996:7–11. PMCID: PMC2232984. [PMC free article] [PubMed]
17. Azevedo R, Lajoie SP. The cognitive basis for the design of a mammography interpretation tutor. Int J Artif Intell Educ. 1998;9:32–44.
18. Sharples M, Jeffery NP, du Boulay B, et al. Structured computer-based training in the interpretation of neuroradiological images. Int J Med Inf. 2000;60:263–280. [PubMed]
19. Crowley RS, Naus GJ, Friedman CP. Development of visual diagnostic expertise in pathology. Proc AMIA Symp. 2001:125–129. PMCID: PMC150358. [PMC free article] [PubMed]
20. Evens M, Brandle S, Change RC, et al. CIRCSIM-Tutor: An intelligent tutoring system using natural language dialogue; Twelfth Midwest AI and Cognitive Science Conference; 2001. pp. 16–23.
21. Anderson JR, Lebiere C. The atomic components of thought. Erlbaum; 1998.
22. Ohlsson S. In: Student Modelling: The Key to Individualized Knowledge-Based Instruction. Greer JE, McCalla GI, editors. Berlin Heidelberg: Springer; 1994. pp. 167–189.
23. Rossi MR, Tanaka D, Shimada K, Rabin Y. An Efficient Numerical Technique for Bioheat Simulations and its Application to Computerized Cryosurgery Planning. Comput Methods Programs Biomed. 2007;85:41–50. [PMC free article] [PubMed]
24. Lung DC, Stahovich TF, Rabin Y. Computerized planning for multiprobe cryosurgery using a force-field analogy. Comput Methods Biomech Biomed Engin. 2004;7:101–110. [PubMed]
25. Tanaka D, Shimada K, Rossi MR, Rabin Y. Cryosurgery planning using bubble packing in 3D. Comput Methods Biomech Biomed Engin. 2008;11:113–121. [PMC free article] [PubMed]
26. Rossi MR, Tanaka D, Shimada K, Rabin Y. Computerized Planning of Cryosurgery Using Bubble Packing: An Experimental Validation on a Phantom Material. Int J Heat Mass Transf. 2008;51:5671–5678. [PMC free article] [PubMed]
27. Keelan R, Shimada K, Rabin Y. Imaging artifacts, training interface and computational challenges. Annual Meeting of American College of Cryosurgery. 2014
28. Sehrawat A, Shimada K, Rabin Y. Generating prostate models by means of geometric deformation with application to computerized training of cryosurgery. Int J Comput Assist Radiol Surg. 2013;8:301–312. [PMC free article] [PubMed]
29. Tanaka D, Shimada K, Rabin Y. Two-phase Computerized Planning of Cryosurgery Using Bubble-packing and Force-field Analogy. J Biomech Eng. 2006;128:49–58. PMCID: PMC2259023. [PMC free article] [PubMed]