The rapid motion of the heart presents a significant challenge to the surgeon during intracardiac beating heart procedures. We present a 3D ultrasound-guided motion compensation system that assists the surgeon by synchronizing instrument motion with the heart. The system utilizes the fact that certain intracardiac structures, like the mitral valve annulus, have trajectories that are largely constrained to translation along one axis. This allows the development of a real-time 3D ultrasound tissue tracker that we integrate with a 1 degree-of-freedom (DOF) actuated surgical instrument and predictive filter to devise a motion tracking system adapted to mitral valve annuloplasty. In vivo experiments demonstrate that the system provides highly accurate tracking (1.0 mm error) with 70% less error than manual tracking attempts.
3D ultrasound; real-time tissue tracking; motion compensation; medical robotics; beating heart surgery
The network of collagen fibers in the aortic valve leaflet is believed to play an important role in the strength and durability of the valve. However, in addition to its stress-bearing role, such a fiber network has the potential to produce functionally important shape changes in the closed valve under pressure load. We measured the average pattern of the collagen network in porcine aortic valve leaflets after staining for collagen. We then used finite element simulation to explore how this collagen pattern influences the shape of the closed valve. We observed a curved or bent pattern, with collagen fibers angled downward from the commissures toward the center of the leaflet to form a pattern that is concave toward the leaflet free edge. Simulations showed that these curved fiber trajectories straighten under pressure load, leading to functionally important changes in closed valve shape. Relative to a pattern of straight collagen fibers running parallel to the leaflet free edge, the concave pattern of curved fibers produces a closed valve with a 40% increase in central leaflet coaptation height and with decreased leaflet billow, resulting in a more physiological closed valve shape. Furthermore, simulations show that these changes in loaded leaflet shape reflect changes in leaflet curvature due to modulation of in-plane membrane stress resulting from straightening of the curved fibers. This effect appears to play an important role in normal valve function and may have important implications for the design of prosthetic and tissue engineered replacement valves.
Finite element model; Aortic valve; Collagen pattern
Catheter devices allow physicians to access the inside of the human body easily and painlessly through natural orifices and vessels. Although catheters allow for the delivery of fluids and drugs, the deployment of devices, and the acquisition of the measurements, they do not allow clinicians to assess the physical properties of tissue inside the body due to the tissue motion and transmission limitations of the catheter devices, including compliance, friction, and backlash. The goal of this research is to increase the tactile information available to physicians during catheter procedures by providing haptic feedback during palpation procedures. To accomplish this goal, we have developed the first motion compensated actuated catheter system that enables haptic perception of fast moving tissue structures. The actuated catheter is instrumented with a distal tip force sensor and a force feedback interface that allows users to adjust the position of the catheter while experiencing the forces on the catheter tip. The efficacy of this device and interface is evaluated through a psychophyisical study comparing how accurately users can differentiate various materials attached to a cardiac motion simulator using the haptic device and a conventional manual catheter. The results demonstrate that haptics improves a user's ability to differentiate material properties and decreases the total number of errors by 50% over the manual catheter system.
Haptics; Medical Robotics; Catheter guidance; Psychophysics
Intracardiac echocardiography (ICE) catheters enable high-quality ultrasound imaging within the heart, but their use in guiding procedures is limited due to the difficulty of manually pointing them at structures of interest. This paper presents the design and testing of a catheter steering model for robotic control of commercial ICE catheters. The four actuated degrees of freedom (4-DOF) are two catheter handle knobs to produce bi-directional bending in combination with rotation and translation of the handle. An extra degree of freedom in the system allows the imaging plane (dependent on orientation) to be directed at an object of interest. A closed form solution for forward and inverse kinematics enables control of the catheter tip position and the imaging plane orientation. The proposed algorithms were validated with a robotic test bed using electromagnetic sensor tracking of the catheter tip. The ability to automatically acquire imaging targets in the heart may improve the efficiency and effectiveness of intracardiac catheter interventions by allowing visualization of soft tissue structures that are not visible using standard fluoroscopic guidance. Although the system has been developed and tested for manipulating ICE catheters, the methods described here are applicable to any long thin tendon-driven tool (with single or bi-directional bending) requiring accurate tip position and orientation control.
While polymeric fabrication processes, including recent advances in additive manufacturing, have revolutionized manufacturing, little work has been done on effective sensing elements compatible with and embedded within polymeric structures. In this paper, we describe the development and evaluation of two important sensing modalities for embedding in polymeric mechatronic and robotic mechanisms: multi-axis flexure joint angle sensing utilizing IR phototransistors, and a small (12 mm), three-axis force sensing via embedded silicon strain gages with similar performance characteristics as an equally sized metal element based sensor.
sensors; embedded; polymer; robotic; shape deposition manufacturing; rapid prototyping
A fiber-based projection-imaging system is proposed for shape measurement in confined space. Owing to the flexibility of imaging fibers, the system can be used in special scenarios that are difficult for conventional experimental setups. Three experiments: open space, closed space, and underwater are designed to demonstrate the strength and weakness of the system. It is shown that when proper alignment is possible, relatively high accuracy can be achieved; the error is less than 2% of the overall height of a specimen. In situations where alignment is difficult, significantly increased error is observed. The error is in the form of gross-scale geometrical distortion; for example, flat surface is reconstructed with curvature. In addition, the imaging fibers may introduce fine-scale noise into phase measurement, which has to be suppressed by smoothing filters. Based on results and analysis, it is found that although a fiber-based system has its unique strength, existing calibration and processing methods for fringe patterns have to be modified to overcome its drawbacks so as to accommodate wider applications.
Surgical repair of the mitral valve is a difficult procedure that is often avoided in favor of less effective valve replacement because of the associated technical challenges facing non-expert surgeons. In the interest of increasing the rate of valve repair, an accurate, interactive surgical simulator for mitral valve repair was developed. With a haptic interface, users can interact with a mechanical model during simulation to aid in the development of a surgical plan and then virtually implement the procedure to assess its efficacy. Sub-millimeter accuracy was achieved in a validation study, and the system was successfully used by a cardiac surgeon to repair three virtual pathological valves.
Aortic valve reconstruction using leaflet grafts made from autologous pericardium is an effective surgical treatment for some forms of aortic regurgitation. Despite favorable outcomes in the hands of skilled surgeons, the procedure is underutilized because of the difficulty of sizing grafts to effectively seal with the native leaflets. Difficulty is largely due to the complex geometry and function of the valve and the lower distensibility of the graft material relative to native leaflet tissue. We used a structural finite element model to explore how a pericardial leaflet graft of various sizes interacts with two native leaflets when the valve is closed and loaded. Native leaflets and pericardium are described by anisotropic, hyperelastic constitutive laws, and we model all three leaflets explicitly and resolve leaflet contact in order to simulate repair strategies that are asymmetrical with respect to valve geometry and leaflet properties. We ran simulations with pericardial leaflet grafts of various widths (increase of 0%, 7%, 14%, 21% and 27%) and heights (increase of 0%, 13%, 27% and 40%) relative to the native leaflets. Effectiveness of valve closure was quantified based on the overlap between coapting leaflets. Results showed that graft width and height must both be increased to achieve proper valve closure, and that a graft 21% wider and 27% higher than the native leaflet creates a seal similar to a valve with three normal leaflets. Experimental validation in excised porcine aortas (n=9) corroborates the results of simulations.
finite element; aortic valve repair; membrane; surgical planning; leaflet graft; pericardium
Registration of three-dimensional ultrasound (3DUS) volumes is necessary in several applications, such as when stitching volumes to expand the field of view or when stabilizing a temporal sequence of volumes to cancel out motion of the probe or anatomy. Current systems that register 3DUS volumes either use external tracking systems (electromagnetic or optical), which add expense and impose limitations on acquisitions, or are image-based methods that operate offline and are incapable of providing immediate feedback to clinicians. This paper presents a real-time image-based algorithm for rigid registration of 3DUS volumes designed for acquisitions in which small probe displacements occur between frames. Described is a method for feature detection and descriptor formation that takes into account the characteristics of 3DUS imaging. Volumes are registered by determining a correspondence between these features. A global set of features is maintained and integrated into the registration, which limits the accumulation of registration error. The system operates in real-time (i.e. volumes are registered as fast or faster than they are acquired) by using an accelerated framework on a graphics processing unit. The algorithm’s parameter selection and performance is analyzed and validated in studies which use both water tank and clinical images. The resulting registration accuracy is comparable to similar feature-based registration methods, but in contrast to these methods, can register 3DUS volumes in real-time.
ultrasound; registration; real-time; image-based
Measurement of the shape and motion of the mitral valve annulus has proven useful in a number of applications, including pathology diagnosis and mitral valve modeling. Current methods to delineate the annulus from four-dimensional (4D) ultrasound, however, either require extensive overhead or user-interaction, become inaccurate as they accumulate tracking error, or they do not account for annular shape or motion. This paper presents a new 4D annulus segmentation method to account for these deficiencies. The method builds on a previously published three-dimensional (3D) annulus segmentation algorithm that accurately and robustly segments the mitral annulus in a frame with a closed valve. In the 4D method, a valve state predictor determines when the valve is closed. Subsequently, the 3D annulus segmentation algorithm finds the annulus in those frames. For frames with an open valve, a constrained optical flow algorithm is used to the track the annulus. The only inputs to the algorithm are the selection of one frame with a closed valve and one user-specified point near the valve, neither of which needs to be precise. The accuracy of the tracking method is shown by comparing the tracking results to manual segmentations made by a group of experts, where an average RMS difference of 1.67 ± 0.63 mm was found across 30 tracked frames.
Mitral valve; Annulus; Tracking; Segmentation; Ultrasound
Force sensors provide critical information for robot manipulators, manufacturing processes, and haptic interfaces. Commercial force sensors, however, are generally not adapted to specific system requirements, resulting in sensors with excess size, cost, and fragility. To overcome these issues, 3D printers can be used to create components for the quick and inexpensive development of force sensors. Limitations of this rapid prototyping technology, however, require specialized design principles. In this paper, we discuss techniques for rapidly developing simple force sensors, including selecting and attaching metal flexures, using inexpensive and simple displacement transducers, and 3D printing features to aid in assembly. These design methods are illustrated through the design and fabrication of a miniature force sensor for the tip of a robotic catheter system. The resulting force sensor prototype can measure forces with an accuracy of as low as 2% of the 10 N measurement range.
Force Sensors; Rapid Prototyping; Sensor Design
Robotic catheters have the potential to revolutionize cardiac surgery by enabling minimally invasive structural repairs within the beating heart. This paper presents an actuated catheter system that compensates for the fast motion of cardiac tissue using 3D ultrasound image guidance. We describe the design and operation of the mechanical drive system and catheter module and analyze the catheter performance limitations of friction and backlash in detail. To mitigate these limitations, we propose and evaluate mechanical and control system compensation methods, including inverse and model-based backlash compensation, to improve the system performance. Finally, in vivo results are presented that demonstrate that the catheter can track the cardiac tissue motion with less than 1 mm RMS error. The ultimate goal of this research is to create a fast and dexterous robotic catheter system that can perform surgery on the delicate structures inside of the beating heart.
Medical Robots; Motion Compensation; Robotic Catheters; Heart Valves
Heart valves are functionally complex, making surgical repair difficult. Simulation-based surgical planning could facilitate repair, but current finite element (FE) studies are prohibitively slow for rapid, clinically oriented simulations. Mass-spring (M-S) models are fast but can be inaccurate. We quantify speed and accuracy differences between an anisotropic, nonlinear M-S and an efficient FE membrane model for simulating both biaxial and pressure loading of aortic valve (AV) leaflets. The FE model incurs approximately 10 times the computational cost of the M-S model. For simulated biaxial loading, mean error in normal strains is <1% for both FE and M-S models for equibiaxial loading but increases for non-equibiaxial states for the M-S model (7%). The M-S model was less able to simulate shear behavior, with mean strain error of approximately 80%. For pressurized AV leaflets, the M-S model predicts similar leaflet dimensions to the FE model (within 2.6%), and the coaptation zone is similar between models. The M-S model simulates in-plane behavior of AV leaflets considerably faster than the FE model and with only minor differences in the deformed mesh. While the M-S model does not allow explicit control of shear response, shear does not strongly influence shape of the simulated AV under pressure.
Finite element model; Aortic valve; Membrane; Surgical planning
Properly selected port sites for robot-assisted coronary artery bypass graft (CABG) improve the efficiency and quality of these procedures. In clinical practice, surgeons select port locations using external anatomic landmarks to estimate a patient’s internal anatomy. This paper proposes an automated approach to port selection based on a preoperative image of the patient, thus avoiding the need to estimate internal anatomy. Using this image as input, port sites are chosen from a grid of surgeon-approved options by defining a performance measure for each possible port triad. This measure seeks to minimize the weighted squared deviation of the instrument and endoscope angles from their optimal orientations at each internal surgical site. This performance measure proves insensitive to perturbations in both its weighting factors and moderate intraoperative displacements of the patient’s internal anatomy. A validation study of this port site selection was performed. cardiac algorithm also Six surgeons dissected model vessels using the port triad selected by this algorithm with performance compared to dissection using a surgeon-selected port triad and a port triad template described by Tabaie et al., 1999. With the algorithm-selected ports, dissection speed increased by up to 43% (p = 0.046) with less overall vessel trauma. Thus, this algorithmic approach to port site selection has important clinical implications for robot-assisted CABG which warrant further investigation.
Medical robotics; port placement; teleoperation
Intra-cardiac 3D ultrasound imaging has enabled new minimally invasive procedures. Its narrow field of view, however, limits its efficacy in guiding beating heart procedures where geometrically complex and spatially extended moving anatomic structures are often involved. In this paper, we present a system that performs electrocardiograph gated 4D mosaicing and visualization of 3DUS volumes. Real-time operation is enabled by GPU implementation. The method is validated on phantom and porcine heart data.
3D ultrasound; electromagnetic tracking; graphic processing unit; volume mosaicing; volume registration
Segmenting the mitral valve during closure and throughout a cardiac cycle from four dimensional ultrasound (4DUS) is important for creation and validation of mechanical models and for improved visualization and understanding of mitral valve behavior. Current methods of segmenting the valve from 4DUS either require extensive user interaction and initialization, do not maintain the valve geometry across a cardiac cycle, or are incapable of producing a detailed coaptation line and surface. We present a method of segmenting the mitral valve annulus and leaflets from 4DUS such that a detailed, patient-specific annulus and leaflets are tracked throughout mitral valve closure, resulting in a detailed coaptation region. The method requires only the selection of two frames from a sequence indicating the start and end of valve closure and a single point near a closed valve. The annulus and leaflets are first found through direct segmentation in the appropriate frames and then by tracking the known geometry to the remaining frames. We compared the automatically segmented meshes to expert manual tracings for both a normal and diseased mitral valve, and found an average difference of 0.59 ± 0.49 mm, which is on the order of the spatial resolution of the ultrasound volumes (0.5–1.0 mm/voxel).
An accurate and reproducible segmentation of the mitral valve annulus from 3D ultrasound is useful to clinicians and researchers in applications such as pathology diagnosis and mitral valve modeling. Current segmentation methods, however, are based on 2D information, resulting in inaccuracies and a lack of spatial coherence. We present a segmentation algorithm which, given a single user-specified point near the center of the valve, uses max-flow and active contour methods to delineate the annulus geometry in 3D. Preliminary comparisons to manual segmentations and a sensitivity study show the algorithm is both accurate and robust.
mitral; annulus; segmentation; ultrasound
Recent developments in cardiac catheter technology promise to allow physicians to perform most cardiac interventions without stopping the heart or opening the chest. However, current cardiac devices, including newly developed catheter robots, are unable to accurately track and interact with the fast moving cardiac tissue without applying potentially damaging forces. This paper examines the challenges of implementing force control on a flexible robotic catheter. In particular, catheter friction and backlash must be compensated when controlling tissue interaction forces. Force controller designs are introduced and evaluated experimentally in a number of configurations. The controllers are based on the inner position loop force control approach where the position trajectory is adjusted to achieve a desired force on the target. Friction and backlash compensation improved force tracking up to 86% with residual RMS errors of 0.11 N while following a prerecorded cardiac tissue trajectory with accelerations of up to 3800 mm/s2. This performance provides sufficient accuracy to enable a wide range of beating heart surgical procedures.
The shape of the mitral valve annulus is used in diagnostic and modeling applications, yet methods to accurately and reproducibly delineate the annulus are limited. This paper presents a mitral annulus segmentation algorithm designed for closed mitral valves which locates the annulus in three-dimensional ultrasound using only a single user-specified point near the center of the valve. The algorithm first constructs a surface at the location of the thin leaflets, and then locates the annulus by finding where the thin leaflet tissue meets the thicker heart wall. The algorithm iterates until convergence metrics are satisfied, resulting in an operator-independent mitral annulus segmentation. The accuracy of the algorithm was assessed from both a diagnostic and surgical standpoint by comparing the algorithm’s results to delineations made by a group of experts on clinical ultrasound images of the mitral valve, and to delineations made by an expert with a surgical view of the mitral annulus on excised porcine hearts using an electromagnetically tracked pointer. In the former study, the algorithm was statistically indistinguishable from the best performing expert (p = 0.85) and had an average RMS difference of 1.81 ± 0.78mm to the expert average. In the latter, the average RMS difference between the algorithm’s annulus and the electromagnetically tracked points across six hearts was 1.19 ± 0.17mm.
Mitral Valve; Annulus; Segmentation; Ultrasound; Graph Cuts
This paper presents a novel miniature uniaxial force sensor for use within a beating heart during mitral valve annuloplasty. The sensor measures 5.5 mm in diameter and 12 mm in length and provides a hollow core to pass instrumentation. A soft elastomer flexure design maintains a waterproof seal. Fiber optic transduction eliminates electrical circuitry within the heart, and acetal components minimize ultrasound-imaging artifacts. Calibration uses a nonlinear viscoelastic method, and in vitro tests demonstrate a 0–4-N force range with rms errors of 0.13 N (<3.2%). In vivo tests provide the first endocardial measurements of tissue-minimally invasive surgery instrument interaction forces in a beating heart.
Beating heart surgery; force feedback; minimally invasive surgery (MIS); mitral valve annuloplasty; optical force sensor
Beating heart intracardiac procedures promise significant benefits for patients, however, the fast motion of the heart poses serious challenges to surgeons. We present a new 3D ultrasound-guided motion (3DUS) compensation system that synchronizes instrument motion with the heart. The system utilizes the fact that the motion of some intracardiac structures, including the mitral valve annulus, is largely constrained to translation along one axis. This allows the development of a real-time 3DUS tissue tracker which we integrate with a 1 degree-of-freedom actuated surgical instrument, real-time 3DUS instrument tracker, and predictive filter to devise a system with synchronization accuracy of 1.8 mm RMSE. User studies involving the deployment of surgical anchors in a simulated mitral annuloplasty procedure demonstrate that the system increases success rates by over 100%. Furthermore, it enables more careful anchor deployment by reducing forces to the tissue by 50% while allowing instruments to remain in contact with the tissue for longer periods.
3D ultrasound imaging has enabled minimally invasive, beating heart intracardiac procedures. However, rapid heart motion poses a serious challenge to the surgeon that is compounded by significant time delays and noise in 3D ultrasound. This paper investigates the concept of using a one-degree-of-freedom motion compensation system to synchronize with tissue motions that may be approximated by 1D motion models. We characterize the motion of the mitral valve annulus and show that it is well approximated by a 1D model. The subsequent development of a motion compensation instrument (MCI) is described, as well as an extended Kalman filter (EKF) that compensates for system delays. The benefits and robustness of motion compensation are tested in user trials under a series of non-ideal tracking conditions. Results indicate that the MCI provides an approximately 50% increase in dexterity and 50% decrease in force when compared with a solid tool, but is sensitive to time delays. We demonstrate that the use of the EKF for delay compensation restores performance, even in situations of high heart rate variability. The resulting system is tested in an in vitro 3D ultrasound-guided servoing task, yielding accurate tracking (1.15 mm root mean square) in the presence of noisy, time-delayed 3D ultrasound measurements.
medical robotics; motion compensation; ultrasound
The manipulation of fast moving, delicate tissues in beating heart procedures presents a considerable challenge to surgeons. We present a new robotic force stabilization system that assists surgeons by maintaining a constant contact force with the beating heart. The system incorporates a novel, miniature uniaxial force sensor that is mounted to surgical instrumentation to measure contact forces during surgical manipulation. Using this sensor in conjunction with real-time tissue motion information derived from 3D ultrasound, we show that a force controller with feed-forward motion terms can provide safe and accurate force stabilization in an in vivo contact task against the beating mitral valve annulus. This confers a 50% reduction in force fluctuations when compared to a standard force controller and a 75% reduction in fluctuations when compared to manual attempts to maintain the same force.
We are working to develop beating-heart atrial septal defect (ASD) closure techniques using real-time 3D ultrasound guidance. The major image processing challenges are the low image quality and the processing of information at high frame rate. This paper presents comparative results for ASD tracking in time sequences of 3D volumes of cardiac ultrasound. We introduce a block flow technique, which combines the velocity computation from optical flow for an entire block with template matching. Enforcing adapted similarity constraints to both the previous and first frames ensures optimal and unique solutions. We compare the performance of the proposed algorithm with that of block matching and region-based optical flow on eight in-vivo 4D datasets acquired from porcine beating-heart procedures. Results show that our technique is more stable and has higher sensitivity than both optical flow and block matching in tracking ASDs. Computing velocity at the block level, our technique tracks ASD motion at 2 frames/s, much faster than optical flow and comparable in computation cost to block matching, and shows promise for real-time (30 frames/s). We report consistent results on clinical intra-operative images and retrieve the cardiac cycle (in ungated images) from error analysis. Quantitative results are evaluated on synthetic data with maximum tracking errors of 1 voxel.
real-time ultrasound; echocardiography; atrial septal defect; tracking; mutual information; block matching; optical flow; block flow
Real-time three-dimensional ultrasound enables new intra-cardiac surgical procedures, but the distorted appearance of instruments in ultrasound poses a challenge to surgeons. This paper presents a detection technique that identifies the position of the instrument within the ultrasound volume. The algorithm uses a form of the generalized Radon transform to search for long straight objects in the ultrasound image, a feature characteristic of instruments and not found in cardiac tissue. When combined with passive markers placed on the instrument shaft, the full position and orientation of the instrument is found in 3D space. This detection technique is amenable to rapid execution on the current generation of personal computer graphics processor units (GPU). Our GPU implementation detected a surgical instrument in 31 ms, sufficient for real-time tracking at the 25 volumes per second rate of the ultrasound machine. A water tank experiment found instrument orientation errors of 1.1 degrees and tip position errors of less than 1.8 mm. Finally, an in vivo study demonstrated successful instrument tracking inside a beating porcine heart.