PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Image Comput Comput Assist Interv. Author manuscript; available in PMC 2011 April 8.
Published in final edited form as:
Med Image Comput Comput Assist Interv. 2010; 13(Pt 3): 303–310.
PMCID: PMC3072686
NIHMSID: NIHMS282462

Micro-force Sensing in Robot Assisted Membrane Peeling for Vitreoretinal Surgery

Abstract

Vitreoretinal surgeons use 0.5mm diameter instruments to manipulate delicate tissue inside the eye while applying imperceptible forces that can cause damage to the retina. We present a system which robotically regulates user-applied forces to the tissue, to minimize the risk of retinal hemorrhage or tear during membrane peeling, a common task in vitreoretinal surgery. Our research platform is based on a cooperatively controlled microsurgery robot. It integrates a custom micro-force sensing surgical pick, which provides conventional surgical function and real time force information. We report the development of a new phantom, which is used to assess robot control, force feedback methods, and our newly implemented auditory sensory substitution to specifically assist membrane peeling. Our findings show that auditory sensory substitution decreased peeling forces in all tests, and that robotic force scaling with audio feedback is the most promising aid in reducing peeling forces and task completion time.

Keywords: Robotic Microsurgery, Vitreoretinal Surgery, Cooperative Robot Control, Force Sensing, Force Scaling, Auditory Sensory Substitution

1 Introduction

Microsurgical manipulation requires precise manual dexterity, fine visual-motor coordination, and application of forces that are well below human tactile sensation [1]. Imprecise movements during these operations are further attributed to physiological hand tremor, fatigue, poor visual and kinesthetic feedback, as well as patient movement. In vitreoretinal surgery, the common microsurgery risks and limitations are further extended by surgical inaccessibility, poor visibility, tissue fragility and the flexibility of delicate (20–25 Ga) surgical instruments.

Epiretinal membrane (ERM) peeling is a typical task where a thin membrane is carefully delaminated off the surface of the retina by grasping the membrane's edge with micro-forceps and pulling it free from the retina. Due to unstable manipulation and unknown forces applied to the tissue, the maneuver is associated with the risks of retinal hemorrhage and tearing, leading to potentially irreversible damage that results in vision loss. Surgeons manipulate the peeling tissue at very slow instrument velocities, observed to be within 0.1–0.5 mm/s range, and simultaneously visually monitor local surface deformation that may indicate undesirable forces between the surgical instrument, the membrane and the retina. A capable surgeon reacts to such cues by retracting the instrument and regrasping the tissue for an alternate approach. This task is extremely difficult to master due to nearly imperceptible visual cues, and a requirement for very precise visuomotor reflexes. Factors such as involuntary patient motion, inconsistent tissue properties, high or variable tool velocities, and changing manipulation directions can dramatically increase undesirable forces applied to the delicate retinal tissue. Actively sensing and limiting these forces has the potential to significantly improve surgical precision and diminish surgical complications.

One approach is to reduce physiological hand tremor and actively improve the surgeon's fine motion control. An example is micron, a hand held device that uses piezo actuators to counteract undesirable hand tremor and scale input motions [2]. Another approach is a use of a tele-operated robot, such as [3], to reduce hand tremor through remote control and motion scaling. The cooperatively controlled JHU Steady-hand Eye Robot [4] relies on its stiff structure and non back-drivable actuators with high resolution encoders to provide stability and precision. To our knowledge, these systems do not provide any tool-tissue interaction force feedback to the surgeon. Logically, combining hand tremor reduction with precise motion control and end-point force sensing feedback is a more complete approach.

There have been some attempts to measure tool to tissue interaction forces with instruments that had sensing elements built into the handle [5] and to incorporate sensed tool-tissue forces into cooperative control (e.g. [7,8]). Handle mounted tool force sensors are not practical for vitreoretinal surgery, because the shaft of the instrument is inserted through the sclera. The friction between the tool and the trocar, and lateral forces from the transacting sclera with the tool can significantly attenuate or distort the propagation of the forces to the tissues inside of the eye. This limitation may be addressed by incorporating force sensing elements into the shaft section of the instrument that is typically located inside the eye [6].

In this paper we investigate and compare micro-force sensing feedback to specifically assist in membrane peeling, for which we developed a novel phantom that approximates peeling forces encountered in retinal surgery. We summarize our microsurgical research platform integrating a 2-DOF force-sensing vitreoretinal surgery instrument and the new cooperative control robot with a remote center-of-motion mechanism (RCM). In addition to the linear “force scaling” cooperative control in [7,8], we introduce a velocity limiting force-cooperative control mode. We also present our new task-specific auditory sensory substitution and assess its effects on task performance, both with freehand instruments and with our various robot cooperative control modes.

2 Experimental Platform

2.1 Robotic Assistant

Our cooperatively controlled “steady hand” robotic assistant is a 5-DOF system (Fig. 1A) similar to [4] and designed as a development platform for microsurgery research. It is an admittance-type system, collaboratively providing steady-hand motion by inherently filtering physiological hand tremor and low-frequency drift found in surgeons’ hand movements during microsurgery [9]. A 6-DOF force/torque sensor mounted at the tool holder senses forces exerted by the surgeon on the tool, for use as command inputs to the robot. This design improves on similar work described in [4] by increasing the range of motion, including a custom mechanical RCM and improving the stiffness and precision of the stages. The RCM mechanism improves the general stability of the system by reducing range of motion and velocities in the Cartesian stages when operating in virtual RCM mode [10], which constrains the tool axis to always intersect the sclerotomy opening on the eye. This is a necessary safety feature to minimize undesirable translations of the eye and also simplifies software control. The actuator motion controller servo loop frequency is 4 kHz, while the robot control loop operates at 400 Hz. A more detailed description is in [11].

Fig. 1
A) Robot with RCM mechanism [11]; B) Force Sensor Instrument Concept [6]; C) Peeling sample and hooked force sensor instrument; D) Experimental setup.

2.2 Micro-force Sensing Instrument

Vitreoretinal microsurgical applications introduce certain limitations on the exact choice of force sensor by demanding sub-mN accuracy required to sense forces that are routinely less than 7.5 mN [1]. A miniature instrument size is necessary to be inserted through a 25 Ga sclerotomy opening and the force sensor must be able to obtain measurements at the instrument’s tip, below the sclera.

A tool with integrated fiber Bragg grating (FBG) sensors was manufactured for this purpose, following the design [6] of Iordachita et al. FBGs are robust optical sensors capable of detecting changes in strain, without interference from electrostatic, electromagnetic or radio frequency sources. Three optical fibers are placed along the tool shaft (Fig. 1B) and by measuring the bending of the tool they allow for calculation of the force in the transverse plane with a sensitivity of 0.25 mN. The tip of the tool is a simple hook, and the tool is mounted in the robot tool holder in a calibrated orientation relative to the robot. The sensor data was collected and processed at 2 kHz and transmitted over TCP/IP.

2.3 Membrane Peeling Phantom

To develop and assess the performance of control and auditory feedback methods, we required a consistent and easily fabricated phantom model, behaving within the parameters of vitreoretinal surgery. The actual peeling procedure involves grasping or hooking a tissue layer and slowly delaminating it, often in a circular pattern. To reduce the factors needed to be controlled, we simplified the target maneuver to a straight line peel using a hooked instrument.

Video recordings of actual epiretinal membrane peeling procedures were analyzed and tool velocities during membrane peeling were found to be 0.1–0.5 mm/s. Gupta et al. found that retinal tissue manipulation forces are likely to be below 7.5 mN [1], while Jagtap et al. found them to be comparable but slightly higher [12]. With these values in mind, after extensive search and trial-and-error testing of many materials, we identified that sticky tabs from 19 mm Clear Bandages (RiteAid brand) to be a suitable and repeatable phantom for delaminating. The tab was sliced to produce 2 mm wide strips (Fig. 1C) that can be peeled multiple times from its backing, with predictable behavior showing increase of peeling force with increased peeling velocity. The plastic peeling layer is very flexible but strong enough to withstand breaking pressures at the hook attachment site. A 10 mm section of the strip is considered, requiring 20 mm of tool travel to complete a peel. Fig. 2 shows the forces observed at various velocities.

Fig. 2
Peeling sample repeatability tests.

3 Methods

We have employed three cooperative control methods which modulate the behavior of the robot based on operator input and/or tool tip forces. The control method parameters considered handle input force range (0–5 N), and peeling task forces and velocities. Audio sensory substitution serves as a surrogate or complementary form of feedback and provides high resolution real-time tool tip force information.

Proportional Velocity Control (PV) paradigm is described in [7], where the velocity at the tool (V) is proportional to the user’s input force at the handle (Fh). A gain of α = 1 was used, which translates handle input force of 1 N to 1 mm/s tool velocity.

equation M1
(1)

Linear Force Scaling Control (FS) maps, or amplifies, the human-imperceptible forces sensed at the tool tip (Ft) to handle interaction forces by modulating robot velocity. Prior applications used γ = 25 and γ = 62.5 scale factors [7, 8] which are low for the range of operating parameters in vitreoretinal peeling. Scaling factor of γ = 500 was chosen to map the 0–10 mN manipulation forces at the tool tip to input forces of 0–5 N at the handle.

equation M2
(2)

Proportional Velocity Control with Limits (VL) increases maneuverability when low tip forces are present. The method uses PV control but with an additional velocity constraint that is inversely proportional to the tip force. With such scaling, the robot response becomes very sluggish with higher tool tip forces, effectively dampening manipulation velocities. The constraint parameters were chosen empirically to be m = −180 and b = 0.9. To avoid zero crossing instability, forces lower than f1 = 1 mN in magnitude do not limit the velocity. Likewise, to provide some control to the operator when tip forces are above a high threshold (f2 = 7.5 mN), a velocity limit (ν2 = 0.1) is enforced.

equation M3
(3)

Where Vlim(Ft), is the function in Fig. 3A.

Fig. 3
A) Velocity limiting function (symmetric about V = −F); B) Audio feedback zones.

Force-to-Auditory Sensory Substitution

In current practice, surgeons indirectly assess the relative stress applied to tissue via visual interpretation of changing light reflections from deforming tissue. This type of “visual sensory substitution” requires significant experience and concentration, common only to the expert surgeons. To provide more clear and objective feedback, we measure these forces directly and convey them to the surgeon in real time with auditory representation. Kitagawa et al. showed that auditory feedback representing force in a complex surgical task improves robot assisted performance and suggests that continuous and real-time feedback is more effective than discrete, single event information [13]. We designed auditory feedback that modulates the playback tempo of audio “beeps” in three force level zones which represent force operating ranges chosen to be relevant in typical vitreoretinal operations. The audio is silent until 1 mN or greater force is measured. A constant slow beeping is emitted until 3.5 mN, which is designated to be a “safe” force operating zone. A “cautious zone” was designated as 3.5–7 mN, and has a proportionally increasing tempo followed by a “danger zone” that generates a constant high tempo beeping.

Experiments

Our objective is to compare the effectiveness of the above methods in decreasing mean and maximum peeling forces while minimizing time taken to complete the task. A single subject was tested in this preliminary experiment, which was configured in the following ways. The phantom was adhered to a stable platform with double-stick tape and the robot was positioned so the hook is ~1.5 mm above the peeling surface. The orientation of the handle was perpendicular to the peeling direction and comfortable to the operator. To eliminate force cues from tool bending, the visibility of the tool shaft was obstructed with the exception of the tool tip (Fig. 2D). The test subject was trained extensively (~3 hours) prior to the trials. Five minute breaks were allowed between trials. The operator was directed to peel the membrane steadily and as slow as possible without stopping. To simplifying the experiments we limited the robot motion to Cartesian translations only; experiments showed no noticeable difference between trials with and without rotational DOFs. No visual magnification was provided to the operator. For all trials we used the same sample and, for consistency, we have tested the behavior of the sample before and after the experiment. For comparison, we have included freehand peeling tests where the operator peeled the sample without robot assistance. Five trials of each method were performed with audio feedback, and five without for the following: Freehand (FH/FHA), Proportional Velocity Control (PV/PVA), Linear Force Scaling Control (FS/FSA), Velocity Limiting Control (VL/VLA).

4 Results

In every method tested, audio feedback decreased the maximum tip forces, as well as tip force variability (Fig. 4). It significantly increased the task completion time for freehand and proportional velocity control trials while the time decreased slightly for the others. The operator was naturally inclined to “hover” around the discrete audio transition point corresponding to 3.5 mN, which was observed in all cases except freehand. This was particularly prominent in force scaling, where the operator appears to rely on audio cues over haptic feedback (see Fig. 5C, time 60–80 s). In velocity limiting trials, audio reduced mean input handle forces by 50% without compromising performance. This indicates that the user consciously attempted to use audio feedback to reduce the forces applied to the sample.

Fig. 4
Table of tip force results from all trials.
Fig. 5
Plots of representative trials of each mode showing tip forces, with/out audio feedback.

Freehand (Fig. 5A) trials showed considerable high force variation due to physiological hand tremor. The mean force applied was around 5 mN, with maximum near 8 mN. Audio feedback helped to reduce large forces but significantly increased task completion time.

Proportional Velocity (Fig. 5B) control performance benefited from the stability of robot assistance and resulted in a smoother force application, while the range of forces was comparable to freehand tests. Likewise, audio feedback caused a decrease in large forces but increased time to complete the task.

Force Scaling (Fig. 5C) control yielded the best overall performance in terms of mean forces with and without audio. Although, the average time to completion was the longest, except for freehand with audio.

Velocity Limiting (Fig. 5D) control resulted in a very smooth response except for the section that required higher absolute peeling forces at the limited velocity. This had an effect of contouring “along” a virtual constraint. Due to matching thresholds, audio had very little effect on the performance.

5 Discussion and Conclusions

Robotic assistance combining a cooperatively controlled manipulation with real time tool-to-tissue force sensing has significant potential to improve surgical practice, especially when combined with audio sensory substitution. We have shown that our experimental system is capable of measuring and reacting to forces under 7.5 mN, a common range in microsurgery. We have found that force scaling with audio feedback provides the most intuitive response and force-reducing performance in a simulated membrane peeling task, where the goal is to apply low and steady forces to generate a controlled delamination.

We designed robot control and audio feedback parameters specifically for our membrane peeling phantom which, according to our expert surgeon co-authors, is a satisfactory surrogate for the target task. The feedback parameters can also be tuned in real-time to accommodate for operator preference and optimize the system for other microsurgical tasks. In our experiment, the force scaling yielded the best results, however the handle input forces were high (~2.5 N) enough to cause fatigue and decreased precision due to prolonged stress. Lowering the force scaling gain will lower the handle input forces, but this may compromise sensitivity since the human finger has a force sensing resolution of 0.5 N [14].

Preliminary user feedback has indicated that auditory sensory substitution can be very useful, especially when combined with cooperative robot control. Our surgeon coauthors believe that continuous audio feedback may be disruptive or overwhelming in already noisy operating room, but occasional and unique sounding feedback during critical moments can be very valuable. During operator training, we have observed significant improvement in task completion rates and decrease in force variation stemming from audio feedback alone. This concept has potential to be a very useful training tool in providing immediate and objective tissue manipulation force information which can be correlated with other cues.

The results from this single-user pilot study are encouraging and provide the basis for a planned multi-user study. Other next steps involve characterizing in-vivo membrane peeling forces and tool trajectories to verify and improve our artificial phantom, and to optimize our robot control parameters for in-vivo system assessment. We are also planning a human factors study to identify intuitive, effective and operating room compatible auditory sensory substitution methods. Finally, to cover a variety of peeling approaches and force application directions, we plan to build a microsurgical forceps that measures axial force in addition to the two axis transverse forces used in this work.

Footnotes

1Supported in part by the NSF EEC9731748, NIH 1R01 EB007969-01 and ARCS Foundation

*The original publication is available at springerlink.com at the following address: *http://www.springerlink.com/content/ym13k12478596100/

References

1. Gupta PK, Jensen PS, d Juan E. Surgical forces and tactile perception during retinal microsurgery. MICCAI. 99:1218–1225.
2. Riviere C, Ang WT, Khosla P. Toward active tremor canceling in handheld microsurgical instruments. IEEE Tr. Rob. & Aut. 2003 Oct.vol. 19(5):793–800.
3. Nakano T, Sugita N, Ueta T, Tamaki Y, Mitsuishi M. A parallel robot to assist vitreoretinal surgery. JCARS. 2009 [PubMed]
4. Mitchell B, Koo J, Iordachita M, Kazanzides P, Kapoor A, Handa J, Hager G, Taylor R. Development and application of a new steady-hand manipulator for retinal surgery. IEEE ICRA. 2007:623–629.
5. Berkelman P, Whitcomb L, Taylor R, Jensen P. A miniature microsurgical instrument tip force sensor for enhanced force feedback during robot-assisted manipulation. IEEE TRA. 2003 Oct.19(5):917–921.
6. Iordachita I, Sun Z, Balicki M, Kang J, Phee S, Handa J, Gehlbach P, Taylor R. A sub-millimetric, 0.25 mN resolution fully integrated fiber-optic force-sensing tool for retinal microsurgery. JCARS. 2009;4(4):383–390. 06. [PMC free article] [PubMed]
7. Kumar R, Berkelman P, Gupta P, Barnes A, Jensen PS, Whitcomb LL, Taylor RH. Preliminary Experiments in Cooperative Human/Robert Force Control for Robot Assisted Microsurgical Manipulation. IEEE ICRA. 2000;1:610–617.
8. Berkelman P, Rothbaum D, Roy J, Lang S, Whitcomb L, Hager G, Jensen P, d Juan E, Taylor R, Niparko J. Performance “Evaluation of a Cooperative Manipulation Microsurgical Assistant Robot Applied to Stapedotomy” LNCS ISSU. 2001;2208:1426–1429.
9. Riviere C, Rader R, Khosla P. Characteristics of hand motion of eye surgeons. IEEE EMBS. 1997;4:1690–1693.
10. Kapoor A, Li M, Taylor RH. Constrained control for surgical assistant robots. IEEE ICRA. 2006:231–236.
11. Uneri A, Balicki M, Handa J, Gehlbach P, Taylor R, Iordachita I. New Steady-Hand Eye Robot with Microforce Sensing for Vitreoretinal Surgery Research. IEEE BioRob. 2010 [PMC free article] [PubMed]
12. Jagtap AS, Riviere CN. Applied force during vitreoretinal microsurgery with handheld instruments. Conf Proc IEEE Eng Med Biol Soc. 2004;4:2771–2773. [PubMed]
13. Kitagawa M, Dokko D, Okamura A, Yuh D. Effect of sensory substitution on suture manipulation forces for robotic surgical systems. JTCS. 2005;129(1):151–158. [PubMed]
14. Taylor CL. The biomechanics of the normal and of the amputated upper extremities (Ch. 7) In: Klopsteg PE, Wilson PD, editors. Human limbs and their substitutes. New York, NY.: Hafner Pub. Co.; 1968. pp. 169–221.