|Home | About | Journals | Submit | Contact Us | Français|
This paper presents a framework to build medical training applications by using virtual reality and a tool that helps the class instantiation of this framework. The main purpose is to make easier the building of virtual reality applications in the medical training area, considering systems to simulate biopsy exams and make available deformation, collision detection, and stereoscopy functionalities. The instantiation of the classes allows quick implementation of the tools for such a purpose, thus reducing errors and offering low cost due to the use of open source tools. Using the instantiation tool, the process of building applications is fast and easy. Therefore, computer programmers can obtain an initial application and adapt it to their needs. This tool allows the user to include, delete, and edit parameters in the functionalities chosen as well as storing these parameters for future use. In order to verify the efficiency of the framework, some case studies are presented.
Virtual reality (VR) is a very interesting technology in the development of tools for medical training because it offers interaction in a virtual environment (VE) using mouse and keyboard or non-conventional devices, such as gloves, head mounted displays, joysticks, and haptic devices with force feedback.1 VR can provide visualizing the details of the procedure in a VE or in a virtual patient. Additionally, it can give feedback to the surgeon about his performance. Computer-generated three-dimensional (3D) environments provide a more advanced human–computer interaction and allow navigation, selection, and manipulation of objects in the virtual world as well as in the real world.2
Computing and communication technologies are widely used in the medical area, and VR has attracted interest, since it raises the possibilities of studies and practice of several techniques and medical procedures.1 The use of VR in medical applications can ensure substituting test objects (like guinea pigs and corpses) for objects modeled to simulate human organs and tissues in the VE. Simulation-based training using VR techniques is a promising alternative to the traditional training of minimally invasive surgeries (MIS). Simulators let the trainee touch, feel, and manipulate virtual tissues and organs using a virtual tool similar to the tool used in actual MIS. The VE allows verifying interactions between the virtual tool and virtual tissues on a monitor as in real laparoscopic procedures.3
VR applications in medicine must have special characteristics and requirements, as the use of haptic devices, development of 3D objects to simulate human organs and equipments used in the procedures, collision detection, stereoscopy, and deformation techniques, in addition to real-time feedback and accuracy.4 In order to make the applications more realistic, it is necessary to plan the VE to reproduce real procedure aspects and actions considering these mentioned aspects. To avoid the implementation of these aspects in every built application, the idea of the source code reuse can be an advantage.4
There are VR applications for teaching, rehabilitation, medical procedures training, and simulation. Good examples provided by the literature are simulators for training in minimally invasive surgery,3 orthopedic simulator,5 prostate cancer diagnosis,6 breast plastic surgery planning,7 and telerehabilitation.8
An object-oriented framework (OOF) is a reuse technique with the object-oriented paradigm. In this context, it can describe a framework as a structure of classes that makes available and originates a non-finished application. These classes can generate a set of applications of a previously established domain. Besides the reusability in the analysis, project, programming, and tests stages, an easier maintenance and a rise in productivity are the advantages of using frameworks to develop applications.9
This paper presents the development of the virtual medical training (ViMeT), an OOF that uses VR to simulate medical training, considering biopsy exams as the case study. It also presents the ViMeTWizard, a tool to assist the ViMeT instantiation, and the ViMeT integration with other projects. It demonstrates an experience report of the development process and the instantiation of the framework, thereby contributing to researchers who wish to increase the productivity in the building process of the computer applications in this area.
Besides this introduction, this paper has the following sections: “OBJECT-ORIENTED FRAMEWORKS AND VR FRAMEWORKS” shows the concept of the OOF, some of their classifications, and some examples of VR frameworks. “THE VIMET FRAMEWORK” describes the ViMeT’s project, the methodology used in its development, and the ViMeTWizard tool. “CASE STUDIES AND DISCUSSION” shows some case studies resulting from the framework instantiation, including a discussion about them. “INTEGRATION OF THE VIMET WITH OTHER PROJECTS” shows the ViMeT’s integration with a module of interaction and a system for simulating 3D objects, while the last section presents the conclusions and future works.
The development of OOF and applications derived from a specific framework has been a research subject since the 1980s, with the rise of the object-oriented paradigm. Reusability through framework occurs by inheritance and polymorphism, and as in every software reuse, it presents some limitations. In order to overcome these limitations, it is possible to use simultaneously some forms of reuse such as frameworks with design patterns10 or frameworks with components.
The way in which the reuse occurs through framework generates a classification according to its extension, and three categories are proposed: black-box reuse, which occurs by composition; white-box reuse, which occurs by inheritance; and gray-box reuse, which has concrete and abstract classes, occurring by inheritance and dynamic linking.9
In the last years, an increasing interest in OOF has been observed due to its flexibility and extensibility. The building of a framework can save time in the development of new applications, as well as provide significant reduction of errors and, consequently, increase software quality. The main desired characteristic when developing a framework is the generalization related to concepts and functionalities of a previously established domain.9
VR deals with computer-generated 3D environments and a set of tools that allows the user to immerse, navigate, and interact with objects in the VE.11 For this interaction, conventional devices are used, such as keyboard and mouse, or more complex ones, such as data gloves, haptic device, and stereoscopic glasses. They provide a more realistic interaction in some applications.
VR frameworks are known for their complexity, instantiation difficulty, and for being used only by researchers or expert programmers. They allow the user to focus on the application development, since it is not necessary to worry about the VR system management. Such frameworks offer device and projection system abstraction, specific Scene Graphs, heuristic interaction with the VE, support to distributed systems, and distributed rendering.1
Some examples of VR frameworks are the following: the Avango, which allows the creation of applications with specific classes that inherit properties of distribution;12 shares simple virtual environment (SSVE), projected to manage small collaborative groups inside highly dynamic, shared and interactive VEs;13 Ivory, which provides easy visualization of Physics-based data;14 basho, used to create a VE which supports different renderings and also has a small shell;15 and simulation open framework architecture (SOFA), which consists of a structure of classes designed to achieve medical simulation in real time.16
As stated before, VR can be helpful to the medical area with regard to applications of computer-aided diagnosis, procedures simulation, 3D visualization, and teaching and training of students and professionals of the medical field. Some researchers previously conducted implementation results in useful functionalities to the medical training such as accurate collision detection, stereoscopy, and deformation.1 However, it is well known that there are a large variety of techniques of deformation, collision detection, and stereoscopy. Each one can be more or less appropriate to VR applications in medicine. The ViMeT was built in order to consider this context, taking advantage of the better characteristics of the techniques, providing a flexible structure of classes to build applications. In this way, ViMeT consists of an OOF that applies VR techniques considering the ViMeT as a domain and focusing especially on the development of applications that simulate biopsy exams.4
The ViMeT framework has computer programmers as its target audience, but the applications generated are from professionals from the health care area (mainly physicians and students) as their target audience. In order to assure the code and functionality access, it is currently being developed under the General Public License,17 and the full package will be available in http://each.uspnet.usp.br/lapis.
A biopsy exam is a procedure meant to extract small samples of organ tissue suspected of anomalies. These parts are sent to pathologists to determine the diagnosis. The simulation of such exams requires common tasks related to manipulation of equipment and of the human body parts, as well as tasks related to the necessary feedback for simulating the reaction of an action by the user, such as to deform any given area of the synthetic object.
Open source technologies in the ViMeT development was used because it implies that several researchers can contribute to its continuity. The technologies used in ViMeT were Java Programming Language18 and the API Java3D,19 database manager system (DBMS) Derby,20 an educational version of 3D StudioMax for modeling,21 and Jude22 for documentation.
Some methodologies are cited in the literature for the development of frameworks. In order to contextualize the methods, the most used are presented here. The methodology proposed by Schmidt23 affirms that the most important thing is not to initiate the project of a framework trying to modulate its variability and flexibility. Instead, a fixed application should be designed inside the frameworks’ domain, and after full comprehension, a generalization can be initialized.
Pree24 proposes a methodology in which the building begins with a definition of the specific object model of an application, and the other activities are successively repeated until the framework is considered satisfactory. In this methodology, a developer and a specialist are required isn the domain of the application and both will be co-responsible for the whole process.
Roberts and Johnson’s methodology25 is called “Evolution of Frameworks” and is related to building the instantiation. The general rule is to build the first application, build the second one quickly and different from the first one, and finally to build the third application differently from the others, but all of them must be inside the same domain; thus, the common abstractions will remain evident.
Bosch26 proposes a methodology in which the building initiates with a domain analysis and six activities. This last methodology was chosen as the defined steps for the building (Fig. 1) were considered the most suitable to the interests of the project. Some adjustments were performed, including the development of a tool for instantiation. The following paragraphs describe the steps:
The ViMeTWizard is a tool developed to make the ViMeT instantiation easier. The interface was developed with guides in order to facilitate the visualization of the functionalities and their parameters.
Using the interface, the user can select the objects that represent human organs and medical instruments, define the objects’ characteristics related to the scale, translation, and rotation, as well as to define which functionalities will compose the application.
After choosing the characteristics and the objects, the system creates the VE, generates and compiles the source code, thus creating a final application. Every selected characteristic is stored in the database for future changes or review. From the available source code, the user can include certain particular characteristics, hence generating a derived application. Figure Figure66 shows the development of an application and presents the “Loader” guide, in which the user can choose the objects or load an existing application and then modify parameters, such as scale, translation, and rotation.
The development of the ViMeTWizard demanded the use of a DBMS to store the data of the modeled objects and generated applications. In order to facilitate the instantiation of the ViMeT classes, the DBMS should be simple; thus; the Derby was chosen because it constitutes a simple Java API. Besides, it is an open source software, which allows developing a low-cost application and source code access.
The database has two tables for data persistence. The first one, named OBJECTS, stores data of the 3D modeled objects, which can be representations of human organs (for instance: breast, leg, arm, and others) or medical instruments used in training (needle, syringe, scalpel, among others). The second one, named APPLICATION, is used to store data of the applications generated for later maintenance.
The expected results of a framework are the reuse of classes in order to generate new applications, simple development, in addition to guaranteeing that the applications comply with the requirements defined in the domain analysis. As aforementioned, there are two ways of using the ViMeT: direct instantiation of the classes and instantiation through the ViMeTWizard. The “white-box” instantiation9 was chosen for the manual use and the “gray-box”9 for instantiation using the ViMeTWizard. The manual instantiation uses the ViMeT classes straightforwardly, and the developer can implement new functionalities, as deemed necessary. In the gray-box manner, the instantiation is automatic because the ViMeTWizard executes it. In order to verify the results, five virtual objects were selected: two referring to medical equipment (a needle and a syringe) and three representing human organs (a breast, a leg, and a buttock) as determined in the domain analysis.
Combinations were performed among these objects, with a variation of size and localization parameters in the VE, selection of functionalities, and parameters inherent to the each developed functionality. It is important to point out that this combination and the parameters variation generate different applications.
Figure Figure77 shows examples of applications generated using the ViMeTWizard tool. Figure Figure7a7a presents an application with an object simulating one breast and a thin-needle syringe, including the functionalities of deformation, collision detection, and stereoscopy. Figure Figure7b7b shows an application with an object simulating the buttocks and using the same syringe of the previous case, and the functionalities used were deformation and collision detection, while Figure 7c shows an object simulating a leg with a larger number of vertices and edges. The functionalities used were deformation and collision detection.
Figure Figure88 presents an application generated directly from the manual instantiation of the ViMeT classes; it used an object simulating a breast and another one simulating the medical instrument, considering the functionalities of deformation and collision detection.
From the ViMeT development and the obtained results, some observations related to the faced difficulties, solutions, and contributions of ViMeT are pointed out. First, it was possible to notice that the methodology used25 did not foresee the use of finished applications. The difficulties found for this point were regarding the standardization of the loader methods, the interaction and interface of each one of the three reused applications, and the lack of documentation. Therefore, we realized that a previous planning should be conducted when the team intends to reuse source code from previously developed applications. This is necessary because different programmers use different ways of writing and documenting their codes. If a standardization is not established for all applications, substantial effort to adapt the code to the framework is necessary, as was the case for the ViMeT project.
In the VR area, particularly, it is interesting to have a standardization in the definition of the VE and in the building of the scene graph, using the same methods to import and create the interaction with the virtual objects. Regarding the documentation, it is extremely important to build it in a clear and objective manner, without ambiguities, and preferably using tools that assist in the standardization. Using the Java language—the case of the present project—a good standardization is offered when javadoc is used because it is already well established in the users’ community of this language.
Another interesting difficulty is the definition of technical questions, like the choice of the best type of VE, the best method to load objects, as well as the best ways for the VE interaction. An in-depth research of these questions and solutions were conducted with the Java language as background. The choice should consider flexibility features and simple integration. Then, we opted for the VirtualUniverse for the VE, Object File for the loader, and Sensor Class for the interaction. With these definitions, the ViMeT provides user flexibility, without which the developer has to worry about these questions related to the basic environment of creation.
The used methodology is very suitable to assist the mentioned standardization process and the correction of errors because it allows for adaptations in its stages. Therefore, it offers the possibility of changing the project during the implementation, but it has a testing stage, parallel to the entire process that enables discovering and correcting errors easily, in the course of any development phase.
Another important issue is that an object orientation allows the development of frameworks in the VR area, making possible faster applications inside a specific domain, hence providing better quality. ViMeT was built in order to act as a standard for the development of new applications. From now on new functionalities must be implemented as part of a bigger project rather than isolated applications. The current results allow future developers to focus only on the implementation of new functionalities once the ViMeT already provides the interface and the VE. Thus, the time spent with the development of future applications can be better used in the specific objectives of each application.
It is important to point out that ViMeTWizard facilitated the ViMeT instantiation for the alterations of the parameters, showing that the manipulation of these parameters in the database is simple and does not imply in performance loss. The generated source code can be adapted by developers accustomed to using the Java programming language to customize new applications, and the application generated through ViMeTWizard can be generated as many times as the user desires, until the expected result is obtained.
In this manner, the main advantages verified for the ViMeT are the simple and quick manner to develop new applications in the framework’s domain, the possibility of manipulating the applications, and the easy integration of new classes representing new functionalities and interaction techniques.
Among the mentioned methodologies, the chosen one contains a testing phase that assists the software’s development with quality, in addition to the fact that it seems to be like the standard development of an object-oriented software for documentation.
Finally, it is important to show a comparison of ViMeT’s characteristics with other VR frameworks mentioned in “OBJECT-ORIENTED FRAMEWORKS AND VR FRAMEWORKS.” Table 1 shows their main features. It is possible understanding that ViMeT is the only one that joins two important characteristics: open source, also including a free DBMS, and multiplatform. The use of Java language makes the application intrinsically multiplatform, so the programmer does not need to know which is the execution environment. It is important to stress that none of them uses an automatic instantiation tool, except for ViMeT. On the other hand, this comparison highlights some aspects of the ViMeT that must be improved in the future: offering other free BDs and extension of domain.
The ViMeT framework, as previously mentioned, is not a finished project, in view of the fact that it is related to a research field that requires high degrees of realism and accuracy, ViMeT is integrated with other projects: (a) simulation of 3D objects, (b) interaction module, and (c) evaluation by doctors and medical students.
In relation to the first one, in parallel to the building of the ViMeT, a 3D simulation system using image processing and VR techniques were developed aiming at extracting the shape and measures from 2D images from a mammography.28 The second project is related to a system that includes classes in the ViMeT to allow interaction by using dataglove and haptic devices. The third project is for the evaluation of generated applications, which will be detailed as follow.
After the interaction module integration, the generated applications were evaluated by health professionals and a computer professional (volunteers for the tests) through training simulations using conventional (mouse and keyboard) and non-conventional (dataglove and haptics) devices. Examples of simulations using non-conventional equipments are shown in Figures 10 and and11.11. The task requested to the users was to manipulate the virtual medical instrument until a collision with the virtual human organ was detected by the system. A list of questions was used to collect information  about the applications, devices, and sensations that the users noticed in the simulations.
In this evaluation, objects considering two ways of texture were available (Fig. 12). The first model used texture composed by colors, and the second one used a file in JPEG format added to the wireframe object. However, the users considered that the objects with textures presented more difficulty to manipulate and to notice a sense of depth. Therefore, for the following tests, it was decided to maintain only the wireframe objects. A research about more suitable textures is currently being conducted, and this question will be improved in future versions of ViMeT.
As described by Correa et al.,28 the volunteers had difficulties mainly with the mouse and haptic device because of the type of task requested. The suggestions given by the volunteers were very important to define ways for improving the ViMeT and for the building of other frameworks of this nature or medical applications. One of the main changes brought about from this evaluation is the modeling of objects. Users considered that a more complex object representing the human body could be better to provide the spatial localization. As a result, new objects were built, as shown in Figure 13. These objects consider other layers besides the surface used before. As can be observed, they include inner structures like a nodule and breast ducts. The volunteers believe that VR systems can help in the medical training that is used in several procedures.
This paper presented the development process of an OOF for medical training using VR techniques, including deformation, collision detection, stereoscopy, and VE dynamic. One of the requirements during the ViMeT development was a structure of classes that allowed the integration of new functionalities and other interaction devices. Such purpose was achieved with the help of the objected orientation paradigm using Java programming language and the Java3D API. Thus, it was confirmed that the use of a framework could avoid the dependency of a single developer. The Java 3D API also allowed the reduction of the number of classes because it enabled the building of a hierarchic structure of classes composed by superclasses, subclasses, and methods that facilitated the implementation of VEs considering a number of already foreseen functionalities.
It was noticed that, in addition to the concern of building an OOF, there is a great concern with regards to the classes hierarchy imposed by the scene graph structure, which contributed to the generalization of parameters, to transforming the objects which became dynamic, as well as the addition and removal of the objects in the VE.
The applications can be generated using the ViMeTWizard innumerable times until the expected result is achieved. Another question was about the performance of the applications generated through ViMeT; however, it was verified that the performance is not altered by the amount of functionalities but by the structures of the modeled objects.
It is important to highlight that, although only biopsy exams case studies were considered, ViMeT classes can be reused in the development of medical training applications in several domains. By the direct instantiation, it is possible to use them in VR applications that belong to other domains and, thus, expand the use of the presented framework.
According to the results obtained in case studies, it can be verified that ViMeT’s performance is highly satisfactory and that every result expected with its implementation was achieved. Initial demonstrations for medical professionals show that the tools have full potential to be included in medical teaching in a near future. Nevertheless, it was verified that the generated applications are at a beginning stage and cannot be applied to the practice of medical teaching for a while. In this version, only wireframe objects were included in order to verify the correction of the implemented functionalities. Nowadays, new objects considering textures and the internal composition of the human organs are under study. The deformation process has also been modified in order to consider the full composition of objects. Meanwhile, this question does not involve the implementation or change in ViMeT but only the acquisition or modeling of more suitable objects. The 3D simulation mentioned in “INTEGRATION OF THE VIMET WITH OTHER PROJECTS” is an initial effort to solve this question. Thus, more realistic applications can be obtained after the implementation of the current studies.
Although the ViMeT framework has some aspects to be improved, the generated applications showed that the use of VR application can be promising to become cheaper, motivating, and more effective as medical training for students. In addition, the procedure can be repeated countless times without injuring patients until the student acquires the desired ability.
We would like to thank the financial support of CNPq.
Ana Cláudia Melo Tiessi Gomes de Oliveira, Phone: +55-14-3471-4723, Fax: +55-14-3471-4723, Email: moc.liamg@isseitana.
Fátima de Lourdes dos Santos Nunes, Email: email@example.com.