|Home | About | Journals | Submit | Contact Us | Français|
A multidimensional image navigation and display software was designed for display and interpretation of large sets of multidimensional and multimodality images such as combined PET-CT studies. The software is developed in Objective-C on a Macintosh platform under the MacOS X operating system using the GNUstep development environment. It also benefits from the extremely fast and optimized 3D graphic capabilities of the OpenGL graphic standard widely used for computer games optimized for taking advantage of any hardware graphic accelerator boards available. In the design of the software special attention was given to adapt the user interface to the specific and complex tasks of navigating through large sets of image data. An interactive jog-wheel device widely used in the video and movie industry was implemented to allow users to navigate in the different dimensions of an image set much faster than with a traditional mouse or on-screen cursors and sliders. The program can easily be adapted for very specific tasks that require a limited number of functions, by adding and removing tools from the program’s toolbar and avoiding an overwhelming number of unnecessary tools and functions. The processing and image rendering tools of the software are based on the open-source libraries ITK and VTK. This ensures that all new developments in image processing that could emerge from other academic institutions using these libraries can be directly ported to the OsiriX program. OsiriX is provided free of charge under the GNU open-source licensing agreement at http://homepage.mac.com/rossetantoine/osirix.
THE RAPID EVOLUTION of digital imaging techniques and the increasing number of multidimensional and multimodality studies constitute a challenge for PACS workstations and image display programs. With the improvement of spatial resolution of multi-detectors CT scanners and the emergence of multimodality exams like PET-CT1 or Cardiac-CT,2 traditional two-dimensional image viewers and image display programs are becoming unsuitable for interpretation of these large sets of images. For most tomographic imaging techniques such as magnetic resonance imaging (MRI) and CT, the traditional 2D acquisition technique of cross-sectional slices is evolving into 3D volume acquisitions with isotropic voxel sizes resulting in very large data sets. The conventional way of reviewing these images slice-by-slice is too cumbersome for interpreting the 800 to 1,000 slices that can be acquired with multi-detector CT scanners. These large sets of images require additional image processing and reformatting to make them suitable for efficient and rapid image navigation and image interpretation. In most cases this can only be achieved on high-end dedicated 3D workstations that can provide thick-slab maximum intensity projections (MIP), orthogonal and oblique multiplanar reformatting (MPR), and 3D volume and surface rendering.3 These recent changes in the acquisition modalities require radiologists to use expensive dedicated 3D processing workstations to properly interpret these exams.4 Furthermore, access to these visualization and rendering tools is usually limited to high-end users in a radiology department, preventing referring physicians, surgeons, and other care providers to benefit from the extraordinary value of the multidimensional imaging techniques for decision making and patient management In most cases static snapshots and preselected movie clips must be exported from the 3D processing workstations to be distributed to other users outside of a radiology department.
The goal of our project is to develop a completely new software platform that will allow users to efficiently and conveniently navigate through large sets of multidimentional data without the need for high-end expensive hardware or software. We also elected to develop our system on new open-source software libraries, thus by allowing the system to be easily adaptable to a variety of hardware platforms. We also benefit from our experience several years ago of having developed one of the first free DICOM viewers, OSIRIS, running on Macintosh and Windows personal computers.5,6 The success and wide adoption of the OSIRIS software has encouraged us to extend our effort into a completely new program more suitable for navigating through large multidimensional data sets. The name given to the new software, OsiriX, marks the transition to a completely new platform with the added “X” indicating the migration to the new Macintosh operating system version 10, also called MacOS X. The X also indicates the compatibility with underlying Unix platform and the adoption of the open-source paradigm.
In our design we also wanted to take advantage of the significant progress in performance and flexibility of 3D animation on personal computers mostly driven by the computer graphics and game industry. Most video games today are developed on OpenGL14 graphic libraries that benefit from hardware acceleration and processing capabilities of today’s ultra-fast video cards. Also, OpenGL, because it is an industry standard, adapts automatically to any hardware configuration and take advantage of any hardware accelerator that is provided for video display of 2D and 3D data.
In the scientific community, new open-source libraries have emerged for the visualization of multidimensional data. The Visualization Toolkit or VTK7 is a well-recognized and widely adopted software library that runs on multiple platforms and has been used for numerous scientific and medical applications so far. The recent adjunction of the Insight Segmentation and Registration Toolkit or ITK,8 mostly funded by the US National Library of Medicine as part of the Visible Human Project,9 adds a wealth of additional rendering and image processing tools for medical applications. These new open-source software toolkits offer powerful functions to do complex images manipulations, and great performance for real-time 3D image visualization.
With the introduction of more advanced image processing and visualization functions, imaging software applications tend to become more complex and usually include numerous processing functions that are not necessarily useful for all users. Depending on the type of images being evaluated and the interpretation task to be performed, most users would only use a fraction of the tools or features of a typical image processing and 3D visualization program. The purpose of our project was to explore new techniques that would allow easy customization of the program for different users and applications. Each user should be able to customize the software to include only the functions and tools needed for their specific purposes.
The OsiriX software program is developed as a stand-alone application for the MacOS X operating system. It includes an image database that is updated automatically when new images are downloaded to a specific directory (Fig.Â 1). Images can either be pushed from the PACS using a DICOM “store” function or they can be “pulled” by a DICOM query-retrieve function of the program. Image files can also be manually copied from off-line media or from other network sources. The OsiriX software was developed based on an open architecture built on existing open-source components as described in FigureÂ 2. The main components are these:
An important and challenging aspect of the development of this project was to integrate all of these technologies. OpenGL, VTK, ITK, DICOM Offis, Papyrus, and Quicktime are C/C++ cross-platform toolkits. These components were not designed to be fully compatible, and some inconsistencies and incompatibilities between header files or compiler flags made the integration somewhat difficult and laborious. Once past the steep learning curve in assessing the structure of the various components, the major advantage of the integrated product is the wealth of functions and features that are invaluable for any software developer of medical imaging applications.
The most important aspect of the project is that all the components function together within a simple and user-friendly graphic user interface. To achieve the best possible user interface design, we chose to develop the OsiriX program on a Macintosh platform to benefit from its well-known user interface features and convenience and ease of use. Additionally, the latest development environment provided by Apple greatly facilitates the rapid development of interactive graphic applications. The first version of OsiriX was developed in less than 6 months including complex 2D and 3D functions, DICOM files, and network management and multimedia file formats management. One of the key features of the latest Macintosh graphic user interface is the ability to interactively add or delete some of the software features by simply dragging icons from a retractable palette (Fig.Â 3) to and from a window toolbar. Each icon can represent a simple program function or a complex processing tool. This feature alone offers a major advantage over other computer platforms, allowing easy customization of the program adaptable to different users’ needs.
The OsiriX program offers all the basic image manipulation functions of zoom, pan, intensity adjustment, and filtering with real-time performance. Additional functions such as multiplanar projection, convolution filters, variable slice thickness adjustments, volume rendering, minimum and maximum intensity projections, and surface rendering are also accessible in quasi-real-time, depending on the hardware used, as well as on the number of slices to reconstruct. For example, on a dual-GS 2Â GHz PowerMac computer, OsiriX can to render two images per second of a 400 CT slices set in volume-rendering mode. All these basic functions being handled by the OpenGL library essentially rely on the processing capabilities of the video hardware and require very minimal processing from the central processor unit (CPU). In essence the most basic image manipulations are processed in parallel by the video graphic processor unit (GPU), and the performance therefore does not depend on the computer processor unit (CPU).
The program enables the rapid review of very large sets of images and does not rely on pre-loading of the images from the disk to memory before review. As soon as an images serie is selected, images will automatically be displayed on the screen in a cine loop using a “streaming” technique, enabling direct display of the images at a very rapid pace. For example, on a G4 1Â GHz mobile PowerBook computer, OsiriX is able to completely load a 355 CT images series in less than 10 seconds (loading rate: 35 images per second) while displaying the first image on the screen in less than 1 second. This streaming technique allows for an unlimited number of image series to be displayed simultaneously in separate windows. The software architecture was tested on very large image series consisting of thousands of images without any limitations. The OsiriX program was developed and mainly tested on the following standard off-the-shelf computers: PowerPC G4 laptop from Apple Computer, running at 1.25Â GHz with 768Â MB of RAM, with a built-in original 64Â MB graphic board from ATI (ATI Mobility Radeon). Even on this standard laptop, the basic image manipulations were extremely fast and performed in real time without noticeable delays in response. For performance comparisons, we also implemented the software on a PowerPC dual G5 processors desktop computer from Apple Computer, running at 2Â GHz with 1Â GB of RAM and with a 128Â MB graphic board from ATI (ATI Radeon 9600 Pro): performance was significantly better on the dual processor unit, with sometimes up to a factor of 6â€“10 faster performance for computer intensive tasks such as MIP and volume rendering, mostly due to the faster graphic board and dual processor architecture. The other basic functions of image manipulation and display such as intensity and contrast, zoom and pan, and navigation through large image sets were not noticeably different, as they are already fast in real time on the standard laptop computer. The program was also tested on lower end computers such as older generation laptops based on 800Â MHz G4 PowerPC and 512Â Mb of RAM, as well as a desktop 1.2Â GHz G4 with 1Â Gb of memory. No perceivable difference in performance could be identified except for moderately-slower 3D rendering and maximum intensity projection functions. The OsiriX software is designed to be multithreading compliant, taking advantage of parallel processing capabilities of multiprocessors when available. In particular, all graphics manipulations and rendering functions are tilled in separate threads and distributed to each available processor when possible. The support of multiprocessor architecture is seamless, performing particularly well on the new generation of dual processor G5 computers: the HyperTransport technology developed by Apple23 greatly accelerates memory transfers from RAM to processors (up to 12.8 gigabytes/second). The RAM to processor transfer is particularly critical for all 3D medical imaging rendering techniques, and significant improvement in performance can be perceived on multiprocessor platforms.
Another potential improvement in performance could be expected for very computational-intensive image processing and rendering techniques by using Grid Computing technology federating several networked computers. Grid Computing technology would allow multiple computers to be networked to perform complex processing tasks such as high-resolution volume rendering and surface rendering of very large data sets.
The development of this new DICOM viewer was motivated by the necessity to a have a tool more suitable for multidimensional and multimodality imaging studies such as PET-CT and ultra-fast multidetector Cardiac-CT.24 Essentially these new imaging techniques provide image data that are beyond the traditional 3D anatomical data. The combined PET-CT adds a new dimension to the data that represent the metabolic activity of the tracer, and blending this functional parameter represents a fourth dimension that the users need in order to navigate. The ultrafast multidetector CT scanners can add a temporal dimension to the data by sequentially acquiring images over time to display temporal behavior such as a beating heart or transit of a tracer or contrast agent across a vascular tree or an organ. The newer generation of PET-CT scanners can now provide data sets that represent all five dimensions for cardiac PET-CT studies. We therefore designed our software to allow users to interactively navigate in the five dimensions.
Although the software does not yet support any realignment technique for images obtained from different modalities, it does provide a very simple and very intuitive way to generate fused images by blending two image sets that are prealigned. For image fusion, the user needs to select and adjust a color scale for one of the image sets that will be overlaid over another image set. Both sets must be open at the same time on the screen, and the fusion can be initiated simply by dragging and dropping the title bar of the window containing the overlay image (PET for example) over the window of the basic images (CT for example) as shown in FigureÂ 4. The program will automatically generate a set of fused images by color blending the two image sets. This new set is still fully synchronized with the original images, and any changes in contrast or intensity in either of the two sets will be reflected in the fused image set, allowing the user to adjust the best rendering setting of the fused image. The fused image set can also be manipulated like any other image data set, and image processing functions such as MIP rendering or MPR can be applied to the four-dimensional data (FigÂ 5).
To facilitate and improve the navigation and image manipulation functions in five-dimension exams, we explored innovative solutions using advanced joystick and multidimensional navigation devices that can be used in conjunction with the standard mouse and keyboard functions. We elected to use a special multipurpose video editing jog-wheel device (FigÂ 6) that is widely used by professional video editors. This allows rapid navigation in multiple data sets and multiple dimensions with a single hand. The addition of this low-cost pointing device increases the ability of the user to navigate rapidly and to switch among different functions in real time.
By using this jog-wheel device we were able to develop an interactive graphic user interface that allows navigation in these five dimensions in all rendering modes: the classic 2D viewer, but also in MPR or volume rendering modes (FigÂ 7). The user is able at any time to move in any of the five dimensions: for example it is possible to modify the dynamic and the fusion parameters while navigating in a 3D MPR view.
Radiology imaging modalities are evolving from conventional sets of 2D tomographic slices to 3D volumetric acquisition extending to a fourth or fifth dimension with temporal and functional data that can be acquired with ultrafast CT and MR scanners with combined PET/CT scanners. To allow radiologists and clinicians to conveniently and efficiently interpret these large exam sets, traditional image viewers have to be re-designed and tailored to a new paradigm of multidimensional image navigation visualization, and manipulation. By combining the performance of new hardware components and the wealth of existing open-source image processing and manipulation tools, it is now possible to develop a new generation of high-performance multidimensional image viewers for off-the-shelf personal computers. These advanced navigation and visualization tools were traditionally only accessible on expensive dedicated 3D workstations, thereby restricting their use to specialist radiologists. With easier access to these multidimensional navigation and visualization tools in standard personal computers, they should soon become complementary tools for routine interpretation of complex diagnostic studies.
One of the key features of the OsiriX software is its flexible user interface allowing users to customize the program by adding and removing tools and functions from the tool bar and menus of the program. This also allows the creation of “customized” versions of the program for specific groups of users. Users who are not computer experts can adopt simpler versions of the software containing only a limited number of essential tools. Specialized versions of the program could be easily customized for specific medical specialties or for specific clinical applications without the need for any additional programming or software development.
The multithreading ability of the software that allows OsiriX to take full advantage of multiprocessor computers ensures a scalability of performance that can be expected from future generation computers. The same software architecture will also allow a migration toward the emerging technology of Grid Computing. With Grid Computing technology25, 26 it is possible to significantly enhance image processing performance by using clusters of computers. The grid technology is already used for large 3D virtual reality software applications to greatly accelerate the rendering time. In the computer animation movie industry, including PIXAR27 and other industry leaders, Grid Computing technology has taken a prominent role. Most of the rendering is performed based on clusters of computers. Grid Computing is also used extensively in large multicentric bioinformatic projects for genome analysis.28, 29 We anticipate that Grid Computing technology could significantly improve the performance of 3D medical imaging rendering techniques. The VTK library already supports Grid Computing rendering through the open-source and cross-platform standard Message Passing Interface (MPI).30 Apple Computer recently announced a new technology, X-Grid,31 that will facilitate the management and configuration of large clusters of computers. With the X-Grid technology it will be easy to connect all computers of a radiology department to share complex processing and rendering tasks during processor idle time. In large academic radiology departments it is common to have a relatively large number of computers that are only partially used and remain idle for significant amounts of time. The Grid Computing technology makes it possible to take advantage of idle computer time for performing computational intensive tasks needed for 3D rendering applications across the network. By adopting the X-Grid technology, OsiriX, which is already multithread compliant, could benefit from a significant improvement in performance if used in a network of multiple computers.
OsiriX is distributed freely as open-source software under the GNU licensing scheme at the following Web site: http://homepage.mac.com/rossetantoine/osirix.