We begin with a brief review of the concept behind TaCTICS and the system architectures, as described in [8
]. TaCTICS is an open source, open access software package as well as hosted software currently containing a set of CT studies and associated contours for a multitude of anatomical sites.
In essence, TaCTICS is intended to be a user-friendly website that allows users to upload their radiotherapy planning structures in DICOM-RT format and to subsequently receive feedback regarding the conformance of these structures with a reference or those of other users. TaCTICS can be configured for a variety of scenarios including educational intervention, multi-site clinical trials, the development of consensus atlases and as a library of “expert” contours for training in resource-poor setting.
The three primary components of TaCTICS are the data, a set of metrics for comparison of structures and a user-interface for visualizing the comparisons and metrics.
As described previously [8
], the hosted TaCTICS website currently contains data from two IRB-exempt projects conducted under the auspices of University of Texas Health Science Center San Antonio Institutional Review Board as well as two prospective studies.
The first retrospective dataset consists of DICOM-RT structured resulting from a double-blind study [18
]of an instructional intervention where users contoured a standardized case presentation of T3N0M0 rectal cancer case twice. Anonymized patient DICOM files of CT studies were used to the target delineation datasets. In this case, half of the participants had access to consensus-based anatomic atlas between the first and second sessions. 15 radiation oncologists, both experts and non-experts observers participated in the study and submitted a Gross target volume (GTV), and 2–3 clinical target volume (CTVs) for each of 2 contouring sessions, resulting in 94 distinct ROI structures available for analysis. Inter and intra-observer variability was evaluated and previously reported [21
The second retrospective dataset consists of a previous study examining the effect of the human-computer user interface device (UID) on target volume delineation efficiency [21
]. In this study, observers contoured the several anatomical sites including a prostate, brain, lung, and head and neck case presentation twice. The first delineation was performed using a standard mouse-keyboard configuration while the second with a graphic tablet–pen interface. Twenty-one observers contoured target volumes with both UIDs resulting in more than 400 structures. Two users had been designated as ‘experts’ based for each site.
TaCTICS is currently capable of calculating a variety of volumetric overlap and surface distance metrics including Volumetric Difference (VD), Dice and Jaccard coefficients, and false positive and false negative Dice. Additionally, surface-based measures including the robust Hausdorff distance can be computed. [8
TaCTICS also provided a number of techniques for creating consensus from a set of contours. These include probability maps as well as the commonly used Simultaneous Truth and Performance Level Estimation (STAPLE), an expectation- maximization algorithm [22
] that computes a probabilistic estimate of true segmentation given a set of manual contours. In addition to the expert derived contours we have created an additional set of “ground truth” contours using this algorithm.
TaCTICS can be configured to allow users to compare their structures to the structures of all other users or to the structures from a particular “expert” user or consensus structures generated using STAPLE or a probability-map based approach.
The TaCTICS website itself was built using open-source software libraries, including the Ruby on Rails1
framework and the ruby-dicom library2
. After a user uploads the DICOM-RT file produced by their treatment planning software, TaCTICS extracts information about the studies contained therein including location of the CT slices containing contours, DICOM header metadata such as region or ROI names, and volume and slice information for each the delineated structures. The system stores these data, together with the metrics derived from all users’ uploaded files, in a relational database. The actual processing of the structures themselves, along with the calculation of the metrics, is performed in C++ using the ITK toolkit3
. The flow of data and user interaction for the system is described briefly.
Users can log in and download CT slices for their desired study (). They can the delineate the target volumes using their usual treatment planning system and upload the resulting DICOM RTSTRUCT file to the TaCTICS website. Based on how the system is configured, users can then select experts or consensus structures for comparison.
A screenshot of the TaCTICS user interface
The users are able to compare their own or other users’ structures against reference structures as previously described, and in doing so may calculate any of the previously-described metrics and review histograms illustrating how their own performance compares to that of the other users of the system. They may also directly visualize their structure contours together with reference or expert contours composed over thumbnails of CT slices. Users can identify their place on a histogram of all users. Users can then, by their relative histogram position, judge visually as well as numerically their agreement with a reference.