PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Clin Trials. Author manuscript; available in PMC 2017 April 1.
Published in final edited form as:
PMCID: PMC4785064
NIHMSID: NIHMS726076

Architecture design of a generic centralized adjudication module integrated in a web-based clinical trial management system

Abstract

Background

Centralized outcome adjudication has been used widely in multi-center clinical trials in order to prevent potential biases and to reduce variations in important safety and efficacy outcome assessments. Adjudication procedures could vary significantly among different studies. In practice, the coordination of outcome adjudication procedures in many multicenter clinical trials remains as a manual process with low efficiency and high risk of delay. Motivated by the demands from two large clinical trial networks, a generic outcome adjudication module has been developed by the network’s data management center within a homegrown clinical trial management system. In this paper, the system design strategy and database structure are presented.

Methods

A generic database model was created to transfer different adjudication procedures into a unified set of sequential adjudication steps. Each adjudication step was defined by one activate condition, one lock condition, one to five categorical data items to capture adjudication results, and one free text field for general comments. Based on this model, a generic outcome adjudication user interface and a generic data processing program were developed within a homegrown clinical trial management system to provide automated coordination of outcome adjudication.

Results

By the end of 2014, this generic outcome adjudication module had been implemented in 10 multicenter trials. A total of 29 adjudication procedures were defined with the number of adjudication steps varying from 1 to 7. The implementation of a new adjudication procedure in this generic module took an experienced programmer one or two days. A total of 7,336 outcome events had been adjudicated and 16,235 adjudication step activities had been recorded. In a multicenter trial, 1144 safety outcome event submissions went through a three-step adjudication procedure and reported a median of 3.95 days from safety event case report form submission to adjudication completion. In another trial, 277 clinical outcome events were adjudicated by a six-step procedure and took a median of 23.84 days from outcome event case report form submission to adjudication procedure completion.

Conclusions

A generic outcome adjudication module integrated in the clinical trial management system made the automated coordination of efficacy and safety outcome adjudication a reality.

Keywords: Clinical trial, outcome adjudication, endpoint adjudication, medical safety monitoring, clinical trial management system

Background

To standardize outcome assessments and prevent ascertainment bias, centralized adjudications for primary safety and efficacy outcomes were recommended by the Food and Drug Administration and the European Medicines Agency1,2 and have been used widely in multicenter clinical trials.311 Independent blinded central adjudication has been considered as an effective measure to prevent potential biases from site investigators when perfect treatment allocation concealment is not available.12 It has also been a practical strategy to reduce the variation in clinical assessments due to uneven expertise among investigators.13 Outcome adjudication coordination encounters four special challenges. First, there is not a fixed model for various adjudication procedures. Some simple procedures involve only one step, and some others involve multiple steps with specific logic interrelationships. For example, a tiebreaker is required only when the previous two adjudicators disagreed with each other. Second, the timing of some outcome events, such as adverse events, is hard to predict. Third, the allowed time frame for completing an adjudication procedure is often narrow. For example, according to the U.S. Code of Federal Regulations (21 CFR 312),14 an expedited adverse event report must be submitted within 24 hours of learning of the event, followed by a complete expedited report within 5 calendar days of the initial 24-hour report. To fulfill the regulatory obligation, adjudication on the seriousness, the relationship to study agent, and the expectedness of the adverse event must be completed within this time frame. Finally, information on the outcome event may change while the event is being adjudicated.

As a result of these challenges, an automated coordination module for outcome adjudication is rarely available. A recent search on www.PubMed.gov and www.google.com with various combinations of key words “web based outcome endpoint adjudication system” returned only two relevant articles.15,16 Nolen et al. reported a web-based endpoint adjudication system for the central adjudication on eligibility criteria and clinical endpoints prior to two interim analyses in the Caspofungin Study for Fungal Infections in Adults in Critical Care Settings (NCT00095316) planned to enroll 1200 subjects.15 Nolen et al. described the background and the process of the central adjudication procedure, and presented the features of the web-based adjudication system. There was no information on the system design and implementation provided. The trial was terminated after 38 subjects enrolled, and therefore no data on the benefit of using the system were available. Kuurstra et al. reported a web-based adjudication system, served as both an image management system and electronic data capture.16 The paper described the adjudication process for fracture healing trials, the functions and advantages of the web-based adjudication system. No information on the development strategies and techniques were covered, and no data on the benefit of using such system were provided. Outcome/endpoint adjudication modules have been included in some commercial clinical trial management system packages.1719 However, system architecture design strategies and database structures are often considered as the company’s intellectual properties, and therefore are not expected to be disseminated in public accessible research journals.

As the Data Management Center for the Neurological Emergencies Treatment Trials network and the National Institutes of Health (NIH) Stroke Trials network, both funded by the National Institute of Neurological Disorders and Stroke, the Data Coordination Unit at the Medical University of South Carolina was in charge of providing full scope information support for data management and project management activities. Motivated by the demands from trials in these networks, a comprehensive homegrown web-based clinical trial management system has been developed and has served for more than twenty multicenter clinical trials.2022 The system uses the three-tier architecture23 with the data tier in a Microsoft Structured Query Language (SQL) Server database, the application tier developed with Microsoft Active Server Pages for data processing and user interface generation, and the presentation tier on internet web browsers. The outcome adjudication module is one of the primary components of this home grown clinical trial management system. (Screenshots of the clinical trial management system are included in the online Appendix 1.)

Methods

Dynamic data-driven user interfaces architecture design

Dynamic data-driven user interfaces were used in the clinical trial management system. Attributes for each data field, data validation rules, and data access permissions on individual data forms were stored in database metadata tables. User interfaces were composed by system functions based on metadata. Data captured through user interfaces were processed by system functions based on the metadata, triggering database actions for inserting, updating or deleting a record. These system functions were developed generically and applied to all data forms, including case report forms (CRFs) and project management forms. This architecture design provided a foundation for the creation of a generic outcome adjudication module, which used the same system functions to process adjudication data for different adjudication procedures across multiple trials.

System specifications for outcome adjudication procedures

Outcome adjudication procedures are heterogeneous. To create a database model supporting different adjudication procedures and also to control the system complexity, the following assumptions were made as system specifications for the generic outcome adjudication module:

  1. Each adjudication procedure is defined on one CRF, which is associated with one backend database table.
  2. Each adjudication procedure consists of one or multiple sequential or parallel adjudication steps.
  3. Each adjudication step can be activated and locked by pre-specified, data-driven conditions.
  4. Each adjudication step has its [view] and [edit] user groups specified. This design allows designated adjudicators to receive the auto emails when an adjudication step is ready, and allows these adjudicators to add and edit adjudication results in the user interface. It also provides a tool to maintain the independence and blindness among adjudicators when needed.
  5. Each adjudication step includes one to five required adjudication questions in multiple-choice format, and one optional comment in free text format.
  6. All adjudication steps are conducted based on the most recent information on the CRF. An adjudication step may be repeated in case significant data changes occurred on the CRF after that adjudication step was submitted.

In practice, not all outcome adjudication procedures satisfy the above six assumptions. Some simplification and reformatting works may be needed when implementing complex adjudication procedures in to this module.

Database structure design for the outcome adjudication module

The database structure of the generic outcome adjudication module is shown in Figure 1. The [Form] table was created to maintain definitions of all data forms. One [Form Table] was created to store data on one data form. Users’ data access permissions were managed by the [User Group Permission] and [User Group Member] tables. Definitions of all adjudication procedures were stored in the [Adjudication Procedure] table. The [Adjudication Step] table was created to store attributes of each adjudication step within an adjudication procedure, including the activate and lock conditions, the categorical adjudication questions, the permissible answers to each of these questions defined in the [Code Item] table, and the view and edit user group IDs.

Figure 1
Database structure for the genetic outcome adjudication module

An adjudication procedure with multiple steps may take days, weeks, or even months to complete. During this period, data changes on the CRF could occur. The [Form Freeze Table] was created for each form table with adjudication tasks. When an adjudication step is submitted and the current CRF is newer than the last saved freeze record or there is no freeze record previously saved, a copy of the current CRF record is saved in the corresponding freeze table. Each adjudication result record is linked to one data freeze record via the freeze record ID. Each adjudication step has the activate condition and lock condition defined in the [Adjudication Step] table, using SQL. A dichotomous value, true or false, is returned when a query condition is evaluated. The activate condition determines if the adjudication step should be performed or not under the current condition composed by information collected in the study database. The lock condition determines if the previously performed adjudication step record can be edited or not. An adjudication step record can be edited only if the step is activated but not locked yet. Depending on the study operation requirements, for each adjudication step, the edit user group and the view user group may have one or multiple members.

Automated coordination of adjudication activities

The primary goal of the web-based outcome adjudication module is to automate the coordination of adjudication activities, by precisely controlling the workflow, in real-time, regarding who and when needs to do what based on data collected in the clinical trial management system. To achieve this goal, the following generic system features were developed in the outcome adjudication module:

  1. When a subject CRF is submitted, the system checks if any adjudication procedures were defined for this CRF in the [Adjudication Procedure] table based on the [Form ID]. If yes, the system evaluates the activate condition and the lock condition for each adjudication step. If an adjudication step is activated and not locked, an automated email is sent to each member of the edit user group for that adjudication step, as shown in Figure 1.
  2. When a CRF associated with at least one adjudication procedure is viewed by an authorized user, and that user has the view permission for an adjudication step, all previously completed adjudication results for that step will be posted on the screen following the CRF. An [Add/Edit] button is placed on the user interface if the user has the edit permission for that step and the activate condition is true and the lock condition is false.
  3. A generic user interface for add/edit adjudication results is composed based on the metadata in the [Adjudication Step] table. Each adjudication step includes one to five required categorical questions and an optional free text field. After submission, the adjudicator step result can be modified by the same adjudicator if that adjudication step remains activated and not locked yet.
  4. When an adjudication step is submitted, the system checks the activate conditions for all adjudication steps within the same adjudication procedure. Notifying emails are sent to members of the edit user group if the corresponding adjudication step is activated and not locked.
  5. In case the CRF is edited and resubmitted by site investigators after the adjudication procedure has started, the first adjudication step is activated. The adjudicator at this step decides if the previously completed adjudication steps need to be re-done.

The work flow of the automated coordination of outcome adjudication activities is shown in Figure 2. The activate condition and the lock condition are created to prevent conflicts among the actions of multiple adjudicators. The use of the freeze records allows the CRF to be edited while the adjudication procedure is ongoing. This design minimizes the risk of delayed reporting and updating of important safety and efficacy information. The roles of these designed features will be illustrated with examples in the results section.

Figure 2
Automated coordination of outcome adjudication procedures

Results

This generic outcome adjudication module has been implemented in 10 multicenter clinical trials managed by the Data Coordination Unit at the Medical University of South Carolina. From 2009 to 2014, a total of 29 adjudication procedures were defined, with the number of adjudication steps varying from 1 to 7. Adjudication procedures managed by the generic system included primary efficacy outcome adjudication, serious adverse event (SAE) medical safety monitoring, and central image review, etc. The following two examples are presented to illustrate the system design features and the implementation of actual outcome event adjudication procedures in to the generic adjudication module of the clinical trial management system.

Example 1: A three-step SAE adjudication procedure

Funded by the National Institute of Neurological Disorders and Stroke, the Progesterone for the Treatment of Traumatic Brain Injury Trial (ProTECT III, NCT00822900) was a randomized, placebo controlled, phase III, multicenter clinical trial to determine if progesterone along with standard medical care for brain injury is more effective at limiting the amount of damage caused by a traumatic brain injury than standard medical care alone.24 The ProTECT III study had a three-step SAE adjudication procedure, which included project manager’s review for CRF completeness, a medical safety monitor’s adjudication for seriousness, unexpectedness, and relationship to study treatment, and project manager’s review of the need for FDA Form 3500 (MedWatch) reporting. Figure 3 shows the work flow of this three-step adjudication procedure. To implement this three-step SAE adjudication procedure in the clinical trial management system, the following tasks were completed by a database programmer:

  1. A designated record was added in the [Adjudication Procedure] table.
  2. Three records were added in the [Adjudication Step] table, specifying attributes of each adjudication step, including the activate condition and the lock condition, as shown in Table 1.
    Table 1
    Definition of adjudication procedure steps
  3. Response options to categorical adjudication questions were added in the [Code Item] table.
  4. A [Form Freeze Table] was created for the SAE CRF.
  5. The [View] and [Edit] user groups were created in the [User Group] table.
  6. User group members were specified in the [User Group Member] table for all adjudicators and those who need adjudication view or edit permissions.
  7. Notification emails were specified in the [Email] table, and email recipient were specified in the [Email Recipient] table for the [Edit] user group for each adjudication steps.
Figure 3
Automated coordination of three-step SAE adjudication procedure

The automated coordination of the adjudication procedure was assisted by the use of event driven emails programmed in the database. As shown in Figure 3, when an SAE CRF was submitted, auto email #1 was sent to the project manager. If the project manager indicated that the CRF had not been properly completed, auto email #2 was sent to the site study coordinator. When the project manager indicated that the SAE was properly completed and required medical safety monitor adjudication, auto email #3 was sent to him/her. If the medical safety monitor voted for [Serious = No] or [Unexpected = No] or [Relationship = Unrelated or Unlikely], auto email #4 was sent to the project manager to close the adjudication procedure. Otherwise, auto email #5 was sent to the project manager to consider the necessity of the FDA MedWatch form. If the project manager indicated that the SAE required a MedWatch form, the computer system composed a draft of the form by auto-populating data fields based on information in the database and then sent auto email #6 to the site, asking the site to edit and complete the MedWatch form. When the site submitted the MedWatch form to the system, auto email #7 was sent to the project manager, who finalized the MedWatch form and submitted it to the FDA. After that, auto email #8 was sent to all clinical sites and other relevant agencies, notifying them of the SAE and the FDA MedWatch form submission.

Example 2: A six-step clinical outcome event adjudication procedure

Funded by the National Institute of Neurological Disorders and Stroke, the Platelet-Oriented Inhibition in New TIA and Minor Ischemic Stroke Trial (POINT, NCT00991029) is an ongoing prospective, randomized, double-blind, multicenter trial to determine whether clopidogrel is effective in preventing major ischemic vascular events at 90 days when initiated within 12 hours of transient ischemic attack or minor ischemic stroke onset in patients receiving aspirin 50–325 mg/day.25 As the primary efficacy outcome of the study, new ischemic vascular events required a central adjudication for the severity of the event, and the type of death if the severity was fatal. The study was planned to enroll 5840 subjects from more than 200 clinical sites in the USA in 8 years. Due to the importance and the complexity of the outcome events, the potential uneven outcome event assessment expertise across the large number of sites and the long study period, a rigorous outcome event adjudication procedure was implemented in order to protect the quality of the study. The workflow of this outcome event adjudication procedure is illustrated in Figure 4. When a clinical outcome event CRF was submitted by the site, the project manager was alerted by auto email #1 to review the event report type (New or Follow-up) and to determine if the clinical event coordinator review was needed. If the project manager indicated that the CRF did not need the clinical event coordinator review, the adjudication procedure was ended. This scenario occurred when site submitted a clinical outcome event CRF with data edits after the event had been previously adjudicated, and the project manager believed that the data edits did not need to be reviewed by the clinical event coordinator and other adjudicators. If the project manager voted “Yes” for requiring clinical event coordinator review, the clinical event coordinator was notified by auto email #2 to check if the uploaded event packet was complete and the event type was neurological, cardiac, or systemic. If the event packet was incomplete, the site was notified by auto email #3 to provide more information. Otherwise, the system started an independent double adjudication for the outcome event category. If the outcome event was fatal, the adjudicators were required to judge if the death type was ischemic, hemorrhagic vascular, or nonvascular. The adjudication team included three neurologists and three cardiologists. When the clinical event coordinator’s submitted review indicated the outcome event packet was complete, two adjudicators were randomly selected by the system based on the outcome event type identified by the clinical event coordinator, and were notified by auto emails #4 and #5 to vote independently on the outcome event category and the death type if applicable. If discrepancies occurred between the results of the two adjudicators, auto email # 6 was triggered, notifying the third adjudicator to independently adjudicate the outcome event. If the third adjudicator disagreed with both previous adjudicators, the adjudication committee chair was notified by auto email #7 to conduct the final adjudication. When the adjudication procedure completed, auto email #8 was sent to the project manager asking for the closure of the adjudication procedure. If the clinical outcome event CRF was edited by the site at any point during the adjudication process, the project manager was required to decide whether or not the entire adjudication procedure needed to be repeated. The attributes of each adjudication step were specified in the [Adjudication Step] table, as shown in Table 1. The workflow of this sixstep clinical outcome event adjudication procedure is shown in Figure 4.

Figure 4
Automated coordination of six-step clinical outcome event adjudication procedure

Although this six-step adjudication procedure was more complex than the three-step SAE adjudication procedure discussed in Example 1, its implementation in the generic outcome adjudication module followed exactly the same seven steps outlined in Example 1.

Table 2 shows the time between two consecutive adjudication steps and the overall adjudication duration length for these two adjudication procedures. In the ProTECT III study, a total of 1253 SAE CRFs were adjudicated. After the adjudication procedure started, 407 (32.5%) CRFs had no data edit submitted, and 82 (6.5%) CRFs had 3 or more submissions. A total of 2521 SAE submissions were reviewed by the project manager. Among them, 1218 SAE submissions were adjudicated by the medical safety monitor. After that, 1144 SAE adjudication procedures were closed by the project manager. For the 2521 SAE submissions, the entire adjudication procedure, from CRF submission to adjudication completion, took a median of 2.76 days. About 95% of the procedures completed within 2 weeks. The 1144 procedures went through all three steps took a median of 3.95 days to complete; about 5% of them took longer than 4 weeks.

Table 2
Number of days between adjudication steps

The six-step clinical outcome event adjudication procedure for the POINT study was more complex than the three-step serious adverse event adjudication procedure used in ProTECT III trial. A total of 673 clinical outcome events had been reported. Among them, 316 (47%) outcome event CRFs had no data edits after adjudication started, and 41 (6%) had 3 or more submissions after adjudication started. A total of 1210 clinical outcome event CRF submissions were reviewed by the project manager. Based on the review decisions made by the project manager and the clinical event coordinator, 277 submissions were independently adjudicated by the two randomly selected adjudicators. The median duration from CRF submission to adjudication completion for these 277 cases was 23.84 days. Sixteen (5.8%) cases took more than 90 days to complete. Among these 277 adjudication procedures, the two adjudicators disagreed with each other on 40 (14.4%) cases, and the adjudication committee chair was called in for 10 cases where the third adjudicator did not agree with either of the previous two adjudicators.

Discussion

The integrated, generic outcome event adjudication module provided an ideal solution for our home grown clinical trial management system to automate the coordination of different adjudication procedures in different trials. The database architecture design presented in this manuscript demonstrates a practical strategy to achieve this goal. Our experiences shown that the implementation of a new adjudication procedure into this generic adjudication module took one programmer one or two days to set up.

The described adjudication system has some limitations, mainly due to the generic database structure. The current generic design does not cover the logical relationships, such as skip patterns and conditional selections, among the adjudication questions. During the adjudication process, adjudicators may want to communicate with site investigators for additional information or data clarification. This requirement has not been covered by the current generic adjudication system. For complex adjudication procedures, investigator may find that the five categorical adjudication questions are not sufficient. For example, in the Established Status Epilepticus Treatment Trial (NCT01960075) which we are currently working on, the central adjudication procedure requires two adjudicators to independently answer 25 questions, including 17 categorical, 1 numerical, 2 date/time, and 5 text questions and 15 skip pattern logics among. The complexity of the ESETT clinical adjudication procedure exceeds the capacity of the generic adjudication system presented in this paper. Therefore, a customized application is developed for this ESETT trial.

It is also worth indicating that the automated coordination of adjudication activities does not eliminate the need for human intervention in the adjudication procedures. Often, a project manager is included in the adjudication procedure to ensure that the CRFs are properly completed and the information for clinical adjudication is sufficient. In some cases, the project manager will also assist clinical adjudicators in communicating with site investigators, and will ensure that all required adjudication activities are completed within the protocol specified time windows.

The implementation of an adjudication procedure in the generic adjudication module requires professional database programmers. The setup of the activate condition and the lock condition requires knowledge about the database structure and the relationship among different adjudication steps. The architecture design strategy, the unified framework for adjudication procedures, the generic database structure, and the implementation methods described in this manuscript will be of particular interest to other homegrown clinical trial management system developers.

Supplementary Material

01

Acknowledgement

The authors gratefully acknowledge the comments and advisors received from all members of the Data Coordination Unit at the Medical University of South Carolina. This research is partly supported by NIH/NINDS grants U01NS0059041, U01NS062778, U01NS062835, U01NS087748.

Partially supported by:

Clinical trials: NCT00822900: ProTECT III, David Wright, Emory University NCT00991029: POINT, University of California, San Francisco

Appendix 1

The Clinical Trial Management System

  1. System architecture design for dynamic data-driven user interfaces
  2. Project Menu Page for the Clinical Trial Management System
  3. Main Menu Page for a Project
  4. Data Collection Schedule
  5. Subject Case Report Binder
  6. Sample CRF – Subject Randomization Form
  7. Sample CRF – Clinical Assessment Form
  8. Sample CRF – End of Study Form
  9. Site/Investigator Regulatory Document Collection Management
  10. Site Payment Management
  11. Clinical Outcome Event Adjudication Procedure Status Report
  12. User interfaces for clinical outcome event adjudication steps
  13. Clinical outcome event adjudication procedure – An example

Reference

1. U.S. Department of Health and Human Services. Food and Drug Administration. Guidance for clinical trial sponsors. [accessed 16 August 2015];Establishment and operation of clinical trial data monitoring committees. 2006 http://www.fda.gov/downloads/Regulatoryinformation/Guidances/ucm127073.pdf.
2. European Medicines Agency. Committee for medicinal products for human use (CHMP) [accessed 16 August 2015];Guideline on data monitoring committees. 2005 http://www.emea.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003635.pdf.
3. Pogue J, Walter SD, Yusuf S. Evaluating the benefit of event adjudication of cardiovascular outcomes in large simple RCTs. Clin Trials. 2009;6:239–251. [PubMed]
4. Hata J, Arima H, Zoungas S, et al. Effects of the endpoint adjudication process on the results of a randomised controlled trial: the ADVANCE trial. PLoS One. 2013;8:e55807. [PMC free article] [PubMed]
5. Vranckx P, McFadden E, Cutlip DE, et al. Clinical endpoint adjudication in a contemporary all-comers coronary stent investigation: methodology and external validation. Contemp Clin Trials. 2013;34:53–59. [PubMed]
6. Tolmie EP, Dinnett EM, Ronald ES, et al. Clinical Trials: Minimising source data queries to streamline endpoint adjudication in a large multi-national trial. Trials. 2011;12:112. [PMC free article] [PubMed]
7. Beyer-Westendorf J, Halbritter K, Platzbecker H, et al. Central adjudication of venous ultrasound in VTE screening trials: reasons for failure. J Thromb Haemost. 2011;9:457–463. [PubMed]
8. Vannabouathong C, Saccone M, Sprague S, et al. Adjudicating outcomes: fundamentals. J Bone Joint Surg Am. 2012;18(Suppl 1):70–74. [PubMed]
9. Heckbert SR, Kooperberg C, Safford MM, et al. Comparison of self-report, hospital discharge codes, and adjudication of cardiovascular events in the Women's Health Initiative. Am J Epidemiol. 2004;160:1152–1158. [PubMed]
10. Pike MC, Kelley L. Data quality challenges in systemic lupus erythematosus trials: how can this be optimized? Curr Rheumatol Rep. 2012;14:324–333. [PMC free article] [PubMed]
11. Serebruany VL, Atar D. Viewpoint: Central adjudication of myocardial infarction in outcome-driven clinical trials--common patterns in TRITON, RECORD, and PLATO? Thromb Haemost. 2012;108:412–414. [PubMed]
12. Viera AJ, Bangdiwala SI. Eliminating bias in randomized controlled trials: importance of allocation concealment and masking. Fam Med. 2007;39:132–137. [PubMed]
13. Wilson JT, Slieker FJ, Legrand V, et al. Observer variation in the assessment of outcome in traumatic brain injury: experience from a multicenter, international randomized clinical trial. Neurosurgery. 2007;61:123–128. discussion 128–129. [PubMed]
14. U.S. Food and Drug Administration. Code of Federal Regulations, Title 21, Part 312.32: IND Safety Reports. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=312.32.
15. Nolen TL, Dimmick BF, Ostrosky-Zeichner L, et al. A web-based endpoint adjudication system for interim analyses in clinical trials. Clin Trials. 2009;6:60–66. [PMC free article] [PubMed]
16. Kuurstra N, Vannabouathong C, Sprague S, et al. Guidelines for fracture healing assessments in clinical trials. Part II: electronic data capture and image management systems--Global Adjudicator™ system. Injury. 2011;42:317–320. [PubMed]
17. ICON’s Electronic Adjudication System. http://www.iconplc.com/technology/products/electronic-event-adjudica/.
19. Tyner C, Zaher E-A. Electronic endpoint adjudication. How an integrated, EDC-based system can optimize the endpoint process. Applied Clinical Trials. 2010 Mar 1;
20. Durkalski V, Zhao W, Dillon C, et al. A web-based clinical trial management system for a sham-controlled multicenter clinical trial in depression. Clin Trials. 2010;7:174–182. [PubMed]
21. Zhao W, Waldman BD, Dillon C, et al. A web-based medical safety reporting system for a large multicenter clinical trial: the ALIAS experience. Contemp Clin Trials. 2010;31:536–543. [PMC free article] [PubMed]
22. Zhao W, Durkalski V, Pauls K, et al. An electronic regulatory document management system for a clinical trial network. Contemp Clin Trials. 2010;31:27–33. [PMC free article] [PubMed]
23. Eckerson WW. Three tier client/server architecture: achieving scalability, performance, and efficiency in client server applications. Open Information Systems. 1995;1:3.
24. Wright DW, Yeatts SD, Silbergleit R, et al. Very early administration of progesterone for acute traumatic brain injury. N Engl J Med. 2014;371:2457–2466. [PMC free article] [PubMed]
25. Johnston SC, Easton JD, Farrant M, et al. Platelet-oriented inhibition in new TIA and minor ischemic stroke (POINT) trial: rationale and design. Int J Stroke. 2013;8:479–483. [PMC free article] [PubMed]