|Home | About | Journals | Submit | Contact Us | Français|
Experts discuss one of the biggest challenges to any research site implementing a clinical trial program: the audit process.
Article three in this Journal of Oncology Practice series, “Quality Assurance and Educational Standards for Clinical Trial Sites” (Journal of Oncology Practice, November 2008) discussed two of the exemplary attributes of clinical trial sites suggested in the ASCO Statement on Minimum Standards and Exemplary Attributes of Clinical Trial Sites, published in the May 20, 2008, issue of Journal of Clinical Oncology. These attributes, quality assurance and formal maintenance of high educational standards, were defined, and their contribution to the optimum functioning of an effective clinical trial site was presented. This, the fourth article in the series, discusses one of the challenges to any research site implementing a clinical trial program—the audit process. Four experts weighed in on the audit process, the purpose of audits, and how to survive an audit.
As noted in the ASCO statement on minimum standards and exemplary attributes, a quality clinical trial research site must protect the welfare and rights of all trial participants. It also needs to assure that the data generated during the conduct of the trial are valid, can be used to draw valid conclusions, and will benefit patients with cancer now and in the future. This requires compliance with accepted standards for all aspects of clinical research, some of which have already been discussed in this series, including Good Clinical Practice guidelines and the use of standard operating procedures. The audit process is a way of assessing and ensuring this compliance.
Entities that conduct audits include the National Cancer Institute (NCI) for those sites participating in cooperative group-sponsored trials, the US Food and Drug Administration, and major cancer centers. Sites conducting research supported by a pharmaceutical company may be subject to monitoring visits by the trial sponsor. There are three general areas of concern in the audit process regardless of which entity is performing the audit: patient safety, including consent forms and institutional review board (IRB) activities; data collection and records; and the pharmacy and investigational drug supply. Most audits are conducted every three years, and if a site doesn't pass because there are significant issues, it will be reaudited within the year.
More frequent audits may be conducted for sites that accrue large numbers of patients. A given site may be audited several times a year by different groups, depending on the mix of trials it conducts.
Sites are usually notified several months in advance of the date an audit will take place. About a month before the audit, the site will be notified of the selected cases that will be reviewed. “I've been telling my staff I hope they'd prepared all along,” says Marge Good, RN, BSN, MPH, OCN, Wichita Community Clinical Oncology Program, Wichita, Kansas, whose site recently underwent an audit.
Susan I. Devine, CCRP, Hospital for Sick Children, Toronto, Ontario, Canada, uses a prereview check sheet (see box) that was developed in-house. “I'm over the top,” she says. “I know the results before the audit. From my perspective, I have always trained staff to work audit-ready. It's too hard to prepare in six weeks. We don't want surprises at audit.” She notes that if they do find errors during a check of the named cases that will be audited, they correct them. “Anything you can fix before the auditor gets there is a good thing.” She notes that at her hospital, they monitor quarterly, selecting one case per disease, “not to create more work for the research coordinators, but to help by finding any potentially serious systematic problems in the process early on.” Good's group also performs internal audits, having a clinical research associate (CRA) examine two charts per week. Stephen S. Grubbs, MD, Principal Investigator, Helen F. Graham Cancer Center, Delaware Christiana Care CCOP, Newark, Delaware, says that it is the principal investigator (PI) who bears ultimate responsibility for the trial and all activities associated with it, including the audit. He agrees that if an error is detected, documented, and corrected, the auditors will not hold it against the site. Good says that this may not be true for all auditors. They may recognize you found the error and tried to correct it, but may still give you a deviation (a lesser versus a major) depending on the degree of significance.
Peter Paul Yu, MD, Palo Alto Medical Foundation, Mountain View, California, notes that a common pitfall in new practices is not having the IRB reapprove study documents in a timely fashion. Regulations require that study documents must be reapproved and redated annually, actually 1 day less than 1 year, rather than every 365 days. However, because most IRBs meet once a month, documents need to be submitted every 11 months for this annual review or the site may miss the deadline by a day or a week, which constitutes a deviation. Yu also points out that updates and changes to consent forms need to be approved within 90 days. It's also important for sites to communicate with trial sponsors (eg, to know when accrual to a study may need to be halted). Timely reporting of adverse events (AE) is also important, and local IRB rules concerning AE reporting requirements need to be documented in writing.
Depending on the number of charts to be examined, audits may take 1 to 3 days. The make-up of the audit team varies with the entity conducting the audit. According to Good, Southwest Oncology Group audit teams generally include nurses or nurse CRAs. Frequently the physicians who participate are retired and want to keep involved,” she says. All cooperative groups work similarly, observes Grubbs. The Cancer and Leukemia Group B, the cooperative group with which he is most familiar, sends teams that include both physicians and CRAs.
Devine explains that when the auditors arrive, they need a place to sit. It should not be in a busy area, and should not be where other medical records are kept. If medical records are computerized, the site needs to make sure the audit team has a place to work with a computer. Someone from the site should be available to facilitate finding the necessary records (eg, laboratory results). Small offices that still rely on paper records would do well to flag those items they know the audit team will need to inspect. Yu points out that the personnel most familiar with the patients and the IRB should be available to assist the auditors. Although physicians involved with the trial aren't expected to be interacting with auditors during the audit, they should be available to speak with auditors afterward when the findings are reviewed. He notes that auditors like to have the source documents immediately available and organized. It can be confusing if the multiple practice sites participating in a given trial organize their charts differently. He recommends color coding charts with different colors for progress reports, laboratory test results, and imaging test results, to help speed things up. Good agrees that tabbing and color-coding is helpful. She conducts audits herself for Southwest Oncology Group and North Central Cancer Treatment Group, and finds that most auditors like having a site staff person available to help find things, although there are teams that don't want anyone with them during the process.
In the category of patient safety, audit teams will determine that all consent forms were approved by the IRB, contain all necessary elements, and were signed before the patients were enrolled and treated. Good says that any revisions or amendments to protocols requiring reconsent also should have been documented. She says that this documentation might be sent to the auditors ahead of time, depending on the cooperative group. Grubbs calls patient rights and protections the number one concern. It is rare for a study to be shut down, he says but serious weaknesses in this area could cause an immediate shut down until IRB failures are corrected.
Selected cases are audited to ensure integrity of the data. Typically, 10% of cases, with a minimum of 10 cases, are examined. Once at the site, the auditors may request additional cases to examine about which the site has not been notified in advance. Auditors determine that patients were eligible, source documentation (eg, case report forms) are present, and that AE are reported. They will also determine if laboratory tests and evaluations of response were performed at the appropriate times. The pharmacies used to supply the investigational drugs are inspected to determine if the appropriate drug accountability forms were used, that drugs were stored properly by study and by type, and that investigational drugs only were administered to appropriately enrolled patients.
Grubbs notes that auditors will check to see that investigational drugs provided by study sponsors were only used for the exact patients on a trial, and that on completion of the trial, leftover drugs were returned or destroyed as required, and that this is documented. Sometimes trials involve drugs that are already approved for a different indication. It is important to ensure that only the investigational supply provided for the study is used, not a commercially available formulation stocked in the pharmacy for patients not on the trial. Yu suggests that if a computerized drug ordering system is used, the investigational drug be clearly identified as a sponsored supplied medication in the ordering system if it is also commercially available. He observes that a common audit pitfall is not using the NCI's official drug accountability record forms. Each study needs its own form for each drug that is used. Auditors will also check the general security of the investigational drug storage area. The room or area should have restricted access, and be locked or secured.
When the audit is complete, the team will conduct an exit interview, meeting with the PI, lead clinical research coordinator, pharmacist—and as many others at the site working on the trials as possible—to talk about findings. This will be followed by a formal, written report. Yu says that although there is a limited opportunity to respond at the exit interview, the site must submit a written plan to correct any deficiencies that were noted during the audit.
An “acceptable” finding means there is very little or nothing wrong at the site. There might be some lesser deficiencies such as transcription or typographical errors, or something scribbled out instead of being crossed out with a single line and a date. “Major” deviations include a noneligible patient enrolled or elements missing from consent forms, or other findings that indicate there is not a good system in place to ensure patient safety or data integrity. Other major deviations include patients receiving an incorrect dose or a dose at the wrong time, response not being checked at the appropriate time, or other deviations that could affect patient care and the objectives of the study. Purposeful deviations or attempts to cover them up, says Good, could shut down a site.
“The worst thing you can do,” observes Devine, “is to be defensive. Findings should be used as a learning experience.” The appropriate response is to identify what the site can do to prevent the deviation from happening again, to avoid systems problems, and to improve performance. “Auditors are there to help improve processes,” Devine adds. Grubbs concurs that the learning process is most important: “Appreciate the audit team. They are there not to penalize us, but to help us improve.”
ASCO will hold conference calls in 2009 during which content providers to the series will discuss these topics in more detail and be available for discussion. See ASCO's Web site at http://www.asco.org/researchresourceswww.asco.org/researchresources for more information.
The next article in this series, to be published in the March 2009 issue of Journal of Oncology Practice, will cover cost neutral clinical research enterprise.