|Home | About | Journals | Submit | Contact Us | Français|
We sought to assess the feasibility of applying Cancer Care Ontario’s quality of care indicators to a single institution’s colorectal cancer (CRC) database. We also sought to assess their utility in identifying areas that require improvement.
We included patients who had surgery for CRC between 1997 and 2006 at Mount Sinai Hospital, Toronto, Ont. We excluded patients who had transanal excisions, carcinoma in situ or recurrences that required pelvic exenteration, as well as those whose information was incomplete. We obtained data from a prospective database and verified the data with hospital and office charts. We evaluated trends over a 10-year period using the Cochran–Armitage trend test.
During the study period there were 1005 surgical procedures performed in 987 patients with a mean age of 65.6 (standard deviation 15) years; the male:female ratio was 1:2. The most frequent tumour sites were the rectum and sigmoid colon (68%). Over the 10-year period, 9 indicators improved, including the proportion of patients with CRC identified by screening (p < 0.001), the proportion of patients who received preoperative liver imaging (p = 0.05), the proportion of rectal cancer patients who received preoperative pelvic imaging (p = 0.04), the proportion of patients with stage II or III rectal cancer who received radiotherapy (p = 0.03), the proportion of surgical specimens with more than 12 lymph nodes (p < 0.001), the proportion of pathology reports that included quantitative distal (p = 0.004) and radial (p < 0.001) margin measurements, the proportion of patients with an anastomotic leak (p = 0.03), the proportion of patients who received a colonoscopy 1 year after surgery (p < 0.001) and the proportion of operative reports that were complete (p < 0.001).
The use of quality of care indicators to assess the quality of colorectal surgery is feasible. This study provides benchmarks that can be used to assess changes in the quality of CRC care at our institution.
Nous avons voulu vérifier si les indicateurs de qualité des soins d’Action Cancer Ontario seraient applicables à la base de données sur le cancer colorectal (CCR) d’un seul établissement. Nous avons en outre tenté d’en évaluer l’utilité dans la reconnaissance des secteurs à améliorer.
Nous avons inclus les patients ayant subi une chirurgie pour CCR entre 1997 et 2006 à l’Hôpital Mount Sinai de Toronto, en Ontario. Nous avons exclu les patients qui avaient subi des excisions transanales, qui souffraient d’un carcinome in situ ou qui présentaient des récurrences nécessitant une exentération pelvienne; nous avons aussi exclu les patients pour lesquels nous disposions de données incomplètes. Nous avons obtenu les données d’une base de données prospective et nous les avons comparées à celles des dossiers de l’hôpital et des cabinets médicaux. Nous avons évalué les tendances sur une période de 10 ans à l’aide du test de tendance Cochran-Armitage.
Au cours de la période d’étude, nous avons recensé 1005 interventions chirurgicales, réalisées sur 987 patients âgés en moyenne de 65,6 (écart-type 15) ans; le rapport hommes:femmes était de 1:2. Les foyers tumoraux les plus fréquents étaient le rectum et le sigmoïde (68 %). Durant la même période de 10 ans, 9 indicateurs se sont améliorés, notamment, la proportion de cas de CCR mis au jour par le dépistage (p < 0,001), la proportion de patients ayant subi une épreuve d’imagerie hépatique préopératoire (p = 0,05), la proportion de patients atteints d’un cancer rectal ayant subi une épreuve d’imagerie pelvienne préopératoire (p = 0,04), la proportion de patients atteints d’un cancer rectal de stade II ou III soumis à la radiothérapie (p = 0,03), la proportion de spécimens chirurgicaux présentant plus de 12 ganglions lymphatiques (p < 0,001), la proportion de rapports de pathologie incluant les mesures quantitatives des marges distales (p = 0,004) et radiales (p < 0,001), la proportion de patients présentant une fuite anastomotique (p = 0,03), la proportion de patients soumis à une colonoscopie un an après la chirurgie (p < 0,001) et la proportion de rapports opératoires complets (p < 0,001).
Il est possible d’utiliser les indicateurs de qualité des soins pour évaluer la qualité de la chirurgie colorectale. Cette étude a produit des points de repère utilisables pour évaluer les changements de la qualité du traitement pour CCR dans notre établissement.
In Canada, colorectal cancer (CRC) is the second most common cancer-related cause of death in men and the third most common in women.1 Although surgery is the most effective treatment, there is great variability in outcomes among surgeons and institutions. Quality of health care is defined as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”2 However, it is often difficult to ascertain whether high-quality care based on best evidence is achieved in surgical practice.
In recent years, groups have published guidelines and recommendations aimed at improving outcomes in patients with CRC, including the American Society of Colorectal Surgeons,3,4 the American Society of Clinical Oncologist Recommendations,5–7 the National Cancer Institute8 and the National Comprehensive Cancer Network.9
Cancer Care Ontario has undertaken quality-improvement programs to improve outcomes in cancer surgery. In 2005, a multidisciplinary panel proposed 15 quality of care indicators that could be used to evaluate the quality of CRC surgery.10 These 15 indicators cover the following domains: patient presentation, preoperative work-up, quality of surgery and pathology, postoperative complications, long-term outcomes and postoperative follow-up.10
Our objectives were to assess the feasibility of using these quality of care indicators to assess the care of patients undergoing surgery for CRC. We also sought to determine if there were changes over the 10-year study period and to identify areas where improvement is required at our institution.
We included patients who had surgery for CRC at the Mount Sinai Hospital between January 1997 and December 2006. The diagnosis of CRC was made by histopathology examination. We excluded patients if they had a transanal excision (n = 106) or pelvic exenteration for recurrent disease (n = 22), if the cancer was in situ only (n = 50) or there was incomplete follow-up (n = 17). We obtained data from a prospectively maintained CRC database. We verified the information by use of hospital and office charts and by contacting the family physicians and patients. The study was approved by the Ethics Review Committee at Mount Sinai Hospital.
CT = computed tomography; MRI = magnetic resonance imaging.
The proportion of patients with colon and rectal carcinoma detected by screening refers to the proportion of patients who were identified by a screening test rather than by presentation with symptoms.
Patients were considered to have had adequate evaluation of their colon if either a barium enema and sigmoidoscopy or colonoscopy was performed 3 months before or 6 months after surgery. We excluded patients who died within 6 months after the surgical procedure.
We evaluated the proportion of patients who received preoperative pelvic computed tomography (CT) or magnetic resonance imaging (MRI) among patients with rectal cancer. We calculated the proportion of patients who received liver imaging among all patients with CRC. This indicator was considered to have been met if an abdominal CT scan, MRI or ultrasound had been performed before surgery.
The Cancer Care Ontario indicators refer to the proportion of rectal cancer patients who had a consultation with a medical or radiation oncologist either before or after operation if they had stage II or III cancer. However, it was not possible to determine if patients who did not receive either radiation or chemotherapy had a consultation with either of these specialists. Thus, we assessed only the proportion of patients who received chemotherapy or radiotherapy. These indicators were applied to patients younger than 85 years old with stage II or III rectal cancer. We included patients who received chemotherapy alone as adjuvant therapy after the operation or in conjunction with radiation before the operation.
We evaluated the operative report on a scale from 0 to 10, with 2 points given to each of the following features if they were mentioned in the report: (1) a total mesorectal dissection was performed, (2) details of the location of the tumour and margins were included, (3) extent of the resection (en-bloc resection and margins) was mentioned, (4) nerves were identified and (5) the extent of the lymphadenectomy was noted. We considered the latter feature to be noted if the level of the ligation of the inferior mesenteric vessels was documented.
We assessed the proportion of patients who had surgery for colon or rectal cancer in whom the number of lymph nodes that were examined and the number of lymph nodes that were positive were recorded. In addition, we evaluated the proportion of patients in whom more than 12 lymph nodes were examined.
The Cancer Care Ontario indicator is the proportion of patients for whom margin status is detailed. We considered the distal margin in rectal cancer to be positive if it was less than 1 cm. In addition, we evaluated the circumferential radial margin (CRM) status and considered the CRM to be positive if it was 1 mm or less or if the pathology report stated that it was positive.11 In addition, we evaluated the proportion of pathology reports that reported the margin status quantitatively. We excluded patients who had a nonresective procedure from the analysis of the CRM. We excluded patients who had end stomas from the analysis of distal margins.
This indicator was applied to patients with rectal cancers who had a resection and anastomosis. We included both symptomatic and asymptomatic leaks detected by contrast studies. Clinical leaks were defined by the presence of peritonitis or other symptoms caused by anastomotic dehiscence and that were corroborated with abdominal CT scans, contrast enema studies or operative findings. Asymptomatic leaks were confirmed by either abdominal CT scan or contrast enema. Routine imaging to detect leaks was only performed in patients who had a defunctioning stoma constructed at the first operation.
Operative mortality was defined as death following colon or rectal cancer surgery occurring in hospital or within 30 days of discharge.
Local recurrence following rectal cancer surgery was defined as any recurrence within the pelvis that was confirmed by histology, diagnostic imaging, colonoscopy and/or biopsy at reoperation. Similarly, for colon cancer, local recurrence was defined as a recurrence near the anastomotic site and was confirmed as above. Patients who had surgery for any stage of cancer, including stage IV cancer, were included in this analysis. We calculated local recurrence rates at 2 years so that the follow-up period was the same for all patients and the rates over time could be compared. We excluded patients who died after operation or who had surgery between 2005 and 2006 and thus had less than 2 years follow-up.
The 5-year disease-free survival rates include all patients who had surgery between 1997 and 2002, including those who died immediately after operation.
We excluded patients who had bypass procedures, total proctocolectomies, stage IV tumours, were older than 85 years or died within the first year after operation in the assessment of the proportion of patients who underwent a 1-year postoperative colonoscopy.
We determined the proportion of patients who fulfilled each of the quality indicators each year and compared the proportions by 2-year periods. We tested trends of increasing or decreasing proportion of patients who met the indicators over 10-year periods using the Cochran–Armitage trend test. This test examines not only if there is a difference in proportions but also whether there is a significant increase or decrease over time. We analyzed the cancer recurrence rates using Kaplan–Meier product limit curves, and we performed nonparametric survival analysis using the log-rank test for differences in recurrence over strata (e.g., stage). We performed the statistical analyses using SAS version 8 (SAS Institute). All statistical tests were 2-sided.
We included 547 men and 440 women (male:female ratio 1:2) with a mean age of 65.6 (standard deviation [SD] 15, range 23–107) years. In total, 51% of tumours were located in the colon and 49% in the rectum. There was no significant change in location over time (p = 0.90). Of the tumours, 21% were stage I (n = 211), 30% were stage II (n = 295), 31% were stage III (n = 306) and 18% were stage IV (n = 175). Follow-up was complete in 99.3% of patients. The mean follow-up was 2.7 (SD 2.5) years, and the median follow-up was 2.1 years. Seventy-nine percent of patients were followed to 2 years.
Table 1 presents the proportion of patients who met each of the indicators over the 10-year period and by 2-year intervals. There was a significant improvement in 9 indicators over the 10-year period, including the proportion of patients with CRC identified by screening (p < 0.001; Fig. 1), patients having liver imaging (p = 0.05), rectal cancer patients who underwent preoperative pelvic imaging (p = 0.04), patients with stage II or III rectal cancer who received radiotherapy (p = 0.03), surgical specimens with more than 12 lymph nodes (p < 0.001; Fig. 2), pathological reports that indicated quantitative distal (p = 0.004) and radial (p < 0.001) margins (Fig. 3), patients with anastomotic leaks (p = 0.03), patients who received a postoperative colonoscopy at 1 year after surgery (p < 0.001), as well as the completeness of operative reports (p < 0.001). The proportion of patients who presented with stage I and II cancer increased during the period while the proportion with stage III and IV cancers decreased (Fig. 4).
The number of lymph nodes that were identified was indicated in all pathology reports throughout the 10-year period. The proportion of surgical specimens in which more than 12 lymph nodes were identified increased from 49% to 88% (p < 0.001), with a mean number of 18 (SD 10) lymph nodes per specimen. The proportion of pathology reports that quantified the size of the radial margin increased from 52% to 83% (p < 0.001), and the proportion that quantified the size of the distal margin increased from 82% to 90% (p < 0.004).
Anastomotic leaks occurred in 16 patients (2.1%): 11 occurred following rectal cancer procedures (3.3%) and 5 following colon cancer procedures. There were 11 symptomatic and 5 asymptomatic anastomotic leaks. All of the asymptomatic leaks occurred following rectal procedures. Overall leaks and rectal leaks decreased from 3.8% to 0.6% (p = 0.049) and from 9.2% to 1.5% (p = 0.03), respectively. Four out of 16 patients who had leaks had a local recurrence. Only 1 patient with an anastomotic leak died after operation.
The overall local recurrence rates at 2 years decreased from 9.2% between 1997 and 1998 to 7.0% between 2003 and 2004. However, this reduction was not statistically significant (p = 0.42). The local recurrence rate for rectal cancer was 6.4%, although there was a nonsignificant decrease from 9.1% to 6.3% in the latter period analyzed (p = 0.50). The local recurrence rate for colon cancer was 6.6% with a range from 9.3% between 1997 and 1998 to 7.7% between 2003 and 2004 (p = 0.81) (Table 1).
The overall survival in this cohort was 75%, with 5-year survival rates of 85% for stage I, 75% for stage II, 67% for stage III and 12% for stage IV cancer (Fig. 5). There was a trend toward better survival over the 6-year period in each stage (Table 2).
There is wide variation in outcomes following many surgical procedures. These differences have been documented by use of institutional- and population-based data and have been found to be present in most developed countries. Performance measurement has become a common exercise for health care organizations interested in assessing and reporting the quality of services using quality of care indicators.12 A fundamental requirement for assessing quality-improvement initiatives is the use of valid, clinically relevant and sensitive indicators.
The Surgical Oncology Program of Cancer Care Ontario was created to coordinate the delivery of cancer surgery and to improve its quality.13 A number of quality initiatives have been implemented, including the development of clinically relevant indicators to assess the quality of care and provide feedback to hospitals and regional networks.
In 2002, a multidisciplinary expert panel was convened to develop indicators for measuring the performance of CRC surgery using a modified Delphi process.10 The panel included 15 individuals who were considered experts in CRC surgery. Initially, a literature search was conducted to identify quality indicators that had been used by other groups and to identify possible indicators based on best evidence from published meta-analyses, systematic reviews, guidelines and consensus statements. Forty-five indicators were presented to the panel for review. Through a series of questionnaires, a final set of 15 indicators was chosen (Box 1).
Although the indicators were developed in 2003 and published in 2005, there has been no study to date assessing the feasibility of using them to assess the quality of CRC surgery in Ontario.
In this study, we used data from a CRC database that is maintained at the Mount Sinai Hospital. We collected data for 13 of the 15 indicators. Information about whether patients are referred to a medical or radiation oncologist is not collected in the Mount Sinai Hospital database. Instead, for these indicators, the proportion of patients with stage II or III rectal cancer who received radiation or chemotherapy is reported. The rates were high — 78.7% and 89%, respectively — and presumably would have been higher had we included individuals who were referred for radiation or chemotherapy but did not receive it.
Overall, based on the quality indicators that were used, care at our institution is as good as or better than that reported in the literature. Our 5-year survival rate was high and comparable to the rates observed in recent rectal and colon cancer trials.14,15 The postoperative mortality rate over the 10-year period was 2.9% and was less than 5% at all time points. The reported in-hospital mortality rate following large-bowel resection for carcinoma ranges from 0.8% to 13.4%, depending on factors such as patient age, comorbid illnesses, tumour site, and whether the procedure was performed electively or emergently.16–21 Because of the relatively small sample size, we did not risk adjust our data.
Our data also compare favourably to those from other institutions in Ontario. Simunovic and colleagues,16 using data from the Ontario Cancer Registry, reported a mean provincial postoperative mortality rate of 4.5% for colonic cancer procedures. In our study, the leak rate was 2.1% overall and 3.7% for rectal surgery. Anastomotic leak rates between 0.5% and 39% have been reported.22,23
There was a high rate of assessment of pathological indicators. In total, 95% of distal margins were greater than 1 cm, 87% of patients with rectal cancer had negative radial margins, and more than 12 lymph nodes were examined in 61.2% of specimens. In a population-based study of 116 995 patients with colon cancer in the United States, Baxter and colleagues24 found that only 37% of patients received appropriate lymph node evaluation, although the proportion increased from 38% in 1998 to 44% in 2001.
In the present study, 79% and 89% of patients who underwent surgery for rectal cancer received radiation and chemotherapy, respectively. The reported rates of chemo-and radiotherapy in CRC are extremely variable and range from 11% to 85% in reported series.25,26 The National Initiative for Cancer Care Quality in United States reported that 83% and 66% of patients with stage II or III rectal cancer received chemotherapy and radiotherapy, respectively.27 A Canadian study that included patients with stage III colon cancers and stage II or III rectal tumours who had a curative resection found that 85% of patients had a consultation with a medical oncologist or radiation oncologist, or both.28
Improvement in some of the indicators occurred over the 10-year period. For example, the proportion of patients whose cancer was detected by screening increased from 7.1% to 16.5%. As shown in Figure 1, the proportion increased dramatically in 2002 after publication of the recommendations of Canadian Task Force on Preventive Health Care for CRC screening in 2001.29 Concomitantly, the proportion of patients who presented with stage I or II cancer compared with stage III or IV cancer increased (Fig. 4). Whereas some of this change may be because of publication of the guidelines, other factors may also have played a role, but given the source of these data, it was not possible to ascertain the specific reason for the increase in cancer detected by screening.
There was also a noticeable increase in the number of lymph nodes identified in the surgical specimens. Ratto and colleagues30 reported that a change in the protocol for lymph node procurement increased the number of lymph nodes retrieved per specimen from 11.3 to 29.6. Other authors have found that the use of a standardized pathology form increased the median number of lymph nodes evaluated per specimen.31 At our institution, the proportion of specimens with more than 12 lymph nodes identified increased from 49% to 88% over the 10-year period. The greatest increase was observed after adoption of a synoptic pathology report for CRC in 2001. At the same time, the proportion of specimens in which the distal and radial surgical margins were measured quantitatively increased. Whether this improvement was because of the adoption of a synoptic pathology report is only speculative. However, improvement in the quality of pathology reporting after adoption of a standardized pathology report has been reported by others. Rigby and colleagues32 demonstrated that the use of a standardized pathology form led to improvement in the reporting of circumferential resection margins and apical node status. Beattie and colleagues31 found that the introduction of a standardized pathology form improved the reporting of the grade of histological differentiation, the extramural vascular invasion, the median number of lymph nodes, the apical node involvement, the adequacy of resection, the distance of the tumour to the dentate line and the relation of the peritoneal reflection.
Whereas there was improvement in 9 of the 15 indicators over the 10-year period, the proportion of rectal cancer specimens with positive radial margins actually increased over time. Overall, the CRM positivity rate was 13%, but it increased from 6% in 1997–1998 to 15% in 2005–2006. It is not obvious why this occurred. Over that period of time, the proportion of patients who received radiation therapy increased and there was also a shift toward neoadjuvant therapy rather than postoperative radiation. One possible explanation is that the complexity of the surgical procedures performed in recent years may have increased. Alternately, it might be because of increased attention paid by pathologists in examining the CRM. This is plausible because the proportion of specimens with quantitative radial margins increased from 52% to 83% during this period. The CRM positivity rate at our institution is in keeping with those reported in the literature. In the Dutch total mesorectal excision trial,33 the rate was 13%, although others have reported rates as low as 6% using preoperative MRI.34 We are not discounting a true increase in the CRM positivity rate, and further evaluation to identify reasons for the change is being undertaken. Quality-improvement initiatives may be required, thus emphasizing the value of quality of care indicators even at an institutional level. A positive CRM status has been shown to be correlated with the rate of local recurrence.35–37 The overall isolated local recurrence rate at 2 years in rectal cancer in our series was 6.4%. Despite there being an increase in the rate of positive circumferential radial margins, there was no statistically significant increase in the 2-year local recurrence rates over time. However, we only report 2-year rates, and these rates may increase with further follow-up.
The mean score for the operative reports was high (8.7 on a scale from 0 to 10) and improved over time. However, we awarded points if the criterion was mentioned even if the specific details were not included. Thus, this is an area in which there could be improvement. The introduction of standardized synoptic operative reports has been shown by others to lead to improved operative reports and possibly improved surgical outcomes.38
An important question is whether improvement in these indicators leads to improved patient outcomes. Despite an improvement in 9 of the 15 indicators over the 10-year period, we found no significant change in survival, which is the usual measure used to assess outcomes of colorectal and rectal cancer surgery. A trend toward improvement was observed, and we may not have detected a significant difference because of the relatively short follow-up and the size of the cohort. As well, the quality of surgery was high, as measured by these indicators, at the beginning of the study; this may be another reason why we did not observe an improvement in survival. Whereas survival is a well-accepted outcome for assessing the success of cancer surgery, there are other clinically relevant outcomes that are important to patients. As well, outcomes such as rates of anastomotic leaks and local recurrences are important and showed significant improvement over the 10-year period.39–42
The purpose of this study was not to compare changes before and after the publication of the indicators because the indicators were only published in 2005. However, our study confirms that it is feasible to apply the quality of care indicators to assess outcomes following CRC surgery using a single institution’s database. This study also provides benchmarks that can be used to evaluate changes in CRC care at our institution. Comparisons with other institutions in Ontario are not possible because the data came from a single institution. Further studies using population-based data will be required to for this type of comparison. Such initiatives are currently being implemented by Cancer Care Ontario.
Our results suggest that the Cancer Care Ontario indicators can be used as quality indicators to assess the quality of CRC surgery at individual institutions. The results from this study are being used as benchmarks for ongoing quality-improvement initiatives at our institution. However, although we have shown that some clinically relevant outcomes (local recurrence rates, anastomotic leak rates) have improved over time, further studies are required to ensure that these quality indicators correlate with other clinically relevant outcomes such as survival. Thus, we have provided evidence that it is feasible to use these indicators to assess quality of care and that they appear to be responsive to changes incurred following the introduction of quality-improvement initiatives; it is also feasible to use these indicators to identify areas in which quality of care after CRC surgery may be improved.
Competing interests: Dr. Vergara-Fernandez was supported by funds provided by Colon Cancer Canada and La Fundación Mexicana para la Salud. Dr. Swallow has received honoraria and travel assistance from Novartis Oncology. None declared for Drs. Gryphe, MacRae, Cohen and McLeod and Mr. Victor and Ms. O’Connor.
Presented at the Canadian Society of Surgical Oncology Meeting, Toronto, Ont., Apr. 27, 2007.
Contributors: Drs. Vergara-Fernandez, Swallow, MacRae and McLeod and Ms. O’Connor designed the study. Drs. Vergara-Fernandez and McLeod and Ms. O’Connor acquired the data, which Drs. Vergara-Fernandez, Swallow, Gryphe, Cohen and McLeod and Mr. Victor analyzed. Drs. Vergara-Fernandez, Swallow and McLeod wrote the article. All authors reviewed it and approved its publication.