|Home | About | Journals | Submit | Contact Us | Français|
This study describes the development, implementation and management of a multi-faceted quality assurance program called Quality Assurance for Aboriginal Medical Services (QAAMS) to support point-of-care HbA1c testing on the Bayer DCA 2000 in Aboriginal people with diabetes from 45 Australian Aboriginal Community Controlled Health Services.
The quality assurance program comprised four elements: production of culturally appropriate education resources, formal training for Aboriginal Health Workers conducting HbA1c testing, an external quality assurance program and on-going quality management support services including a help hotline and an annual workshop. Aboriginal Health Workers were required to test two quality assurance (QAAMS) samples in a blind sense every month since July 1999. Samples were linearly related and comprised six paired levels of HbA1c. The short and long term performance of each service’s DCA 2000 was reviewed monthly and at the end of each six month testing cycle.
The average participation rate over 7 six-monthly QAAMS testing cycles was 88%. 84% of 3100 quality assurance tests performed were within preset limits of acceptability. The median precision (CV%) for HbA1c testing has averaged 3.8% across the past 5 cycles (range 3.4 to 4.0%) and is continuing to improve. The introduction of a medical rebate for HbA1c testing has ensured the program’s sustainability.
Through continuing education and training, Aboriginal Health Workers have achieved consistent analytical performance for HbA1c testing on the DCA 2000, equivalent to that of laboratory scientists using the same instrument. This unique quality assurance model can be readily adapted to other Indigenous health settings and other point-of-care tests and instruments.
Type 2 diabetes is a significant cause of morbidity and mortality among Aboriginal Australians. Aboriginal people suffer between 12 to 17 times more deaths due to type 2 diabetes than non-Indigenous Australians, while overall prevalence rates are within the range of 10–30% and at least two to four times that of the non-Aboriginal population.1,2
Studies such as the Diabetes Control and Complications Trial (DCCT) and the United Kingdom Prospective Diabetes Study (UKPDS) have confirmed that good control of diabetes is critical to prevent the long-term debilitating complications of this disease such as nephropathy, neuropathy and retinopathy.3,4 Haemoglobin A1c (HbA1c) is a well-established biochemical marker of diabetes control and can be conveniently measured on the small, portable DCA 2000 point-of-care analyser (Bayer Diagnostics, Melbourne, Australia).
In 1998, the National Diabetes Strategy and Implementation Plan, Australia recommended that a trial of the DCA 2000 for HbA1c testing be conducted in Aboriginal primary health care services.5 The DCA 2000 was considered to have particular application in the Aboriginal health care setting, notably in rural and remote locations, where access to basic health care is often limited and services may be several hundred kilometres from the nearest hospital laboratory.6
In June 1999 the Australian Government’s Department of Health and Ageing, through its Office for Aboriginal and Torres Strait Islander Health (OATSIH) and in partnership with the National Aboriginal Community Controlled Health Organisation (NACCHO), commenced a pilot program for on-site HbA1c testing using the DCA 2000. Forty-five Aboriginal Community Controlled Health Services (ACCHS) around Australia participated in the pilot. Aboriginal Health Workers (Aboriginal people trained in primary health care and living in the community setting) performed the testing on behalf of their local or visiting medical officer. Testing was only conducted on Aboriginal people with known diabetes, with a maximum of four HbA1c tests being performed annually on each person. An HbA1c of 7% was considered the target for good control of diabetes in Aboriginal people.6
As a requirement of the pilot, the Department of Health and Ageing determined that an on-going surveillance mechanism was needed to ensure point-of-care HbA1c results generated in the field were acceptable for patient care. As a result, the RCPA Quality Assurance Programs Pty Ltd was initially contracted by the Department of Health and Ageing to develop, implement and manage a multi-faceted quality assurance program to support HbA1c testing by Aboriginal Health Workers. (The RCPA Quality Assurance Programs Pty Ltd provides quality assurance programs to laboratories in Australasia in collaboration with the Royal College of Pathologists of Australasia and the Australasian Association of Clinical Biochemists). The Department of Health and Ageing (through its Diagnostics and Technology Branch) have subsequently extended this contract to the end of 2005 and the program has now been integrated into mainstream Aboriginal health care.
The quality assurance program developed by our group is believed to be the first of its type for Indigenous people anywhere in the world. We describe the novel aspects of this program, present the scientific results obtained over its first three and a half years of operation, and comment on the potential of this model to assist chronic disease prevention and management in other Indigenous countries and communities.
The DCA 2000 point-of-care analyser is small (25cm high by 21cm wide by 25cm deep) and portable (weighing 5 kg). Only one microlitre of whole blood (capillary or venous) is required for the test, with the result available in 6 minutes. The DCA 2000 determines both the concentrations of HbA1c and total haemoglobin, with the ratio being expressed as the percentage HbA1c. HbA1c is measured immunochemically by a method based on inhibition of latex agglutination. Total haemoglobin is measured spectrophotometrically following its oxidation to methaemoglobin by potassium ferrocyanide and the subsequent complexing of methaemoglobin with thiocyanate to form thiocyan-methaemoglobin.7
Forty-five Aboriginal Community Controlled Health Services formed the initial intake of program participants (Figure 1). Every State and Territory of Australia was represented and the location of participants encompassed a broad spectrum of urban, rural and remote Australia. There have been minor changes to the mix of participants over the past three and a half years due to changing health service priorities and/or Aboriginal Health Worker staff resources. Fifty sites were enrolled in the program at the beginning of 2003.
The quality assurance program to support HbA1c testing in the Aboriginal community was called QAAMS (Quality Assurance for Aboriginal Medical Services). As tests were being conducted using a point-of-care medical instrument in the community setting by Aboriginal Health Workers, the design of the program needed to be broader and more intensive in its focus than a conventional laboratory-based quality assurance program. Four key elements, namely education, training, quality assurance, and on-going management support services for communities, formed the cornerstone of the QAAMS program.
A culturally appropriate set of education resources was developed in conjunction with senior Aboriginal health representatives, Aboriginal Health Workers, diabetes specialists and scientists. The resources comprised a laminated book, video and a series of posters. Topics covered included diabetes and its complications, the importance of controlling diabetes, the HbA1c test, the DCA 2000 machine and the principles and practice of quality control and quality assurance. The posters included step-by-step guides on how to perform the HbA1c test on the DCA 2000 and how to conduct quality assurance testing on the DCA 2000.
Formal training sessions for 84 Aboriginal Health Workers (and other allied health professionals) from every participating site were held at eight regional centres around Australia over an eight week period from May to July 1999. Training focussed on hands-on practical instruction in how to use the DCA 2000 machine, perform the HbA1c test, and conduct quality assurance testing. Each service also received their education resource kit at the training session. Once back in the community, these resources were used as a refresher course, to train other Aboriginal Health Workers, or for general community education about diabetes.
A quality assurance program commenced in July 1999 to monitor the performance of DCA 2000 machines across all participating sites. Since that time, 7 six-monthly testing cycles have been completed to the end of 2002.
At the beginning of each year, all participants were sent a package containing the materials needed to conduct their QAAMS testing. The package included a testing kit containing 24 lyophilised QAAMS samples, each numbered and dated, vials of water for reconstitution of those samples, an information sheet, testing schedule and result sheets. An international Reference Laboratory for Glycohaemoglobin produced the samples for the program. The samples comprised paired and linearly related levels of HbA1c across a range of concentrations from 5 to 14%. Services were required to test two QAAMS samples per month, according to the defined testing schedule that comprised two six-monthly testing cycles per year. Testing could be performed at any time during the month that was convenient to the service, provided the results were received at the QAAMS reporting office by fax or post by the last day of each month. To ensure confidentiality of results, each service was allocated a specific community code number that was stamped on their result sheets and all other printed matter.
Within a week after the closing date for return of results for each month, the QAAMS reporting office sent a graphical report to each individual service (Figure 2). The report compared the results returned by that service with the pre-set target values for each sample and with the range of results from all other services. In addition, the report displayed graphs that documented all previous results returned by the service for that particular testing cycle. Through this method of reporting, each service received regular monthly information on the short and long term performance of their DCA 2000 machine.
At the end of each cycle, summary analysis of six months of data (12 specimens) was used to comment on each individual site’s precision and accuracy. This data was used to identify individual services that were experiencing significant analytical problems. Services were contacted and an action plan implemented to redress poor performance.
A help hotline telephone service was established whereby Aboriginal Health Workers could phone the QAAMS office immediately a problem arose, particularly in relation to instrument breakdown or other technical problems and interpretation of quality assurance results. A regular newsletter was also sent to all participants to update them on current issues. Workshops for participants were held in November 2001 and September 2002 and have now become an annual feature on the program’s calendar. The workshop provides Aboriginal Health Workers with opportunities for retraining, networking, interactive discussion on all aspects of the program and presentations from participants on how their service is using the program. Representatives from the Department of Health and Ageing and Bayer Australia also attend the workshop. In addition to external quality assurance assessment, the QAAMS program also monitored internal quality control testing conducted by services using kits provided by the manufacturer (Bayer Normal and Abnormal Quality Control kits, catalogue number 5068001, Bayer Australia).
An overall participation rate of 88% (range 81% to 93%) has been maintained across the first three and a half years (7 testing cycles) of the program. This represents the return of 3100 QAAMS results from a possible 3524. On average, 81% of participating services returned between 10 and a maximum of 12 results each cycle.
Across three and a half years of testing, the percentage of returned results that were within preset limits for acceptability was 84% (range 81% to 86%) (Table 1). This represents 2590 acceptable results from 3100 results returned. The limits of acceptability, called allowable limits of performance, were set by the QAAMS program organisers at ±0.5 up to an HbA1c of 10% and ±5% at HbA1c ≥10%. These are the same limits used for the parallel Glycohaemoglobin Quality Assurance Program run by the RCPA Quality Assurance Program Pty Ltd for laboratories in Australasia and other parts of the world.
As stated earlier, the 12 samples in each cycle comprised six paired levels of HbA1c concentration that were tested in random order pre-determined by the program organisers.
The use of paired and linearly related samples for QAAMS testing also enabled a direct measure of the precision and accuracy across time of both individual DCA 2000 machines and the group as a whole. The median coefficients of variation (CV%) and bias achieved by services participating in the program is shown for each testing cycle in Table 1. The median precision (CV%) for HbA1c testing has averaged 3.8% across the past 5 testing cycles, with 3.4% being recorded for the most recent testing cycle. Notably the precision achieved by QAAMS participants has been steadily improving since the program began.
How does this level of performance compare to laboratories using the DCA 2000? The RCPA Quality Assurance Programs also runs a laboratory-based Glycohaemoglobin Quality Assurance Program, which uses the same material as that for the QAAMS program. There are approximately 75 DCA 2000 laboratory users in this program. The comparative analytical precision (median CV%) recorded for the QAAMS and the laboratory-based Glycohaemoglobin programs since the QAAMS program began shows that, for the past four testing cycles in particular, Aboriginal Health Workers achieved an analytical performance equivalent to that of trained laboratory scientists and technicians (Figure 3).
How does this performance compare to international consensus criteria on desirable performance standards for HbA1c analysis? The importance of imprecision as a key component of HbA1c analysis is being increasingly recognised by the medical and scientific community world-wide. Based on clinical need from studies like the UKPDS and DCCT and recent recommendations from key professional bodies, there is now widespread international consensus that the desirable precision goal for HbA1c analyses is a CV of less than 3%.8–13 While there is little specific published information on desirable performance standards for point-of-care HbA1c analysis, it is a generally accepted principle that analytical performance goals for point-of-care testing should be close or equivalent to those used for laboratories. In the most recently completed QAAMS testing cycle (Cycle 7), the point-of-care DCA 2000 analysers in the QAAMS program achieved a median CV of 3.4%, with 43% of QAAMS sites attaining the desired precision goal of less than 3% during this cycle. Whilst acknowledging that the point-of-care DCA 2000 analyser cannot achieve the performance base of state-of-the-art laboratory HPLC technology, the DCA 2000 is unquestionably the most practical and robust HbA1c analyser for use in rural and remote Australia where access to laboratory services is often very limited or non-existent.
A total of 419 calls were received by the QAAMS help hotline over the past three and a half years, 49% of which were taken during the first year of the program as teething problems and issues of process were being addressed. In addition to incoming calls, the QAAMS Program Manager provided on-going advice and follow-up action for sites whose DCA 2000 machines exhibited poorer performance.
1235 Bayer HbA1c quality control results were returned to the QAAMS reporting office across the three and a half years of the program. As up to seven different lot numbers of Bayer quality control material were being used by QAAMS participants at any one time, it has not been possible to conduct detailed statistical analysis of Bayer quality control results. In the most recent cycle of testing, 95% of all Bayer quality control results returned were within the acceptable limits for the particular lot number of control tested (with the limits of acceptability the same as those used for the testing of QAAMS samples). During 2003/4, a single lot number of Bayer HbA1c control will be sourced for use in the program.
In December 2000 Australia's Federal Health Minister announced that a rebate could be claimed under the government's Medical Benefits Schedule (MBS) for HbA1c tests conducted on the DCA 2000 in Aboriginal Community Controlled Health Services. Provision of this rebate has facilitated the program's integration into mainstream Aboriginal health care by ensuring there was a long-term sustainable funding mechanism for the program. The rebate remains conditional on services continuing to participate in the QAAMS program.
The charter of the QAAMS Program has been to provide education, training and quality management services to support community HbA1c testing rather than to collect or analyse patient data, which remains the property of the participating services. However, participant discussion at QAAMS workshops has revealed HbA1c testing is being carried out in a variety of different community settings. These include the diabetic clinic (with some services establishing new diabetic clinics as a result of being able to perform the test on-site), opportunistic testing in the health service, home visits for patients with diabetes who are unable to attend clinics, community functions and health promotion activities, and field visits to outstations and distant communities serviced by health services.
Further, data collected independently by NACCHO in the first 18 months of the program indicated 2315 Aboriginal people with diabetes were monitored using the DCA 2000 during this period.14 Subsequently data derived from reagent consumption figures indicate approximately 3000 HbA1c tests were performed in each of the second and third years of the program.
Significant advances in medical technology over the past decade have seen the development of point-of-care instruments such as the DCA 2000 that can perform tests for the early detection and management of chronic diseases in the community. Apart from advantages such as portability and small sample size, the DCA 2000 has other advantages that make it particularly applicable to the Aboriginal health care setting.15 Through appropriate training, Aboriginal Health Workers can perform HbA1c tests, thereby empowering them to take greater responsibility for the health of their own community members. By conducting the tests on-site, ownership and control of health information remains with the community, a factor crucial to the acceptance and success of Indigenous health programs. Immediate availability of a result means that the Aboriginal patient with diabetes does not have to return for a follow-up visit.
The usefulness of the DCA 2000 machine for the early detection of renal disease in a single remote Aboriginal community has been previously demonstrated.15–17 However, the QAAMS program represents the first time the DCA 2000 instrument has been used in a multi-centred, nation-wide program for monitoring control of diabetes in Aboriginal people.
In March 2001 NACCHO released an independent evaluation on the first 18 months of the QAAMS program.14 The executive summary of this report viewed the use of the DCA 2000 point-of-care technology as a major opportunity to better care for and manage Aboriginal clients with diabetes within the community setting. It stated that the ability of the point-of-care technology to generate rapid results served as a catalyst to enhance patient self-management, while simplicity of use led to high levels of acceptance by Aboriginal Health Workers nationally. It concluded that the sense of community control was enhanced as a result of management of diabetes becoming more focussed within Aboriginal medical services.
Participation rate and all key performance indicators measured in the QAAMS program have remained constant across the program's three-and-a half year duration. Tight precision for HbA1c measurements in patients with diabetes is particularly important because clinicians monitor serial measurements across time and adjust treatment according to changes in control of diabetes. If the analytical imprecision of the HbA1c method is too wide, it may mask a clinically significant change in control of diabetes. With appropriate education, training and on-going support, Aboriginal Health Workers have shown they can consistently achieve a level of analytical precision for HbA1c testing using the DCA 2000 point-of-care analyser that is equivalent to that of laboratory scientists and technicians using the same instrument. Further, the analytical performance base achieved by QAAMS participants using point-of-care technology is close to current internationally recommended precision goals for HbA1c. We believe this is reflective of the program’s intensive on-going commitment to continuing education, training and support for the participating services. Given the geographic isolation of many Aboriginal health services and their often poor access to laboratory services, the point-of-care DCA 2000 clearly represents the most practical and culturally appropriate option for community-based on-site HbA1c measurements on Aboriginal patients with diabetes.
The greatest challenge to the sustainability of the QAAMS program is Aboriginal Health Worker turnover in participating sites and the provision of education and training for new health workers who come into the program from those sites. Since the program's inception in June 1999, 33 services now have a different health worker responsible for the QAAMS program; in 10 of these services there have been more than one health worker change across this period. It is a remarkable testament to the goodwill and vision of the participating services and to the dedication and commitment of the Aboriginal Health Workers concerned that the program's major performance indicators have remained stable across this period in spite of such change. The QAAMS workshop provides at least one opportunity for formal training in a group environment every year.
The QAAMS HbA1c model, based on education, training, quality assurance and on-going quality management support services, provides a sound framework for the appropriate and sustainable use of point-of-care technology outside the laboratory environment. The model can be readily adapted and is transferable to both Indigenous communities in other parts of the world and to other point-of-care instruments and tests. In 2003 an international participant from the Western Pacific has joined the QAAMS HbA1c program. With further Government support, a new QAAMS program for the measurement of urine albumin:creatinine ratio on the DCA 2000 commenced in February 2003, thereby maximising the analytical capabilities of this point-of-care analyser and facilitating further on-site management for Aboriginal people with diabetes.
The QAAMS program was made possible by funds from OATSIH (1999–2000) and the Diagnostics and Technology Branch of the Commonwealth Department of Health and Ageing (2001 to the present). The support of Bayer Australia, particularly from Linda Walsh, Stuart Moody, Dean Whiting and Alison Halfnights, is gratefully acknowledged. Lloyd Penberthy, Business Manager of the RCPA Quality Assurance Programs Pty Ltd, is thanked along with the company's Board of Directors for their commitment to the program. Most importantly, as organisers of the QAAMS program, we thank NACCHO and all the participating services for their vision in taking up this program and making it work so successfully at the grassroots level.