|Home | About | Journals | Submit | Contact Us | Français|
Older adults in small residential board and care facilities are at a particularly high risk of fire death and injury because of their characteristics and environment.
The authors investigated computer-based instruction as a way to teach fire emergency planning to owners, operators, and staff of small residential board and care facilities. Participants (N = 59) were randomly assigned to a treatment or control group.
Study participants who completed the training significantly improved their scores from pre- to posttest when compared to a control group. Participants indicated on the course evaluation that the computers were easy to use for training (97%) and that they would like to use computers for future training courses (97%).
This study demonstrates the potential for using interactive computer-based training as a viable alternative to instructor-led training to meet the fire safety training needs of owners, operators, and staff of small board and care facilities for the elderly.
Older adults represent one of the highest fire risk populations in the United States (USFA, 2001). Fires are the fourth leading cause of accidental death for people age 65 and over (National Safety Council, 2003). Adults between 65 and 75 have a fire death rate twice that of the national average. Between ages 75 and 85, the risk grows to three times the national average. For adults over 85, the risk of death increases to four times the national average (Tridata Corporation, 1999).
Older adults present unique challenges in the field of fire safety. Changes associated with aging increase the likelihood of older adults accidentally starting a fire yet, at the same time, reduce their chances of surviving it” (USFA, 2001). Some of the characteristics that increase the risk of fire death and injury for older adults in residential board and care facilities include:
Research has shown that lack of training is one of the leading factors contributing to loss of life in residential board and care facility fires (Hall, 1993). Training is critical for owners, operators, and staff of small facilities because they are responsible for all facets of fire prevention and protection. Unlike large facilities that have maintenance staff, safety departments, and in-service educators, the owners and operators of small facilities must inspect for fire hazards, purchase/test/maintain fire protection equipment, and train their own staff and residents.
Prior to licensing, state agencies typically require a fire safety inspection, conducted either by licensing inspectors or the fire department. While an initial inspection may point out major deficiencies, such as an inadequate number of exits or lack of smoke detectors, these inspections do not ensure a continuing level of safety in the facility. For example, even if a home has an adequate number of exits, if the residents do not practice using all of the exits, they could be trapped if the primary exit route is blocked by fire or smoke. If caregivers are not trained how to identify and eliminate fire hazards, they can create situations which put residents at risk of fire death or injury.
Research has revealed the limitations of traditional instructor-led training with residential board and care owners, operators, and staff (Walker, Beck, Walker, & Shemanski, 1992; Walker, 1993; Walker, 1998; Walker & Harrington, 2004; Walker, Harrington, & Cole, 2006). Successful completion of instructor-led training required a large investment of time. Caregivers had to travel off-site to attend the workshop. Owners/operators of small facilities had difficulty getting away from their facilities to attend the training, sometimes they brought residents with them to the training; other times they arrived late or had to leave early. Another major impediment was the high staff turnover. New staff members could not receive immediate training because they had to wait until a group session was held.
Research has shown that well-designed computer-based training (CBT) can offer many advantages over traditional instructor-led workshops and video-tape presentations (Harrington & Walker, 2002; Harrington & Walker, 2003; Harrington & Walker, 2004; Harrington & Walker, 2006; Walker & Harrington, 2003; Walker & Harrington, 2004; Walker, Harrington, & Cole, 2006). Because most board and care facilities have few caregivers, they find it difficult to leave their residents for even a few hours to attend training classes. Immediate resident's needs come before training needs. CBT allows caregivers to complete the training at their own convenience. They can complete the training at the facility or at their own homes. CBT is not dependent on time of day or availability of instructors and so it is easier to schedule, especially for the many part-time and traveling caregivers.
The computer serves as the trainer and facility administrators or in-service educators do not have to take the time to prepare to teach the course. In addition, the average time required to complete training using a computer-based format has been shown to be significantly less than comparable training using an instructor-led format (Harrington & Walker, 2002; Harrington & Walker, 2003; Walker & Harrington, 2003).
Studies have shown that CBT is more motivating than conventional training and makes the learning experience more enjoyable (Harrington & Walker, 2004). Motivating caregivers to focus on the information being presented is often a difficult task for trainers. One advantage of CBT is that it engages the participant and forces participation. Because participants use visual, auditory, and kinetic senses, they are more involved in the learning process. In a study of the effectiveness of CBT with nursing facility staff, researchers found that CBT was more motivating and significantly increased participants' enthusiasm towards fire safety training as compared to instructor-led training (Harrington & Walker, 2004).
CBT is more cost-effective than instructor-led training because there is no need to pay an instructor, rent a room, or pay for travel. When the CBT is available at a Web site, caregivers can go on-line to complete a course. Cost is particularly important to small residential board and care homes with limited budgets for training.
CBT also allows new staff members to receive training without waiting for a group session. Providing immediate fire safety training is essential for homes with high staff turnover rates. If new staff do not know how to correctly react in a fire emergency, lives can be lost.
In a 2003 study, Walker and Harrington demonstrated that CBT was an effective training tool for board and care owners, operators, and staff. In a pilot test of a fall prevention program, 78 participants were randomly assigned to a computer-based or instructor-led group. Both groups completed falls prevention training with identical content, but in different formats: CBT vs. instructor-led. Over half of the participants were direct caregivers; the others were owners and administrators of small facilities. About half of the participants had previously used a computer. The test instrument measured knowledge, attitudes, and practice and was administered prior to and after the instruction to all participants. Both groups significantly improved their knowledge of fall prevention from pretest to posttest. In addition, members of the computer-based group significantly outperformed the instructor-led group on the knowledge subtest F (1, 76) = 4.5, p = .038 (Walker & Harrington, 2003). In a similar study of an injury prevention program, a group of board and care staff (N = 57) were randomly assigned to a computer-based or instructor-led group. Both groups improved significantly from pre- to posttest. In this study there were no between group differences (Walker, Harrington, & Cole, 2006).
Successful adoption of CBT in residential board and care facilities is dependent upon the availability of computers for training. Recent research findings indicate a growing number of small facilities not only have computers available for training, but are also connecting to the Internet with broadband services. In a 2003 injury prevention study involving caregivers from small residential board and care homes in Maryland, Walker and Harrington found that 71% (41) of the participants had computers with an Internet connection available at their facilities (Walker & Harrington, 2003). The type of Internet connection reported included 63% dial up modems, 35% broadband service (DSL, cable), and 2% (not known). In a 2005 injury prevention study, Walker and Harrington similarly found that 72% (109) of the participants in Virginia and Oklahoma had computers with an Internet connection available at their facilities (Walker & Harrington, 2005). The type of Internet connection reported included 51% dial up modems, 43% broadband service (DSL, cable), and 6% (not sure of type of connection). In the study reported in this paper, researchers found that two-thirds of the participants indicated that they had a computer available for training at their facility. The type of Internet connections reported included: DSL or cable (63%), dial-up modem (25%), not connected (5%), and don’t know (7%).
In the future, it is likely that more and more small facilities and their caregivers will have access to a computer, both in the facility and in their own homes. In 2003, the proportion of U.S. households with computers reached 62%, compared to 37% in 1997 (U.S. Department of Commerce, 2004). Computers are also available at local libraries and at department of social services buildings. Caregivers without computer access could complete the training in these computer labs where first-time computer users could receive help from a course facilitator. Caregivers with computer access could take advantage of distance learning via the Internet. The trend nationally is for states to provide information and resources to long term care facilities electronically, making it almost mandatory for small businesses such as board and care facilities to own or have access to a computer.
The purpose of the study reported in this paper was to evaluate the effectiveness of a computer-based fire safety training program with owners, operators, and staff of small residential care facilities for the elderly.
The design for the study was a two group comparison study with random assignment to groups. The 59 caregivers who completed the pretest were sorted by gender and then randomly assigned to one of two conditions: treatment or control. The paradigm for the study had the following configuration:
|Treatment||M1 (pretest)||X1 (training)||M2 (posttest)||M3 (follow-up)|
|Control||M1 (pretest)||no training||M2 (posttest)|
The interval between pretest and training/posttest was 2–3 weeks; the interval between training/posttest and follow-up was 3 weeks.
The research questions were:
Researchers developed a 45-minute computer-based module titled, “Fire Emergency Planning.” The content was adapted from a previously validated print/video training program titled The Fire Safety Workshop (Walker, 1990). The researchers updated the content with the assistance of an advisory committee consisting of a fire marshal, fire protection engineers, fire safety educators, state licensing agency members, and licensing inspectors. See Figure 1 for the program learning objectives.
The computer-based training program combined text, full motion video, illustrations, photographs, animation, and sound to provide a fully interactive, media-rich learning environment. The program had 54 screens, with 147 color photographs. Screen-to-screen navigation was designed so that users could move forward and back, repeat a topic, or quit the lesson. The program had interactive exercises throughout, where learners clicked on their answers by selecting a button, picture, or word.
Employees from the Department of Social Services in eight Maryland counties assisted researchers with recruitment. They sent a recruitment letter to each of the small residential board and care facilities licensed in their counties. The recruitment letter invited owners, operators, and staff to volunteer for the study and included a letter describing the project, a consent form, and a pretest.
In the three weeks prior to the training, 59 caregivers from 42 facilities completed the pretest at their facilities and mailed it back to researchers. Participants signed up for a training date and session time. The researchers conducted 10 training sessions over a two week period. Sessions were held in conference rooms or computer labs in local social services buildings or in local libraries. Session sizes ranged from 3 to 17 participants.
When participants arrived for the training, they were randomly assigned to either the treatment group (n = 31) or the control group (n = 28). Assignments were handled so that the 11 male participants were equally divided between the two groups.
The treatment group completed the Fire Emergency Planning module using laptops provided by researchers or computers that were part of a computer lab. Immediately after the training, treatment group participants completed a posttest and course evaluation form. The control group completed the posttest prior to the training, then completed the training module and course evaluation form. A member of the research staff monitored the training.
Three weeks after the training, researchers mailed a follow-up instrument to members of the treatment group. Treatment group participants completed the follow-up at their facility and mailed the instrument back to researchers.
Researchers adapted a previously validated instrument that measured the fire safety knowledge, attitudes, and practices of board and care owners, operators, and staff (Walker, Beck, Walker, & Shemanski, 1992). Each test item was criterion-referenced to a training objective, as well as to specific information in the computer-based training. The advisory committee reviewed all of the items for content validity. The final instrument consisted of 25 items: 6 knowledge items, 4 attitudes items, and 15 practices items. The posttest instrument included the same 25 items that appeared on the pretest with two changes. First, researchers presented the items in reverse order. Second, the 15 practice items were changed from the pretest version to indicate participants’ intended practices. For example, researchers revised the pretest item “I have identified two escape routes from each bedroom and living area in my residential care home,” to “I plan to identify two escape routes from each bedroom and living area in my residential care home.” The follow-up test included the same 25 items that appeared on the pretest.
The pretest form also requested the following demographic information from participants: name, date, job title, gender, ethnicity, age, and educational level. The pretest asked participants specific questions about their facility (facility ownership; number and characteristics of residents; availability of a computer; type of Internet connection) and their experience (previous use of computers for training; previous fire safety training).
The participant evaluation was administered anonymously at posttest. The form provided space for respondents to identify their gender and job title. Eleven items elicited the participants’ opinion about the usefulness of the program and the content.
At pretest, posttest, and follow-up, the participants selected one of three choices: Agree, Disagree, or Don’t Know for each item. Researchers entered participant data into an Excel spreadsheet. Items were coded in the spreadsheet and scored as correct or incorrect. In each case, Don’t Know or “no response” was scored as incorrect. The total number of correct responses by each participant was calculated. Subtest scores were calculated for Knowledge (6 items), Attitude (4 items) and Practice items (15 items).
Responses on the Participant Evaluation Forms were also analyzed. For items 1–9 on the evaluation instrument, a mean for each item was calculated and converted to a percentage. Responses to items 10 and 11 were placed into tables and reviewed to determine what changes should be made to the program for future research. An item analysis was also performed. This information will be used to revise poor items.
Data were analyzed using SPSS 11 (2005). The level of significance required to reject the null hypotheses was established at p < .05.
The 59 study participants consisted of 11 males and 48 females who completed a pre- and posttest. Of those, 40 belonged to a minority group (35 African American, 4 Asian, 1 Pacific Islander), 18 were white and 1 was not reported. Their ages ranged from 27 to 79 years. The mean age was 53.77 (SD = 13.13) years. Years of experience ranged from 1 to 40. The mean experience was 13.17 (SD = 7.7) years. Half of the group (n = 29) had high school education or less; the others had some college (n = 15), a Bachelor’s Degree (n = 6) or some graduate work (n =5). The educational levels of the remaining four participants are unknown.
Participants cared for residents aged 65 or older with physical and/or mental disabilities. Disabilities cited included dementia, schizophrenia, autism, mental retardation, cerebral palsy, depression, diabetes, stroke, high blood pressure, hearing impairment, visual impairment, and mobility impairment. Nearly three-quarters of participants (n = 44) cared for clients in their private residences. The mean number of residents per home was 3.33 (SD = 2.09) and the mean number of staff per home was 1.81 (SD = 0.85).
A majority of participants (n = 39) had used a computer before and slightly over one-third (n = 20) had previously used a computer for training. Two-thirds of participants (n = 40) indicated that they had a computer available for training at their facility. The types of Internet connections reported included: DSL or cable (63%), phone modem (25%), not connected (5%), and don’t know (7%).
A majority of participants (n = 37) indicated that they test their smoke detectors at least once a month and one-third (n = 20) inspect their fire extinguishers at least once a month. Slightly over one-third of participants (n = 21) indicated that they inspect their facility for fire hazards daily. A majority of participants (n = 36) complete yearly fire safety training.
Researchers performed a Chi-square analysis to determine whether there were significant differences between the two groups (treatment and control) related to gender, ethnicity, education, and previous use of computers. There were no significant differences between the treatment and control groups on any of the measured variables.
Reliability of the test instrument was investigated to allow researchers to improve the instrument for future research. A total of 59 people completed the pre- and posttest; Cronbach alpha was calculated for the pretest (.51) and posttest (.73). The analysis identified specific items that did not contribute to the variance and which researchers will delete or revise for future studies. An item analysis was also performed to examine specific item results. The information from that analysis will be used to modify areas of the program where posttest gains were low or did not reach 90% or higher.
The means and standard deviations for the pre- and posttest scores were calculated for the treatment and control groups. See Table 1.
Analysis of covariance (ANCOVA) was used to compare posttest differences between the treatment and the control groups, while controlling for any differences that existed at pretest. At posttest, the treatment group significantly outperformed the control group for total score F (1, 57) = 21.99, p = .001. A similar analysis was performed for each of the three subtests. The treatment group significantly outperformed the control group on the knowledge subtest F (1, 57) = 56.82, p = .001 and intended practices subtest F (1, 57) = 4.72, p = .034. Differences on the attitude item subtest were not significant.
High attitude scores on the attitude subtest prior to treatment explain the lack of effect. At pretest nearly all of the participants agreed with the attitude statements. The mean score at posttest on the attitudes subtest for the treatment group was 3.8 (SD = 0.5) and for the control group was 3.9 (SD = 0.3). In future research, attitude items will be reviewed and additional items added to increase test discrimination. High attitude scores were also reported in the pilot test of the print version of The Fire Safety Workshop (Walker, 1990).
Participants from the treatment and control groups (n = 59) completed an anonymous course evaluation immediately following the training. For each item, participants were asked to read a statement about the training and respond: Agree, Disagree, or No Opinion. For the purposes of this analysis, the response to Agree was considered positive and responses to Disagree and No Opinion were grouped together and considered to be negative responses to the program (except for Item #3 for which the opposite is true). See Table 2.
Participants were also invited to respond to two open-ended items on the course evaluation. The first item was "What did you like best about the program? " The most common responses were: "using a computer for training” (n = 15), "everything" (n = 5), "easy to use and understand" (n = 4) and “working at your own pace” (n = 3). The second item asked participants to "Describe any problems you had using the program and any recommendations you have for improving it. " The most common responses were "no problems or recommendations" (n = 51), and "I had to use the again button several times" (n = 2).
Of the 31 participants in the treatment group, 18 (58%) completed a follow-up three weeks after the training. The control group did not participate in the follow-up study because, as a courtesy, they were permitted to complete the fire safety training immediately after the posttest.
An ANOVA analysis showed that those who completed a follow-up and those who did not were similar at pretest and at posttest. Their mean ages were also similar. Chi-Square analyses of the two groups showed no significant differences in sex, educational level, and ethnicity.
The members of the follow-up group had significantly improved their test scores from pre- to posttest (M = 4.056, SD = 2.235, p = .0005). Three weeks later at follow-up, their test scores were on average 0.611 points lower, however the gain between pretest and follow-up continued to be significant. The difference between posttest and follow-up scores was not significant. See Table 3.
Follow-up study participants were also asked to respond to the question: “What changes have you made at your facility as a result of the Fire Emergency Planning module?” Responses provided by the 18 participants are included in Table 4.
This study demonstrates the potential for using interactive computer-based training as a viable alternative to instructor-led training to meet the fire safety training needs of the owners, operators, and staff of small board and care facilities. The treatment group significantly increased overall scores between pre- and posttest when compared to the control group. In the follow-up study, researchers found that after a three-week interval treatment group participants continued to perform significantly better than they had on the pretest.
Participant self-reports of the changes made to their fire emergency plan as a result of the training were encouraging. Important changes included conducting more frequent fire drills, assessing the strengths and limitations of staff and residents, increasing the number of emergency exits, and moving a resident’s bedroom closer to the exit.
Caregiver reaction to the use of computers for training was positive. Ninety-seven percent of participants (n = 57) indicated the computer was easy to use for training and they would like to use the computer for future training courses, even though 34% (n = 20) had never used a computer before. Ninety-eight percent (n = 58) indicated that they would recommend the program to others, that they learned a great deal from the program and that the program was useful to them as caregivers of older adults. Eighty-one percent (n = 48) thought they could complete a computer based training program in the future without assistance. The eleven participants who indicated they believed they would need help were all first time computer users. Two-thirds of participants (n = 40) indicated that they had a computer available for training at their facility and over half had a high-bandwidth Internet connection.
This study also demonstrates the need for fire safety training for this population. Over one-third of participants (n = 20) indicated that they do not test their smoke detectors at the industry recommended monthly interval and two-thirds (n = 39) do not inspect their fire extinguishers at the industry recommended monthly interval. Nearly two-thirds of the participants (n = 38) indicated that they do not conduct daily fire hazard inspections. At pretest, over one-quarter of participants did not recognize how quickly fires can grow; over two-thirds did not recognize that residents are at an increased risk for death and injury in fires and that smoke causes more fatalities in fires than flames and heat; and over three-quarters did not recognize the term “alternate escape route.”
For this study, changes in participant practices were evaluated by measurement of “intended future” practice at posttest and by actual “change in practice” at follow-up. However, the duration between posttest and follow-up was limited to 3 weeks due to the short time frame of the study. Future studies should include a long-term follow-up with all study participants (treatment and control) to determine what actual changes participants have made to the facility environments and fire safety procedures as a result of the training.
Participants completed the training program in a controlled environment, where they could ask the course monitor for assistance if they had a question. In the next phase of this research, participants will complete the training in their own facilities using their own computers.
The small sample size limits the broader application of the findings. In addition the score on the attitudes items was very high at pretest which limited the ability of the researchers to assess change in that domain.
This paper was prepared pursuant to the National Institute on Aging Grant Number R43 AG026193. The statements and conclusions herein are those of the authors and do not necessarily reflect the views or policies of the sponsoring agency.