|Home | About | Journals | Submit | Contact Us | Français|
Objectives: To describe the development and initial validation of the Northwick Park Therapy Dependency Assessment (NPTDA) as a measure of therapy interventions in neurorehabilitation.
Design: An iterative development process, followed by comparison with systemic prospective activity analysis, and parallel application of prospective and retrospective scores
Setting: A tertiary specialist inpatient neurorehabilitation service
Participants: A total of 37 patients (M:F 21:16, mean age 41.8 (SD 14.7) years) with complex neurological disability in two consecutive cross-sectional cohorts.
Methods: The NPTDA was developed and refined over 18 months, together with an algorithm that converts ordinal scores to estimated therapy hours/week. NPTDA-estimated hours were compared with ‘actual' therapy hours/week, identified from activity analysis. In a subsequent cohort analysis, prospectively rated NPTDA scores (reflecting intended levels of intervention) were compared with retrospective NPTDA scores (actual interventions).
Results: NPTDA-estimated therapy hours/week were strongly correlated with those identified from activity analysis, for total scores (Spearman rho 0.77, P<0.0001), and also for all five subdomains for direct (hands-on) intervention (rho 0.70–0.93, P<0.0001). The initial test algorithm overestimated therapy hours (Wilcoxon z=3.9, P<0.001). After adjustment, reanalysis using a revised algorithm showed this bias to be removed (Wilcoxon z=1.4 P=0.15). Prospective and retrospectively applied total NPTDA scores were strongly correlated (rho 0.61, P<0.0001). Although intended levels of intervention were higher than those actually delivered (Wilcoxon z=3.30, P<0.001), the differences corresponded to real deviations from intended practice.
Conclusion: In this initial evaluation, after revision of the algorithm, the NPTDA provided acceptable estimate of therapy interventions. Further evaluation is now required in other populations and settings.
A substantial literature now supports the benefits of higher intensity rehabilitation, at least for certain patients,1–3 but ‘higher intensity' has yet to be properly defined. Patients with neurological disabilities have widely varying needs for rehabilitation, often involving several disciplines. Simply recording hours of therapy input has little meaning unless the nature of interventions can be also be described. Many authors have called for practice-based research to ‘open the black box', in order to provide clearer description of the rehabilitation content.4 A number of tools have been developed to facilitate the systematic recording of therapy interventions,5–12 which include tools to describe the type of interventions offered for patients with stroke5,7–10 and spinal cord injury.11,12 However, these can only be applied to describe interventions that were actually given, rather than looking at what might be needed. Moreover, existing tools focus only on physical interventions (physiotherapy, occupational therapy and in some cases speech and language therapy8) and omit other interventions such as psychology, dietetics and social work, which play an important role in holistic neurological rehabilitation programmes.
Since the mid-1990s, work has been underway at Northwick Park Hospital in the UK to develop a comprehensive set of tools for rehabilitation, which are practical to apply in the course of routine clinical practice; and which may be used to measure nursing and therapy intervention, in relation to need, and to quantify this in terms of staff time. A common underlying principle of these instruments is that they are designed to be applied both prospectively to measure ‘needs' for rehabilitation intervention, and retrospectively to describe what the patient actually ‘gets', so that in future they could be applied as a framework for quantifying gaps in service provision. They also include a computerized algorithm, which translates the description of dependency into a generic estimation of implications for staff time.
The Northwick Park Nursing Dependency Scale (NPDS) was published in 1999 as a tool to assess nursing needs of patients in rehabilitation settings. It translates, by means of a computerized algorithm (the Northwick Park Care Needs Assessment) into an estimate of care hours required.13 It is shown to be a valid measure of nursing needs14,15 and has been increasingly applied in the context of routine clinical practice in the UK,16 as well as abroad.17
In 2004, a project grant was awarded by the UK Department of Health (Grant ref. 030/0066) to develop an equivalent tool to assess therapy dependency.18 The Northwick Park Therapy Dependency Assessment (NPTDA) was developed through an iterative process over two years. This paper provides a brief description of its development and initial validation.
The setting for this development and initial evaluation was the Regional Rehabilitation Unit at Northwick Park Hospital. The unit provides a tertiary specialist inpatient neurorehabilitation service for younger adults (mainly aged 16–65 years) with complex neurological disabilities. An experienced multidisciplinary staff team includes specialty-trained rehabilitation doctors and nurses; and a range of allied health professions which include physiotherapy, occupational therapy, speech and language therapy, psychology, dietetics, and social work, all of whom contributed to the project. Ethics permission was obtained from the Local Research Ethics Committee.
Development of the NPTDA involved an iterative process of consultation with senior multidisciplinary team members to identify the factors that describe requirements for different levels of therapy intervention. This led to the development of a draft tool in 2004. Over the next 18 months, successive periods of cross-sectional application of the tool provided an extensive prospective observational dataset. Refinement through serial analysis and team reflection in the context of clinical use ensured content validity for this setting, and the NPTDA evolved to its final form in late 2005. A full description of the early development and testing process is beyond the scope of this article, but is detailed in the Department of Health project report.18
The NPTDA is a measure of therapy intervention designed for use in specialist neuro-rehabilitation settings, where high intensity rehabilitation is provided by a multidisciplinary team.
Key principles of the tool are as follows:
As noted above, the NPTDA is designed to be applied in various ways depending on the intended purpose of measurement. For the assessment of therapy needs, NPTDA scores may be applied prospectively, based on the judgement of the therapy team in respect of the level of input required. For the assessment of therapy interventions, NPTDA scores may be applied retrospectively, based on the levels of intervention actually provided. In this way it is theoretically possible to record both, and to compare the needs for intervention with the levels of input provided (see Discussion).
This initial validation took part in two stages:
In a cross-sectional cohort analysis, routinely rated NPTDA scores for all inpatients on the unit were compared with the results of activity analysis for the same period (four consecutive working weeks between 21 November 2005 and 16 December 2005). All 24 therapists (20.3 whole-time equivalents) working on the unit at the time participated. Disciplines included physiotherapy, occupational therapy, speech and language therapy, dietetics, psychology and social work. The patient cohort consisted of 8 women and 9 men: mean age 45.5 years (SD 17.1). All had complex neurological disabilities arising from acquired brain injury (8 strokes, 5 traumatic), spinal cord injury (n=2) or Guillain-Barré syndrome (n=2).
Data were collected in the course of routine clinical practice. Patients on the unit are normally divided into two teams (‘Red' and Blue') and the weekly ward round alternates between the teams, so that each patient is reviewed fortnightly. Inevitably there were admissions and discharges during the four-week study period, so that 12 patients were rated on two occasions and 5 were rated only once, giving a total of 29 parallel sets of ratings of NPTDA scores with activity analysis for the corresponding period.
NPTDA scores were applied during the weekly ward round, by the treating team. They were rated retrospectively for each patient to reflect a week's therapy intervention, based on the average of the previous two weeks, thus allowing for week-to-week fluctuations. Scoring took 5–10 minutes per patient, and this time reduced as therapists became familiar with the tool.
NPTDA algorithm to calculate estimated therapy hours: In order to calculate ‘estimated therapy hours/week' from the NPTDA scores, we applied a test algorithm. Within the NPTDA manual, each item scoring level carries an approximate range of hours per week (see Appendix 1b). Our first ‘test algorithm' simply applied the mid-point time value for each range (e.g. for a time range 3–4hours, value 3.5; range 1–2hours, value 1.5, etc.). Only direct interventions could be compared, as the NPTDA did not record hours for indirect interventions at this point in its development.
Activity analysis: Over the same four-week period, each therapist systematically recorded all activity at half-hourly intervals throughout their working week. Activity was coded by each therapist onto a pre-piloted daily timesheet. Activity codes (full list available from the authors on request) were divided into patient-related and non-patient-related activity. Patient-related activity codes were designed to reflect the NPTDA item headings. Patient identity codes were used to assign activities to each individual patient. For simplicity and practical utility in the context of a busy service, where more than one therapeutic activity was undertaken within one 30-minute session, therapists recorded only the principal activity. Completed timesheets were handed at the end of each day and retained by the independent investigator, so that NPTDA estimations at the subsequent ward round were conducted independently of the activity analysis. Out of over 420 timesheets due for the four-week period, only two were missing.
Data were collated for each patient under each item heading in the NPTDA, to build up a series of individual patient records of therapy intervention received over the two-week period. The times were then halved to derive the average hours per week. As well as recording the ‘actual hours' per item for each patient, we also mapped these by reverse transcription to derive NPTDA scores from the activity analysis (‘activity analysis-derived NPTDA' scores), using the time range stated for each scoring level (see Appendix 1b) which, as noted above, varies somewhat for the different items.
Data were collated in specifically developed software written in Microsoft Excel, and transferred to SPSS version 11.5 or STATA version 8 for statistical handling.
There was a moderately strong correlation between total NPTDA ordinal scores and the total intervention hours, as recorded through activity analysis (rho 0.64 P<0.0001).
Table 1a shows the comparison of the ‘NPTDA-estimated' therapy hours for direct intervention subscales with ‘actual' hours of therapy identified by activity analysis. There was a strong correlation in total hours (rho 0.77, P<0.0001, see Figure 1a), and in all five subscores A–E (rho 0.70–0.93, all P<0.0001). However, the total NPTDA-estimated therapy hours were significantly higher than the ‘actual' hours (median 24 versus 17; Wilcoxon z=–3.9, P<0.001) in this analysis. The same trend was observed for all the subscale scores, except for ‘activities of daily living'.
Table 2 compares NPTDA scores rated by the team, with scores derived from activity analysis by reverse transcription. Nine of the 22 direct intervention items achieved ‘substantial' or ‘almost perfect' agreement (weighted kappa >0.65), a further eight achieved moderate agreement. Four direct intervention items and two indirect intervention achieved only fair agreement (kappa 0.2–0.4). Amongst these ‘personal self-care', ‘formal family support', and ‘key-working' also showed significant bias towards higher ratings on the team-rated NPTDA scores.
In summary, using the algorithm that applied the mid-point time values, we found a strong overall relationship between the therapy hours identified through activity analysis, and those estimated from the NPTDA, but the latter were consistently higher than those observed.
We therefore explored a number of different algorithms. Simply using the low point of the time range provided a better match overall, but led to underestimation of therapy hours for some items. Our final algorithm was therefore based on a mixture of low and mid-point time values informed by our activity analysis. The algorithm times for each item scoring level are shown in Appendix 2. Figure 1 compares scattergrams of the therapy hours estimated by the two algorithms, and the results of re-analysis using the revised algorithm to compare NPTDA-estimated hours with those derived from activity analysis are summarized in Tables 1b and and33.
Using this algorithm, the match appears to be closer. Table 1b demonstrates that there is now no significant difference between the NPTDA-estimated and actual total therapy hours. Although the correlations for each subscale are somewhat less strong than with the mid-point algorithm, the correlation between the total estimates of therapy time remains high (rho 0.70, P<0.0001). Similarly, on item-by-item analysis (Table 3), correlations between estimates of therapy time were significant for all direct intervention items (rho 0.49–0.89, P<0.001), with the exception of two items only (‘benefits and finances' and ‘key-working' – see Discussion).
Before the NPTDA can be applied to a hypothetical situation to assess ‘needs' for rehabilitation, it was necessary to determine the extent to which prospective application NPTDA provides a valid advance prediction of the levels of therapy intervention under existing conditions. We compared prospectively rated ‘intended levels of intervention' with retrospective ratings of ‘actual intervention'. We anticipated an approximate relationship but not an exact one, as there are often unpredicted changes in timetabling and staff availability. Moreover, patients' needs can sometimes change, and a flexible rehabilitation team should be able to adjust interventions in response to changing need. Any useful measure of therapy intervention, however, should be able to identify and describe these differences.
Stage 2 was undertaken in a second cross-sectional cohort analysis, during the subsequent 15-week period (January–April 2006). We compared the parallel application of prospective and retrospective NPTDA scores. In order to avoid excessive rating burden for the team, instead of applying the NPTDA retrospectively at each fortnightly meeting, prospective and retrospective scoring were alternated as illustrated in Figure 2. Again all patients were included, but only if they were present on the ward for the full two weeks. In total, 51 paired ratings were collected from a total of 31 patients – 16 males and 15 females; mean age 39.2 years (SD 14.6). Twenty-seven had acquired brain injury (14 strokes, 7 traumatic, 6 other, including hypoxia, inflammation and tumour), two had spinal cord injury and two Guillain–Barré syndrome.
At the beginning of each two-week block, the treating team rated ‘prospective NPTDA' scores for each patient, based on the average level of input per week they intended to give for each item during that period. Scores were rated during the routine goal-planning meeting, during which the team normally sets short-term goals and plans treatment for the coming fortnight. At the end of the same period, ‘retrospective NPTDA' scores were assigned by the team based on the average level of interventions actually given over that same two-week period. NPTDA scores were retained by the investigator rather than being filed in the patient records, so that at each scoring point, therapists were unable to refer to any previous scores, and in this sense were ‘blinded' to the scores they had given two weeks earlier.
As for stage 1, associations between prospective and retrospective scores or hours were tested using Spearman rank correlations. Agreement was tested using linear-weighted Cohen's kappa statistics. Significant differences were tested by paired Wilcoxon signed rank tests. The cut-off point for significance was again adjusted to P<0.01 to account for multiple tests. In the absence of pre-existing data to make a formal power calculation, our sample size was based on the crude calculation of 2K2 which, for a 5-point scale, is 50.
Table 4 summarizes the comparison between prospective and retrospectively rated scores. There was a strong association between the total NPTDA scores (rho 0.61, P<0.0001, see Figure 3) and subscale scores were also significantly correlated (rho 0.44–0.81, all P<0.001). On item-by-item analysis, weighted kappas ranged from 0.28 to 0.77, with 12/22 direct intervention items showing ‘moderate' to ‘substantial' agreement (kappa >0.40), but 10 showed only ‘fair' agreement. Agreement for the subscale scores was ‘fair' to ‘moderate.'
Overall there was a small tendency to overestimate predicted input through prospective scoring, leading to a significant difference in total scores (Wilcoxon z=–3.30, P<0.001). There was some considerable variation, however. Within the individual items, only scores for ‘equipment provision' were significantly different.
In the absence of an established gold standard against which to test criterion validity of the NPTDA, we used activity analysis to examine concurrent validity in stage 1. We found a strong overall relationship between the two estimates of therapy hours. However, those derived from the NPTDA using the mid-point algorithm were consistently higher than those observed through activity analysis, and there were several possible explanations for this bias:
Strenuous efforts had been made to ensure complete recording of activity analysis, with only two forms missing over the study period. The decision to record only the dominant activity for each session may have led to some inaccuracies, but should have equally under- and overestimated time for different activities, so avoiding systematic bias. However, short activities such as phone calls may not have been adequately captured, which might explain the poor correlation in items such as key-working and discharge planning.
Review discussions with the team revealed the following:
In stage 2, there was a strong relationship between prospectively allocated and retrospective scores, but prospective scores were fairly consistently higher. Again there may be several reasons:
Team debriefing identified a number of issues that were thought to have contributed to the discrepancy:
After reviewing the scores for specific instances of disagreement, the team agreed that, in the majority of cases, discrepancies between NPTDA scores had appropriately identified a real deviation from the level of intervention intended. This provides some support for the notion that the tool may have potential future application in describing the difference between the level of input provided, and hypothetical situations such as level of service ‘intended' (as tested here) or the level of services ‘needed' – although this must be tested separately. In the meantime, it underlines the importance of specifying the mode of application when results are reported.
In comparison with the NPDS, on which the NPTDA is modelled, it is important to recognize that the estimation of ‘requirements for therapy' is inevitably more subjective than that for ‘basic care needs' which most people would reasonably regard as essential. Previous experience suggests that these tools continue to evolve and develop over a decade or more, and much wider testing and validation will required before the NPTDA can be accepted on a similar footing to the NPDS as an estimation of ‘needs'. That said, the potential for hypothetical application makes the NPTDA unique, in comparison with other existing tools5,7–12 which can only be applied to describe interventions that were actually given. However, these other tools offer the advantage of more detailed analysis of specific therapy interventions than is possible with the broad-brush approach of the NPTDA. In this respect, the two different approaches may be found to complement each other and may usefully be applied in combination in future attempts to characterize black box of rehabilitation.20
There are a number of clear limitations to this study:
The NPTDA has been developed in the context of post-acute inpatient neurological rehabilitation. This particular service was chosen for its cohort of patients with complex rehabilitation needs, but further work is now required to test the algorithm in different settings, with different teams and different patient groups and other areas of rehabilitation. We have started to explore its adaptation for use with children and cognitive behavioural settings.
Despite the recognized limitations, this article describes the initial development of a potentially important tool to inform clinical practice. The results presented provide encouraging early support for its potential to provide a reasonable estimate of therapy interventions, which is practical to apply in the context of routine clinical care. Further exploration and evaluation is now warranted.
Full details of the NPTDA and computerized software are available from the corresponding author.
The authors would like to thank all the patients and staff who took part in this study.
Special thanks are due to the senior therapy staff on the Regional Rehabilitation Unit, who contributed to the development of the various versions of the NPTDA score over the course of this project. They include: Frances Clegg, Beverley Fielding, Jane Johnson, Sarah Harris and Pip Taylor. Heather Williams has also played a major part in to the on-going tool development. We are grateful to the Department of Health for providing funding for the project (Grant ref 030/0066) and to the Luff Foundation who provided support for the preparation for this manuscript.
|A: Physical handing programme||1) Medical management||Doctor||0–4|
|2a) Splinting orthotics (upper limb)||O/T||0–4|
|2b) Splinting orthotics (lower limb)||Physio||0–4|
|4) Physical therapy–active/passive handling||Physio||0–4||0–20|
|B: Basic functions||5) Tracheostomy management||SLT||0–4|
|8) Supported communication||SLT||0–4|
|9) Speech and language interventions||SLT||0–4||0–20|
|C: Activities of daily living||10) Personal/self-care||O/T||0–4|
|11) Domestic/community-based activities||O/T||0–4|
|D: Cognitive/psychosocial/ family support||13) Cognitive interventions||Psychology||0–4|
|14) Behavioural management||Psychology||0–4|
|16) Formal family support||Psychology||0–4|
|17) Emotional load on staff||Psychology||0–4||0–20|
|E: Preparing for discharge||18) Planning discharge/housing/care package||SW||0–4|
|19) Benefits and finances||SW||0–4|
|20) Equipment/adaptations for home||O/T||0–4|
|21) Community/home visits||O/T||0–4|
|F: Indirect interventions+ additional activities||23) Multidisciplinary meetings||State||0–2|
|25) Groups/extra therapies||State||0–2|
|26) (Accompanied) clinic attendance||State||0–2||0–8|
|G: Special input||27) Special facilities||(Select from lists)||Text|
|28) Special equipment hire||(Select from lists)||Text|
|29) Investigations||(Select from lists)||Text|
|30) Procedures||(Select from lists)||Text|
O/T, occupational therapist; SLT, speech and language therapist; SW, social worker.
|Score||Approximate hours/weeka (Example only)||Descriptor||Level description of therapy input|
|0||0||None||Not relevant, or no planned therapy at the current time|
|1||<1||Low||Minimal intervention, or review only|
|2||1–2||Medium||Basic intervention, or intervention by assistant only|
|3||3–4||High||More intensive intervention by qualified therapist +/− assistant|
|3.5b||<4||Interdisciplinary||Interdisciplinary intervention, but for limited total time|
|4||>4||Complex||Interdisciplinary intervention, and/or very high intensity input|
|Items||Lead discipline||Hours per week ascribed to lead discipline||Individually recorded hours per discipline|
|1) Medical management||Doctor||0||1||2.5||4.5||(–)||6|
|2a) Splinting orthotics (upper limb)||O/T||0||0.5||1||3||Individualized|
|2b) Splinting orthotics (lower limb)||Physio||0||0.5||1||3||Individualized|
|4) Physical therapy – active/passive handling||Physio||0||1||2.5||4.5||Individualized|
|5) Tracheostomy management||SLT||0||0.5||1||3||Individualized|
|8) Supported communication||SLT||0||0.5||1||3||Individualized|
|9) Speech and language interventions||SLT||0||0.5||1||3||Individualized|
|11) Domestic/community-based activities||O/T||0||0.5||1||3||Individualized|
|13) Cognitive interventions||Psychology||0||0.5||1||3||Individualized|
|14) Behavioural management||Psychology||0||0.5||1||3||Individualized|
|16) Formal family support||Psychology||0||0.5||1||3||Individualized|
|17) Emotional load on staff||Psychology||–||–||–||–||Individualized|
|18) Planning discharge/housing/care package||SW||0||0.5||1||3||Individualized|
|19) Benefits and finances||SW||0||0.5||1||3||Individualized|
|20) Equipment/adaptations for home||O/T||0||0.5||1||3||Individualized|
|21) Community/home visits||O/T||0||1||3||5||Individualized|
|23) Multidisciplinary meetings||State||0||Individualized||(–)||(–)||(–)|
|25) Groups/extra therapies||State||0||Individualized||(–)||(–)||(–)|
|26) (Accompanied) clinic attendance||State||0||Individualized||(–)||(–)||(–)|
O/T, occupational therapist; SLT, speech and language therapist; SW, social worker.
The algorithm is designed to provide a generic estimate of the implications for staff time associated with each item level of the NPTDA. The times were derived from the initial activity analysis presented in this article, but require testing and further development for other settings.
The computerized NPTDA data entry sheet automatically ascribes the hours of intervention shown for each level to the lead discipline as shown for levels 0–3. When levels 3.5 or 4 are entered staff are prompted to supply the estimated times for each discipline involved.
Lynne Turner-Stokes, King's College London, School of Medicine, Department of Palliative Care, Policy and Rehabilitation and Regional Rehabilitation Unit, Northwick Park Hospital.
Hilary Rose, Regional Rehabilitation Unit, Northwick Park Hospital, Harrow, UK.
The authors are the originators of the NPTDA and are naturally keen to encourage its wider use, but have no financial or other competing interests.