Comparing cost of different methods
This study provided empirical cost data for use in cost models that can facilitate decision making and planning of future studies, as well as investigations of cost efficiency. For the current study design, exposure questionnaires were the least costly way to produce posture data, followed by observation and then inclinometer.
It is important to note that method-specific measurement costs are not always the most costly aspect of a study; in the present study the fixed and variable costs that apply to all methods make up between 43% and 78% of the total study cost (Table
). This provides a substantially different result from a previous report of cost in ergonomic exposure assessment that disregarded travel and recruitment components, resulting in interview costs roughly 10% that of inclinometry [18
]. Clearly this has a substantial effect on research planning and also the relative trade-off between methods; choosing a self-report method does not necessarily mean that one can conduct 10 times as many exposure assessment for the same price; one must consider the overall logistics of the study.
The effect of study design and cost assumption
The simulations in Table
demonstrate the budget impact of several different situations. The general form of the model is intended to be a tool for researchers during project planning, and the empirical costs are intended to provide a guideline for similar situations. Although the potential inputs to the model are infinite, here we will use the simulations in Table
to comment on research planning and budgeting issues.
Study design and planning involves a lot of logistics, with some cost-relevant aspects seldom reported in the research literature. Decisions around sample size, measurement scheduling and spacing, concurrent measurements, location and travel, and equipment acquisition can have wide-ranging impacts on costs. Sample size affects generalizability, study power, and confidence in research results, but is limited by budget constraints. The ċR
unit cost of € 22 included in the model quantifies explicitly the cost increase when a new worker is recruited. Retention and attrition are acknowledged problems in public health research [21
], and it can be increasingly difficult to recruit and maintain participation. One of the simulated unit cost assumptions (simulation 6) examined a case where recruitment costs 10 times as much as in our study, resulting in an 8-14% increase in total study costs. It may be that as the sample size approaches an exhaustive sample of the worksite, the cost of recruitment does not stay constant as in the cost model presented here; costs may go up if employers are resistant to additional recruitment, or they may go down if social facilitation encourages workers to participate (i.e. they see their colleagues participating and want to join). These types of non-linear relationships have been hypothesized [20
], but the current study did not collect empirical data to be able to implement them.
Although the inclinometers were robust enough not to require replacement over the span of the study, some types of equipment may require replacement that would make the direct measurement cost even higher. The inclinometer had the highest fixed costs, and therefore the most opportunity to amortize the fixed equipment costs over many measurements. The variable costs are also high since the inclinometer measurements were made for the full day, but this is a conservative estimate of time required for the inclinometer as this method only requires hands-on researcher input for set-up and take-down and occasional troubleshooting during the shift. This means the inclinometer lends itself very well to multi-tasking or to concurrent measurements. In multi-tasking researchers could analyze previously-collected data, write reports, or do administrative work during the seven hours between inclinometer set-up and take-down. If researchers were to perform useful work during the inclinometer measurements, a day’s inclinometer measurement would include only the cost of supplies and preparation time (approximately €175); the total study cost for an inclinometer study with the parameters of the current study would be €48 359, a savings of €16 983.
Work context could have an effect on the estimated costs, particularly for video tasks. Baggage handling work is very dynamic and takes place over a large workplace, so researchers must actively track workers with a camera for the entire measurement. This strategy would be similar in industries like health care, construction, agriculture, and resource industries such as mining. However, office work and assembly line jobs in manufacturing where the tasks are carried out in a small area may be good candidates for passive video capture using a tripod or a surveillance camera, in which case concurrent video measure would be possible. This would increase nc, the number of concurrent workers, and decrease the total study cost. If it were possible to have perfectly overlapping measurements that required no extra time, the total study cost would decrease slightly to 88-99% of current study costs. Despite a cost savings of 12%, concurrent measurement using video seems impractical, if not impossible under the conditions of the current study. In concurrent measurements, several workers would be measured within the same workshift for the same amount of preparation, travel, and waiting time and this could yield savings. Simulation 3 examines concurrent measurement of 4 workers per day and is the cheapest of the simulated options, costing 68-79% of the current study costs. Conversely, measuring only one worker per day (Simulation 2) requires more time and travel costs, resulting in costs 19-30% higher than the current study. The feasibility of concurrent measurements will depend on the start times of the workers and how flexible employer and/or workers are to modifying work times to allow for set up and measurement of several workers. Based on our experience, we estimate that for one researcher, 4 workers could be measured simultaneously using the inclinometer, 10 or more using the questionnaire, and only 1 additional worker for the video camera (unless the amount of recording time was shortened).
Although not explicit in the cost model, it is possible for the spacing of measurements to affect cost as well as the more well-known effects on exposure values [19
]. In the current study we chose consecutive shifts to avoid the travel costs and administrative hassle of scheduling workers on a rotating shift schedule, but the trade-off is autocorrelation in the data. We also elected to conduct the study over a 3-month period during the winter, which surely impacts the type of exposures encountered and is unlikely to have full generalizability to the other parts of the year. However, a year-long sampling campaign would have been cost-prohibitive, given the need to retain part-time staff for the full year or retrain new staff.
Travel and accommodation needs are likely to vary substantially between research studies depending on the location of the researchers and institution relative to the data collection site. The current study showed substantial variation in travel distance and cost, represented by a ċV coefficient of variation of 58%. The distances in the current study required considerable travel, and introduced a trade-off between the total number of trips and length of stay. Simulation 1 shows the effect of making a new trip to the measurement site for each measurement day, which increases the costs less than 1%. Conversely, if each researcher made only 1 trip to the measurement site and stayed in a hotel for the duration of the study, costs decrease to 88-92% of the current study. If unit costs for travel or accommodation were zero because the measurement site was adjacent to the university as in simulation 4, the costs would be only 61-78% of the current study costs. Naturally cost is not the only factor when planning travel logistics and accommodation schedules; an additional consideration will be the tolerance of data collectors and local labour laws.
This study assumed no depreciation of the equipment cost, so the whole cost of research equipment purchase price is included in ČE. The costs presented therefore represent the assumption of ‘starting from scratch’ and having to purchase all equipment. However, were the researchers to pursue a similar study in the future, the decision between which method to select would be weighed without the fixed costs of equipment, since the equipment has already been acquired. This is demonstrated in simulation 5, where total study costs are only 87% of the current study cost for the equipment-intensive inclinometer method, 92% for the observation and 93% for the questionnaire.
Who’s paying? Researcher- and employer-borne study costs
Over 90% of costs tracked in the current study were borne by researchers. It seems likely that time tracking biases would tend to overestimate this proportion, since researchers may be more motivated and diligent in reporting time spent on the study. It is possible that the time tracked by employer administrators (3.4% of the total study costs) is underestimated. For example, short tasks might be deemed ‘not worth tracking’ but cumulatively might represent a relevant cost. When industry stakeholders decide whether to participate in research, information about the time and resource commitments can help manage expectations and plan resources, as well as demonstrating researchers’ sensitivity to balancing business interests with research needs, the prioritization of which is different between researchers and industry stakeholders. Researchers forecasting costs for industry stakeholders may not have a strong influence on participant recruitment, but it seems likely to foster better trust and stronger relations between research stakeholders and almost certainly enhances retention or re-contact of study participants.
In the current study, employer administration and worker’s production time were both borne by the employer. In other contexts, the employer might not be able to pay for workers’ time, and transferring this cost (in terms of time spent or opportunity cost) onto workers is likely to affect the participation rate. The opportunity cost is the value of what is given up in order to participate in a study. Adding an hour to the workday may affect carpooling arrangements or transit/parking habits that can change direct costs of participation; it also means less time with family or sports and leisure pursuits. To address this, some studies provide an economic incentive equivalent to wage replacement [22
]. This could increase the research budget substantially as worker production time accounted for over 6% of the total study cost in the current study. Although workers were paid for their time when they showed up early and stayed late, a limitation of the current study’s cost tracking method is that it does not account for opportunity costs associated with extending the workday or filling out questionnaires during work time. These types of costs are difficult to quantify and so were not included in the current study, although they may have an impact in participants’ decision making.
Method performance: another consideration for decision making
If cost were the only consideration, the findings presented here would suggest that self-report is always the best option for assessing work postures. However, cost is far from the only criterion for selecting an exposure assessment method. It is possible for less tangible characteristics to render a method more desirable to worker, employer, union, insurance board, or regulatory stakeholders. These partners may have a perception that self-report will always be biased, that direct measurements are infallible, or that ‘expert’ observation is adequate in every situation. In order to foster collaboration, these perceptions need to be discovered and addressed either through education or compromise.
Researchers also need to consider the scientific quality of the data in terms of accuracy, precision, or predictive validity for health outcomes. In order to address this issue, the cost efficiency of each method must be determined by comparing the price to the performance of each method. To determine cost efficiency, cost data (such as that contained in this report) could be combined with components of variance from collected exposure data to quantify the cost efficiency of each method and sampling schemes as described previously [11
]. Variance components are often used to guide allocation of measurement efforts within and between individuals [19
]. Most studies investigating this issue have considered only statistical efficiency, not measurement cost [11
]. However, optimization based on both costs and statistical performance could yield a substantially different study design than what follows from optimizing only with respect to statistical performance [16
]. For example, when within-worker variance is higher than between-worker variance and recruitment costs for engaging participants are high compared to costs for collecting more data from subjects already in the study, multiple measures on fewer workers may prove to be a more cost efficient sampling strategy than that suggested when only statistical performance is considered, i.e. distributing measurements among as many subjects as possible [14
]. Bias is also an important consideration; self-report has been shown to, in general, overestimate physical exposures [24
], particularly in workers with musculoskeletal symptoms [25
]. Some observation methods may also be associated with bias when compared to results obtained by inclinometry [12
]. This type of misclassification could have deleterious effects on an epidemiological study of health outcomes. Previous cost-efficiency studies have shown that for simple models of cost and statistical performance, cost efficiency can be explicitly optimized. However, the complexity of the cost model, including possible non-linearity, and the structure and degree of errors in some exposure variables may preclude a fully-optimized cost efficiency model and instead favour a numerical rather than analytical approach.
Thus, future research should investigate not only the price, but also the cost efficiency of the exposure assessment methods in terms of value outputs like precision and bias of the resulting information.
Performance of the cost model
The accounting protocols used in this study allowed estimation of variability in unit costs, and variability tended to be high. The coefficient of variation (CV) in recruitment and travel were both over 50%, demonstrating that these costs are influenced by other factors than just the number workers recruited or number of trips. The cost for concurrently measured workers was also highly variable (CV
32-88%), a result of the varying amount of time required to complete an additional measurement. Together, the variability of the unit costs gives some insight as to the stability of total study cost if the study were to be repeated, introducing the notion of confidence intervals around a projected study cost. The uncertainty of cost estimations, as well as the impact of such uncertainty on total costs, could be an avenue for future research.
The Rezagholi and Mathiassen review lists several simplifying assumptions in existing cost models: assuming that all measurements have an equal cost [13
], or that costs can be split into two or three stages or levels (e.g. ‘worker’ level and ‘repeated measures’ level), or can be represented by limited components such as ‘measurement’ and ‘recruitment’ [14
]. The current model has attempted to expand on previous models by including these missing aspects. However, with nine types of costs included, it is worth considering whether the cost model could be simplified without hampering its performance. For example, the cost of supplies was very low, with a unit cost of €0-4 and a supply cost of less than 1% of the total study cost. Similarly, recruitment accounted for less than 2% of the total study cost. These costs would have so little impact on decision making that they are probably not worth tracking. The most important cost components are those involving researcher time. Thus, another issue for further research would be the cost efficiency of the cost tracking per se
; for instance whether a simpler and cheaper cost tracking procedure would free resources in a study that could then be used to collect more exposure data.
There are a number of other ‘hidden’ costs that were not explicitly separated out in the model. For example, energy and infrastructure costs were considered to be included in the University overhead which was incorporated into researcher work time. As a result, researchers looking to use the model cannot separate institutional overhead components. This limitation could be navigated by adjusting unit costs for researcher time to apply different overhead values.
The data presented do not currently include the costs related to post-processing and analyzing collected data into postural exposure variables. Rather, the present study stops accounting when all the acquired electronic files are downloaded onto University servers and the questionnaires are delivered to University file storage. These additional costs in a full exposure assessment include all tasks between data collection and statistical analysis, including data entry for paper questionnaires, data processing, visual inspection and quality control for inclinometer data, and observer time spent recording postures from video still frames. A recent study investigating the cost-efficiency of different observation sampling protocols suggests these costs can be substantial, but also that they depend significantly on the technique used for data processing [12
]. There are many options for processing and analyzing data depending on the research questions; the present results may be viewed as a common starting point for a complete cost assessment of basic data collection up to the point of finalized exposure data, where the processing costs differ depending on the techniques used for retrieving exposure variables. Since processing and analysis costs could comprise a considerable portion of total research costs, the comparisons between exposure assessment methods could evolve as these components are included.
Some time was spent multitasking in the current study, and it is difficult to separate the time spent on each method. For example, inclinometer and video data could be downloaded while the questionnaire was being filled out. Although it would seem that the inclinometer took the most preparation time, it is not always possible to separate how much time was spent on what. For this reason, the time estimates for each method (and especially the questionnaire) could be overestimated. Conversely, when questionnaire methods are applied alone, the time it takes to identify a participating worker and introduce an instrument is non-trivial but was not explicitly accounted for in our data, so overestimates on the researcher time spent on the questionnaire are likely minimal.