We followed the stepped approach previously described by Ryan and Farrar.12
Step 1. Identifying attributes and levels
Through a review of the existing literature and discussion with patients and members of the local primary care collaborative, we identified three attributes as important when making a routine appointment for a GP and we assigned plausible levels to each of these ().
Attributes and levels included in the study.
Step 2. Designing choice sets
This number of attributes and levels, gives rise to a complete set of 16 (4×2×2) possible combinations (or profiles) which, when paired with one another, result in 120 choice sets (excluding duplicate choices). For the purposes of this study and orthogonal design of 21 choice sets were selected by excluding choices where one profile clearly dominated the other (that is, was better on all levels for each attribute). In this case we assumed that the less time to appointment, own choice of doctor, and own choice of time would be the preferred options.
Step 3. Obtaining preferences for choice sets
Choices were presented in a self-completion questionnaire (Box 1
for sample choice). Participants were asked to make their choices in the context of telephoning for an appointment to discuss a routine non-urgent problem, and to express their preference for each pair presented.
Box 1. Example of a discrete choice experiment choice.
Three versions of the questionnaire, each containing seven choice pairs, were constructed. This was done in order to present participants with a reasonable number of choices that could be completed while waiting for their appointment. Two of the questionnaires also included choices where one option dominated the other on some attributes (while other attributes were held equal) in order to identify non-traders, who were defined as those failing both dominant questions in their questionnaire. The questionnaires also asked for basic demographic information and contained previously validated items on reason for appointment.14
The questionnaire was piloted for comprehension and ease of completion on 63 patients from a single practice. It was mailed to the same patients 1 week later to establish test-retest stability.
The three versions of the questionnaire were randomly distributed to participating practices, to be given to all patients aged 18 years and over attending for appointments on each day and for each session for a 1- or 2-week period. On completion, participants returned their forms to a drop box in the reception area, from where they were collected at the end of each surgery. Practices were asked to provide data on the number of doctors working at each practice, practice list size and time to third available appointment (a measure of appointment availability).
Step 4. Statistics and data analysis
There is no single accepted method for calculating sample size in discrete choice experiments. However, given a sampling frame of six GP practices and a time frame of 2 weeks we anticipated recruiting 1500 patients, giving a maximum possible total of 10 500 observations (1500×7).
Data were managed in SPSS and analysed using the statistical package STATA 8.0. Within the discrete choice experiment framework, it is assumed that if A is preferred to B then the utility or benefit derived from choosing A (with a given set of attributes and levels) will be greater than that of B (with a given set of attributes and levels) (equation 1: see Supplementary Information). We can only observe this indirectly (that is, through the choices made) as the difference in utility between the two choices and their associated attribute levels (equation 2: see Supplementary Information).
We used both fixed and random effects (to account for the fact that individuals provided multiple responses) probit regression models to analyse responder's choice of preferred appointment (either choice A or B) as the dependent variable. A linear additive utility model was specified (equation 3: see Supplementary Information).
In addition to analysing the main effects (the three main attributes) specified in equation 3, it was hypothesised that individual characteristics, such as socioeconomic variables, would also influence preference for a GP appointment. We chose to include variables for sex, working/educational status and age. Despite stating the context in which the choices should be made, we also hypothesised that responders' preferences would be contextualised by their current experience and therefore included variables for who was attending the appointment, the reason for attendance, and the number of whole-time equivalent GPs in the practice. Including these effects in the analysis minimises the effects of any biases that would otherwise be present in the regression result estimates.
Given that these characteristics do not differ between each choice and they simply drop out of the equation (equation 2: see Supplementary Information), they were entered into the model analysis through interactions with the main effects ( and equation 4: see Supplementary Information). The segmented model included all main and interaction effects. To create a more parsimonious model, this was reduced stepwise by excluding insignificant main and interaction effects one at a time using P>0.05.
The ß coefficient values derived from regression equation 4 (see Supplementary Information) were used to estimate the relative importance of attributes (the significance and sign of the coefficient value) and the trade-offs responders would be willing to make between them (the marginal rate of substitution [MRS] calculated by dividing the respective coefficient values of the attributes in question). In this case we calculated the marginal rate of substitution values using the ‘time to appointment’ attribute as the denominator so that responder's preferences and the trade-offs could be compared on a common value scale in terms of ‘willingness to wait’ (formula 1 see Supplementary Information).