One of the more time-consuming parts of survey research involves the creation and validation of the survey instrument or questionnaire. Many steps are involved in the development of a series of items that address the research question or questions. The most efficient way to develop appropriate items is to create a Table of Specifications (ToS). The ToS delineates the main topics of the survey; these topics should be directly related to the research question. Under each topic area, there may be subconcepts or subtopics that the ATI wishes to investigate more specifically. In essence, the ToS becomes an outline of the content of the survey. Table demonstrates an excerpt from a typical ToS for a research question that examines the effectiveness of CIs; this example is continued through each section of the article.
Table 2. Sample Table of Specifications
The ToS is used as a guide to develop appropriate questions and to determine criterion-related validity and the plan for item analysis. As questions or items are developed, they should be assigned to a topic area in the ToS. Items should fit into one of the categories of the ToS; an item can be reworded to fit more appropriately into a category, or it may be placed aside for use in a future study.
Writing questions depends on the kind of information being sought, question structure, and actual choice of words. Most questions involve assessments of attitudes, beliefs, behaviors, or attributes.4
Attitude questions indicate direction of the respondents' feelings: whether they are in favor of or oppose an idea, or if something is good or bad. For example, an attitude question might read, “It is important for all clinical instructors to possess good communication skills.” Belief questions assess what a person thinks is true or false, and may be used to determine knowledge of a specific fact.4
For example, a belief question might read, “To be a clinical instructor in athletic training, I must be an Approved Clinical Instructor as recognized by the National Athletic Trainers' Association and the Commission on Accreditation of Allied Health Education Programs.”
Behavior questions are designed to elicit respondents' beliefs about their behaviors.4
Because there is no direct observation of the respondent, there is no way to ensure that the respondent is truly telling the ATI what he or she really does. Belief questions tell the ATI what respondents believe they do; this is a weakness of the self-reporting system. One way to cross-check the authenticity of a respondent's belief is to ask someone else to evaluate how the respondent responds to the situation or to use interview techniques. For example, if a CI is asked to evaluate how often he elicits questions from his athletic training student, he may indicate several times throughout a day. The athletic training student then could be asked how often his CI elicits questions from him. This type of questioning may confirm that the respondent's beliefs are consistent with his student's beliefs about his or her behavior. Consistency of response is evaluated during the analysis phase of the study.
The fourth type of question that can be used in a survey is an attribute question that provides primarily demographic data.4
Most surveys involve some collection of demographic information, so that the investigator can use those variables later to analyze the data based upon specific demographic considerations. For example, by acquiring information as to sex of a CI and years of clinical experience as a CI, the investigator may be able to determine later if sex or years of clinical experience or both influenced the outcome of CI effectiveness. Investigators should be cautioned to have a designated purpose or need to collect specific demographic information and to collect only that data germane to the research question. Collection of personally sensitive data, such as annual income or political affiliation, should only be included if it is important to the overall purpose of the study.
Survey questions can be written in many ways, and each method requires different considerations for item analysis, or how the data will be used to determine the results of a specific question.5
Close attention should be paid to how the data will be analyzed upon the completion of the study, as different statistical techniques require different formats of data (eg, ordinal, nominal). For example, age can be assessed as an exact number or as part of a range of numbers. The ATI must determine in advance if it is important to analyze the results of the survey based upon a specific age or if age ranges will be sufficient.
When writing survey questions, open-ended versus closed-ended questions provide unique challenges for the ATI. Open-ended questions may allow respondents to answer completely, creating personalized answers using their own words and without investigator bias or limits4
: for example, “Please describe the attributes of an effective clinical instructor.” This method is very appropriate for qualitative studies but may create more concerns if the analysis is quantitative. Open-ended questions may be difficult to code and analyze because respondents may answer in many different ways.2
The ATI may be unable to interpret the open-ended response without clarification from the respondent.4
One method for using the open-ended question technique is to delimit the response. For example, the ATI might ask the respondents for their opinions on the best method to tape an ankle for a recent injury (2 weeks old) to the calcaneofibular ligament, rather than asking for the best method to tape an ankle.
Closed-ended questions are one of the more common types used in athletic training literature. Closed-ended questions require the subject to select a response from a list of predetermined items developed by the investigator. These types of questions are typical of standardized multiple-choice tests; they allow for consistency in response and may be coded more easily. Potential responses may be presented in either random or purposeful order. The disadvantage to this style of question development is that it may limit the expression of respondents' opinions in “their own ways,” thus potentially biasing the data.2,4
The items developed should be exhaustive in nature, providing the respondent with all possible responses, and they should be mutually exclusive. With mutually exclusive items, each choice should clearly represent a unique option.2
An example of a poorly worded item would be, “I feel that I am prepared academically and clinically for my role as a clinical instructor.” This example is asking for feedback on 2 separate concepts: academic and clinical preparation. It would be preferable to separate the concepts into 2 different questions: “I feel that I am prepared academically for my role as a clinical instructor,” and “I feel prepared clinically for my role as a clinical instructor.” Table provides additional considerations for developing both open-ended and closed-ended questions.
Table 3. Developing the Questionnaire and Related Materials2,4,5
The other common data-collection method is the scale. The scale is an ordered system that provides an overall rating representing the intensity felt by a respondent to a particular attitude, value, or characteristic. Scales allow the ATI to distinguish among respondents.2
While several types of scales are used in questionnaires, the 2 most common in athletic training literature are the Likert and Guttman scales. Likert scales are summative scales that are used most often to assess attitudes or values. A series of statements expressing a viewpoint is listed, and the respondents are asked to select a ranked response that reflects the level with which they agree or disagree with the statement.2
Potential responses are presented in rank order. A large number of items or statements, usually 10 to 20 items, is required when using a Likert scale. An equal number of the items should reflect favorable and unfavorable attitudes to truly discriminate the respondents' opinions.2
Responses generally are provided in 5 categories (strongly agree, agree, neutral, disagree, and strongly disagree), but some support exists for the use of an even number of categories to require respondents to take a definitive position (either positive or negative) on the response. In other words, if 3 responses are positive and 3 are negative, respondents are forced to make a directional decision.
Guttman scales are cumulative scales that present a set of statements reflecting increasing intensities of the characteristics being measured. This technique is designed to ensure that only one dimension exists within a set of responses; only one unique combination of responses can achieve a desired score.2
In this cumulative scale, if respondents agree with one item, they also should agree with designated other items. For example, if a respondent believes that an effective CI communicates well with coaches, he or she also would select responses that indicate that the CI has open communication with colleagues and students. Success using Guttman scales relies on having a large number of respondents to assess patterns accurately.2