Home | About | Journals | Submit Manuscript | Contact Us 
pmc logo imagepmc logo image
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
From:
Am J Prev Med. Author manuscript; available in PMC Oct 1, 2008.
Published in final edited form as:
Am J Prev Med. Oct 2007; 33(4): 336–345.
doi: 10.1016/j.amepre.2007.05.007
Appendix A
Study Design Quality Tabulation and Coding Criteriaa
Physical Activity Studies
Individual
randomization
Control
group
Isolate
technology
Pre/postRetention
≥80%
BL
groups
equiv
Missing
data
Sample
size calc
Validated
measures
Score
(% of
maximum)
Bickmore (2005) 15YYYNYUKNYN56%
Croteau (2004)16NNNYYNYNY44%
Dinger (2004)17NNNYYNNNY33%
Goran (2005)18NYNYNYNNY44%
Hager (2002) 19YYNYNNNNY44%
Hageman (2005) 20YYNYYUKNYY67%
Marshall (2003) 21YYYYYYYYY100%
McKay (2001)22YYNYYYNNY67%
Napolitano (2003) 23YYYYYYNNY78%
Palmer (2005)24NYYYNUKNNN33%
Pinto (2002) 25YYNYYYYNY78%
Rovniak (2005)26YYNYYYNNY67%
Yoo (2003)27NNNYYNNNUK22%
Dietary Behaviors Studies
Individual
randomization
Control
group
Isolate
technology
Pre/postRetention
≥ 80%
BL
groups
equiv
Missing
data
Sample
size calc
Validated
measures
Score
(% of
maximum)
Anderson (2001) 28YYYYNYNNY67%
Baranowski (2003)29YYYYYNYYY89%
Block (2004) 30NNNYNUKYNY33%
Block (2004) 31YYNYYYNYN67%
Carpenter (2004) 32YYYYYYYNN78%
Delichatsios (2001) 33YYNYYYYNY78%
Glasgow (2000, 2002)34,35YYNYYYNNY67%
Irvine (2004)36YYYYYYNNY78%
Long (2004)37NYYYUKYNYY67%
McKay (2002)38YYNYYYNNY67%
Oenema (2005)39YYYYNYNNY67%
Papadaki (2005)40NYYYNNYYN56%
Stevens (2002, 2003)41,42YYYYYYNNY78%
Verheijden (2004) 43YYYYYYYYY100%
Combined Activity, Diet, and Weight Loss Studies
AuthorsIndividual
Randomization
Control
Group
Isolate
Technology
Pre/PostRetention
≥ 80%
BL groups
equiv
Missing
Data
Sample
Size
Calc
PA
validated
measures
Diet
validated
measures
Score (%
of
maximum)
Abroms (2004) 44NYYYNNNNYN40%
Baranowski (2003) 45YYNYYNYYYY80%
Calfas (2002)46YYNYYYNNYY70%
Frenn (2005)47NYYYNYNNYY60%
Frenn (2003)48NYNYNUKNNYY40%
Harvey-Berino (2002)49YYYYNYYNYY80%
Harvey-Berino (2002)50YYYYYNNNYY70%
Harvey-Berino (2004)51YYYYNYYYYY90%
Kypri (2005)52YYYYYYNYNN70%
Patrick (2001)53YYNYNYNNYY60%
Plotnikoff (2005)54YYYYYYNYYY90%
Prochaska (2004)55YYYYYYYYYY100%
Southard (2003)56YYYYYYYYNY90%
Tate (2001)57YYNYNYYYYY80%
Tate (2003)58YYNYYYYNYY80%
Veverka (2003)60YYYYYYNNNANA60%
Vandelanotte (2005)59YYYYNUKNNYY60%
Williamson (2005)61YYNYYYYYYY90%
Womble (2004)62YYNYNYYYNANA60%
Wylie-Rosett (2001)63YYNYYNNYYY70%
Study Quality Coding Criteria
Table headingScoring criteria
Individual randomizationWere participants randomized to study conditions? If so, was randomization at the individual level? Stratified and blocked randomization is acceptable. Studies that used individual randomization combined with a small proportion of randomized matched pairs are also considered YES. Appropriately designed and powered group randomization would also be acceptable if group was also unit of analysis. Individual randomization is NO when the authors fail to mention randomization, specify that another method of assigning group status was used, or randomize at the group level and analyze at the individual level.
Control groupDid the study include a comparison group? Comparison group could be a no treatment, treatment as usual, or alternate treatment group.
Isolate technologyDid study design allow for test of effectiveness of the technology? E.g., web-based delivery verses no treatment. To isolate the technology, the authors had to test the technology alone and compare to a group with no technology (YES). Packaged interventions where the technological components can't be parsed out are coded as not isolating the technology (NO).
Pre/post test designWas assessment of behavior completed pre intervention and post intervention?
RetentionWas study retention at least 80% of subjects who initially agreed to participate in the study? Retention is calculated for the entire sample and not by group. For studies that did not report retention or dropout rates, retention can be calculated using the sample sizes used for analyses (e.g. 300 randomized, but only 250 were included in analyses = 83.3% retention).
BL Groups equivalentWere tests conducted to determine if groups were equivalent at baseline on important variables (e.g., gender, age, weight)? If no tests mentioned, then = UK. If subset of tests indicated any group differences at baseline, then = NO.
Missing dataWere analyses conducted with consideration for missing data that maintains the fidelity of the randomization (e.g., intent-to-treat, imputation). Listwise case deletion (completer analysis) = NO, if only analysis conducted. If 100% retention then completer analysis is appropriate = YES. If authors compared the ‘dropped subgroup’ to the selected or randomized sample, but did not consider the impact of the dropped subgroup on randomization (e.g. ITT or imputation), then code as NO.
Sample size calculationWas power analysis reported to determine study sample size?
Validity of measuresDid description of measures include reliability and validity information? If reference or coefficients, then YES. If well established measure that is known to be validated, then YES. For objective measures without validity evidence, if the objective measure is used as a proxy (e.g. food receipts for nutrition intake), then NO. If the objective measure is used as a direct measure of behavior (e.g. food receipts for food purchase), then YES. If validity not reported and measure unknown, then UK.
TotalSum of Yes's
aAll quality criteria rated as yes (Y), no (N), or unknown/unclear (UK)