Appendix A
Study Design Quality Tabulation and Coding Criteriaa
| Physical Activity Studies
| ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Individual
randomization | Control
group | Isolate
technology | Pre/post | Retention
≥80% | BL
groups equiv | Missing
data | Sample
size calc | Validated
measures | Score
(% of maximum) | |
| Bickmore (2005) 15 | Y | Y | Y | N | Y | UK | N | Y | N | 56% |
| Croteau (2004)16 | N | N | N | Y | Y | N | Y | N | Y | 44% |
| Dinger (2004)17 | N | N | N | Y | Y | N | N | N | Y | 33% |
| Goran (2005)18 | N | Y | N | Y | N | Y | N | N | Y | 44% |
| Hager (2002) 19 | Y | Y | N | Y | N | N | N | N | Y | 44% |
| Hageman (2005) 20 | Y | Y | N | Y | Y | UK | N | Y | Y | 67% |
| Marshall (2003) 21 | Y | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| McKay (2001)22 | Y | Y | N | Y | Y | Y | N | N | Y | 67% |
| Napolitano (2003) 23 | Y | Y | Y | Y | Y | Y | N | N | Y | 78% |
| Palmer (2005)24 | N | Y | Y | Y | N | UK | N | N | N | 33% |
| Pinto (2002) 25 | Y | Y | N | Y | Y | Y | Y | N | Y | 78% |
| Rovniak (2005)26 | Y | Y | N | Y | Y | Y | N | N | Y | 67% |
| Yoo (2003)27 | N | N | N | Y | Y | N | N | N | UK | 22% |
| Dietary Behaviors Studies
| ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Individual
randomization | Control
group | Isolate
technology | Pre/post | Retention
≥ 80% | BL
groups equiv | Missing
data | Sample
size calc | Validated
measures | Score
(% of maximum) | |
| Anderson (2001) 28 | Y | Y | Y | Y | N | Y | N | N | Y | 67% |
| Baranowski (2003)29 | Y | Y | Y | Y | Y | N | Y | Y | Y | 89% |
| Block (2004) 30 | N | N | N | Y | N | UK | Y | N | Y | 33% |
| Block (2004) 31 | Y | Y | N | Y | Y | Y | N | Y | N | 67% |
| Carpenter (2004) 32 | Y | Y | Y | Y | Y | Y | Y | N | N | 78% |
| Delichatsios (2001) 33 | Y | Y | N | Y | Y | Y | Y | N | Y | 78% |
| Glasgow (2000, 2002)34,35 | Y | Y | N | Y | Y | Y | N | N | Y | 67% |
| Irvine (2004)36 | Y | Y | Y | Y | Y | Y | N | N | Y | 78% |
| Long (2004)37 | N | Y | Y | Y | UK | Y | N | Y | Y | 67% |
| McKay (2002)38 | Y | Y | N | Y | Y | Y | N | N | Y | 67% |
| Oenema (2005)39 | Y | Y | Y | Y | N | Y | N | N | Y | 67% |
| Papadaki (2005)40 | N | Y | Y | Y | N | N | Y | Y | N | 56% |
| Stevens (2002, 2003)41,42 | Y | Y | Y | Y | Y | Y | N | N | Y | 78% |
| Verheijden (2004) 43 | Y | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Combined Activity, Diet, and Weight Loss Studies
| |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Authors | Individual
Randomization | Control
Group | Isolate
Technology | Pre/Post | Retention
≥ 80% | BL groups
equiv | Missing
Data | Sample
Size Calc | PA
validated measures | Diet
validated measures | Score (%
of maximum) |
| Abroms (2004) 44 | N | Y | Y | Y | N | N | N | N | Y | N | 40% |
| Baranowski (2003) 45 | Y | Y | N | Y | Y | N | Y | Y | Y | Y | 80% |
| Calfas (2002)46 | Y | Y | N | Y | Y | Y | N | N | Y | Y | 70% |
| Frenn (2005)47 | N | Y | Y | Y | N | Y | N | N | Y | Y | 60% |
| Frenn (2003)48 | N | Y | N | Y | N | UK | N | N | Y | Y | 40% |
| Harvey-Berino (2002)49 | Y | Y | Y | Y | N | Y | Y | N | Y | Y | 80% |
| Harvey-Berino (2002)50 | Y | Y | Y | Y | Y | N | N | N | Y | Y | 70% |
| Harvey-Berino (2004)51 | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | 90% |
| Kypri (2005)52 | Y | Y | Y | Y | Y | Y | N | Y | N | N | 70% |
| Patrick (2001)53 | Y | Y | N | Y | N | Y | N | N | Y | Y | 60% |
| Plotnikoff (2005)54 | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | 90% |
| Prochaska (2004)55 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Southard (2003)56 | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | 90% |
| Tate (2001)57 | Y | Y | N | Y | N | Y | Y | Y | Y | Y | 80% |
| Tate (2003)58 | Y | Y | N | Y | Y | Y | Y | N | Y | Y | 80% |
| Veverka (2003)60 | Y | Y | Y | Y | Y | Y | N | N | NA | NA | 60% |
| Vandelanotte (2005)59 | Y | Y | Y | Y | N | UK | N | N | Y | Y | 60% |
| Williamson (2005)61 | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | 90% |
| Womble (2004)62 | Y | Y | N | Y | N | Y | Y | Y | NA | NA | 60% |
| Wylie-Rosett (2001)63 | Y | Y | N | Y | Y | N | N | Y | Y | Y | 70% |
| Study Quality Coding Criteria
| |
|---|---|
| Table heading | Scoring criteria |
| Individual randomization | Were participants randomized to study conditions? If so, was randomization at the individual level? Stratified and blocked randomization is acceptable. Studies that used individual randomization combined with a small proportion of randomized matched pairs are also considered YES. Appropriately designed and powered group randomization would also be acceptable if group was also unit of analysis. Individual randomization is NO when the authors fail to mention randomization, specify that another method of assigning group status was used, or randomize at the group level and analyze at the individual level. |
| Control group | Did the study include a comparison group? Comparison group could be a no treatment, treatment as usual, or alternate treatment group. |
| Isolate technology | Did study design allow for test of effectiveness of the technology? E.g., web-based delivery verses no treatment. To isolate the technology, the authors had to test the technology alone and compare to a group with no technology (YES). Packaged interventions where the technological components can't be parsed out are coded as not isolating the technology (NO). |
| Pre/post test design | Was assessment of behavior completed pre intervention and post intervention? |
| Retention | Was study retention at least 80% of subjects who initially agreed to participate in the study? Retention is calculated for the entire sample and not by group. For studies that did not report retention or dropout rates, retention can be calculated using the sample sizes used for analyses (e.g. 300 randomized, but only 250 were included in analyses = 83.3% retention). |
| BL Groups equivalent | Were tests conducted to determine if groups were equivalent at baseline on important variables (e.g., gender, age, weight)? If no tests mentioned, then = UK. If subset of tests indicated any group differences at baseline, then = NO. |
| Missing data | Were analyses conducted with consideration for missing data that maintains the fidelity of the randomization (e.g., intent-to-treat, imputation). Listwise case deletion (completer analysis) = NO, if only analysis conducted. If 100% retention then completer analysis is appropriate = YES. If authors compared the ‘dropped subgroup’ to the selected or randomized sample, but did not consider the impact of the dropped subgroup on randomization (e.g. ITT or imputation), then code as NO. |
| Sample size calculation | Was power analysis reported to determine study sample size? |
| Validity of measures | Did description of measures include reliability and validity information? If reference or coefficients, then YES. If well established measure that is known to be validated, then YES. For objective measures without validity evidence, if the objective measure is used as a proxy (e.g. food receipts for nutrition intake), then NO. If the objective measure is used as a direct measure of behavior (e.g. food receipts for food purchase), then YES. If validity not reported and measure unknown, then UK. |
| Total | Sum of Yes's |
aAll quality criteria rated as yes (Y), no (N), or unknown/unclear (UK)