Appendix Table 3. Criteria for Grading the Internal Validity of Individual Studies

The Methods Work Group for the U.S. Preventive Services Task Force developed a set of criteria to determine how well individual studies were conducted (internal validity) (22). The Task Force defined a 3-category rating of "good," "fair," and "poor," based on these criteria. In general, a good study is one that meets all criteria well. A fair study is one that does not meet, or it is not clear that it meets, at least one criterion but has no known important limitation that could invalidate its results. A poor study has important limitations. These specifications are not meant to be rigid rules but rather are intended to be general guidelines, and individual exceptions, when explicitly explained and justified, can be made.

Randomized Controlled Trials

• Adequate randomization, including concealment and equal distribution of potential confounders among groups.
• Maintenance of comparable groups (includes attrition, cross-overs, adherence, contamination).
• Important differential loss to followup or overall high loss to followup.
• Equal, reliable, and valid measurements (includes masking of outcome assessment).
• Clear definition of interventions.
• Important outcomes considered.
• Intention-to-treat analysis.

Case-Control Studies

• Accurate ascertainment of cases.
• Nonbiased selection of case patients and controls with exclusion criteria applied equally to both.
• High response rate.
• Diagnostic testing procedures applied equally to each group.
• Measurement of exposure accurate and applied equally to each group.
• Appropriate attention to potential confounding variables.

Cohort Studies

• Consideration of potential confounders with either restriction or measurement for adjustment in the analysis; consideration of inception cohorts.
• Maintenance of comparable groups (includes attrition, cross-overs, adherence, contamination).
• Important differential loss to followup or overall high loss to followup.
• Equal, reliable, and valid measurements (includes masking of outcome assessment).
• Clear definition of interventions.
• Important outcomes considered.
• Adjustment for potential confounders in analysis.

Diagnostic Accuracy Studies

• Screening test relevant, available for primary care, adequately described.
• Study uses a credible reference standard, performed regardless of test results.
• Reference standard interpreted independently of screening test.
• Handles indeterminate results in a reasonable manner.
• Spectrum of patients included in study.
• Adequate sample size.
• Administration of reliable screening test.

Return to Document