Appendix A. USPSTF Quality Rating Criteria

Diagnostic Accuracy Studies

Criteria

• Screening test relevant, available for primary care, adequately described.
• Study uses a credible reference standard, performed regardless of test results.
• Reference standard interpreted independently of screening test.
• Handles indeterminate results in a reasonable manner.
• Spectrum of patients included in study.
• Sample size.
• Administration of reliable screening test.

Definition of Ratings Based on Above Criteria

Good: Evaluates relevant available screening test; uses a credible reference standard; interprets reference standard independently of screening test; reliability of test assessed; has few or handles indeterminate results in a reasonable manner; includes large number (more than 100) broad-spectrum patients with and without disease.

Fair: Evaluates relevant available screening test; uses reasonable although not best standard; interprets reference standard independent of screening test; moderate sample size (50 to 100 subjects) and a "medium" spectrum of patients.

Poor: Has important limitations such as: uses inappropriate reference standard; screening test improperly administered; biased ascertainment of reference standard; very small sample size of very narrow selected spectrum of patients.

Randomized Controlled Trials (RCTs) and Cohort Studies

Criteria

• Initial assembly of comparable groups: RCTs—adequate randomization, including concealment and whether potential confounders were distributed equally among groups; cohort studies—consideration of potential confounders with either restriction or measurement for adjustment in the analysis; consideration of inception cohorts.
• Maintenance of comparable groups (includes attrition, cross-overs, adherence, contamination).
• Important differential loss to followup or overall high loss to followup.
• Measurements: equal, reliable, and valid (includes masking of outcome assessment).
• Clear definition of interventions.
• Important outcomes considered.
• Analysis: adjustment for potential confounders for cohort studies, or intention-to-treat analysis for RCTs (i.e. analysis in which all participants in a trial are analyzed according to the intervention to which they were allocated, regardless of whether or not they completed the intervention).

Definition of Ratings Based on Above Criteria

Good: Meets all criteria: Comparable groups are assembled initially and maintained throughout the study (followup at least 80 percent); reliable and valid measurement instruments are used and applied equally to the groups; interventions are spelled out clearly; important outcomes are considered; and appropriate attention to confounders in analysis.

Fair: Studies will be graded "fair" if any or all of the following problems occur, without the important limitations noted in the "poor" category below: Generally comparable groups are assembled initially but some question remains whether some (although not major) differences occurred in followup; measurement instruments are acceptable (although not the best) and generally applied equally; some but not all important outcomes are considered; and some but not all potential confounders are accounted for.

Poor: Studies will be graded "poor" if any of the following major limitations exists: Groups assembled initially are not close to being comparable or maintained throughout the study; unreliable or invalid measurement instruments are used or not applied at all equally among groups (including not masking outcome assessment); and key confounders are given little or no attention.

Return to Document