Page 1 of 1

Table 7: Criteria for assigning strength of evidence

Assessing the Evidence for Context-Sensitive Effectiveness and Safety

Does this evidence help me decide whether I can implement this PSP in my organization and get a similar result?

Type of evidence:
Randomized trial = high

Decrease score if:

  • No explanation of why the PSP might work, either in terms of theory, logic models, or prior success in other fields or in pilot studies (-1).
  • No reporting of contexts, including at least structural organizational characteristics, external factors, patient safety culture, leadership, teamwork, or implementation tools (-1).
  • PSP not described in sufficient detail to permit replication (-1).
  • No reporting of the implementation process, assessment of unplanned events, or changes to workflow (-1).
  • No assessment of the effect of contexts on implementation effectiveness (-1).

Observational study = low

Increase score if:

  • Consistent results obtained in multiple studies (+2).
  • Use of observational study designs of stronger internal validity (controlled before-and after, time series, statistical process control) (+1).
  • Very strong effect (+1).
  • Use of theory/logic models, assessment of contexts, reporting of implementation process, and fidelity of implementation (+1).

Any other evidence = insufficient

Across all study types, decrease score if:

  • Serious (-1) or very serious (-2) limitation to study quality.
  • Important inconsistency across studies (-1).
  • Imprecise or sparse data (-1).
  • High probability of reporting bias (-1).

Return to Document

Current as of December 2010
Internet Citation: Table 7: Criteria for assigning strength of evidence: Assessing the Evidence for Context-Sensitive Effectiveness and Safety . December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/findings/final-reports/contextsensitive/contexttab7.html