Table 7: Criteria for assigning strength of evidence

Assessing the Evidence for Context-Sensitive Effectiveness and Safety

Does this evidence help me decide whether I can implement this PSP in my organization and get a similar result?

Type of evidence:
Randomized trial = high

Decrease score if:

• No explanation of why the PSP might work, either in terms of theory, logic models, or prior success in other fields or in pilot studies (-1).
• No reporting of contexts, including at least structural organizational characteristics, external factors, patient safety culture, leadership, teamwork, or implementation tools (-1).
• PSP not described in sufficient detail to permit replication (-1).
• No reporting of the implementation process, assessment of unplanned events, or changes to workflow (-1).
• No assessment of the effect of contexts on implementation effectiveness (-1).

Observational study = low

Increase score if:

• Consistent results obtained in multiple studies (+2).
• Use of observational study designs of stronger internal validity (controlled before-and after, time series, statistical process control) (+1).
• Very strong effect (+1).
• Use of theory/logic models, assessment of contexts, reporting of implementation process, and fidelity of implementation (+1).

Any other evidence = insufficient

Across all study types, decrease score if:

• Serious (-1) or very serious (-2) limitation to study quality.
• Important inconsistency across studies (-1).
• Imprecise or sparse data (-1).
• High probability of reporting bias (-1).

Return to Document