Appendix 2. Fatal Flaws

Fatal Flaws:

1) "General" or "Methodological"—any single flaw is fatal

a) No (incremental) comparison of costs and benefits
b) No sensitivity analysis
c) Focus on intermediate outcomes, rather than final health outcomes
d) Inappropriate focus on short-term outcomes only

2) "Topic-specific"—evaluated in the context of other analysis characteristics

a) Exclusion of relevant cost category
b) Inappropriate cost sources and/or measurement
c) Exclusion of important harms
d) Inappropriate interventions and/or populations
e) Inappropriate comparators—e.g., comparing only with usual care (or doing nothing) when a current alternative is well known to be more cost-effective than usual care
f ) Inappropriate time horizon
g) Not conducted from a societal perspective and not representing a valid perspective clinically relevant to the topic
h) Not using best evidence to estimate effectiveness
i) Not capturing important utilities or preferences in effect measures
j) No discount rate or different discount rates for costs and health benefits

Return to Document