Assembling a Systematic Review Team:

Balancing Expertise and Potential Conflicts of Interest

Slide presentation from the AHRQ 2011 conference.

On September 20, 2011, Stephanie Chang made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (910 KB). .


Slide 1

Slide 1. Assembling a Systematic Review Team: Balancing Expertise and Potential Conflicts of Interest

Assembling a Systematic Review Team: Balancing Expertise and Potential Conflicts of Interest

Avoiding bias in systematic reviews
AHRQ Annual Meeting
Bethesda North Marriott Conference Center
September 20, 2011

Slide 2

Slide 2. IOM Recommendations

IOM Recommendations

Standard 2.1 Establish a team with appropriate expertise and experience to conduct the systematic review.

  • 2.1.1 Include expertise in the pertinent clinical content areas.

Standard 2.2 Manage bias and conflict of interest (COI) of the team conducting the systematic review.

  • 2.2.3 Exclude individuals whose professional or intellectual bias would diminish the credibility of the review in the eyes of the intended users.

Standard 2.4 Manage bias and COI for individuals providing input into the systematic review.

  • 2.4.2 Exclude input from individuals whose COI or bias would diminish the credibility of the review in the eyes of the intended users.

Standard 2.7 Submit the protocol for peer review.

  • 2.7.1 Provide a public comment period for the protocol and publicly report on disposition of comments.

Slide 3

Slide 3. Definitions

Definitions

  • Bias:
    • A systematic error or deviation from the truth, in results or inferences (Higgins JPT, et al. Cochrane handbook 2009).
    • Inclination to present or hold a partial perspective at the expense of (possibly equally valid) alternatives.
  • Conflict of interest (COI):
    • A set of conditions in which professional judgment concerning a primary interest (such as the validity of research) tends to be unduly influenced by a secondary interest (such as financial or professional gain) (Thompson, 1993).
  • Expertise:
    • Extensive knowledge based on research, experience, or occupation and in a particular area of study. Expert is someone who is a widely recognized authority by peers and public.

Slide 4

Slide 4. Potential biases in systematic reviews

Potential biases in systematic reviews

  • Holding an established opinion on the answers to the research questions of the review; the inclination to present or hold a partial perspective at the expense of (possibly equally valid) alternatives:
    • Individuals in a particular specialty may have an unbalanced familiarity for one type of intervention or technology.
    • Individuals livelihood may depend on finding greater benefit of one type of intervention or technology over a competitor.
    • Experts may hold a previously formed opinion and be unlikely to change.
  • Holding an established opinion on the quality of studies to be included:
    • Individuals who conducted a study or trial may not be able to assess the quality of their own or other "competitor" studies in an unbiased manner.

Slide 5

Slide 5. My take home points

My take home points

  • We are concerned with bias, but this is difficult to measure.
  • Non-financial conflicts of interest can result in bias:
    • Perceived conflicts of interest may still reduce the credibility of the review in the eyes of the intended users.
    • Conflicts due to familiarity or livelihood based on an intervention or technology can be managed by balancing the team and input from different specialty areas.
  • Expertise can be a form of bias:
    • May be difficult to discern unless they have published an editorial or commentary on the subject.
    • Not including appropriate expertise may reduce the credibility of the review in the eyes of the intended users.
    • It is important to hear opinions from all sides, but the core team should be without bias.
  • Individuals should not be involved in grading their own study, either at the individual study level or at the body of evidence level.

Slide 6

Slide 6. Introductions and Acknowledgements

Introductions and Acknowledgements

  • Susan Norris, Oregon Health and Science University.
  • Gillian Sanders Schmidler, Duke University.

AHRQ staff:

  • Elise Berliner.
  • Christine Chang.
  • Yen-Pin Chiang.
  • Supriya Janakiraman.
  • Elisabeth Kato.
  • Mary Nix.

Evidence-based Practice Center (EPC) workgroup:

  • Eric Bass.
  • Stanley Ip.
  • Melissa Mcpheeters.
  • Sydney Newberry.
  • Susan Norris.
  • Margaret Piper.
  • Paul Shekelle.
  • Meera Viswanathan.
  • Evelyn Whitlock.
  • Renee Wilson.
  • Michael White.

Slide 7

Slide 7. Objectives

Objectives

  • Provide a framework for thinking about intellectual conflict of interest.
  • Review of the existing literature on the potential effects of intellectual conflict of interest.
  • Work through examples of potential conflicts of interest when conducting a systematic review.

Slide 8

Slide 8. Case Examples

Case Examples

Slide 9

Slide 9. Procedure specialist

Procedure specialist

A review is comparing surgical, medical or alternative treatments. Should a specialist who makes a living doing only surgical interventions be the principal investigator?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 10

Slide 10. Procedure specialist

Procedure specialist

A review is comparing surgical, medical or alternative treatments. Should a specialist who makes a living doing only surgical interventions be part of the core team?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 11

Slide 11. Professional society staff

Professional society staff

Should a scientist who works for a specialty professional society be the principal investigator of a review?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 12

Slide 12. Professional society staff

Professional society staff

Should a scientist who works for a specialty professional society be a co-author of a review?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 13

Slide 13. Professional society staff

Professional society staff

Should a scientist who works for a specialty professional society be on the technical expert panel?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 14

Slide 14. Professional society staff

Professional society staff

Should a scientist who works for a specialty professional society be a peer reviewer?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 15

Slide 15. Active research in area

Active research in area

A scientist has federal grants to study one of the intervention of interest. Should this person be the principal investigator of the review?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 16

Slide 16. Active research in area

Active research in area

A scientist has federal grants to study one of the intervention of interest. Should this person be a co-author of the review?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 17

Slide 17. Active research in area

Active research in area

A scientist has federal grants to study one of the intervention of interest. Should this person be on the technical expert panel?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 18

Slide 18. Active research in area

Active research in area

A scientist has federal grants to study one of the intervention of interest. Should this person be a peer reviewer?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 19

Slide 19. Published

Published

A scientist was an author of a study that would potentially be included in the review. Should this person be the principal investigator?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 20

Slide 20. Published

Published

A scientist was an author of a study that would potentially be included in the review. Should this person be on the core team?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.

Slide 21

Slide 21. Published

Published

A scientist was an author of a study that would potentially be included in the review. Should this person be on the technical expert panel?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.

Slide 22

Slide 22. Published

Published

A scientist was an author of a study that would potentially be included in the review. Should this person be a peer reviewer?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 23

Slide 23. Involved in study

Involved in study

A scientist coordinated one center of a multi-center trial that would potentially be included in a review. Should this be the principal investigator?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 24

Slide 24. Involved in study

Involved in study

A scientist coordinated one center of a multi-center trial that would potentially be included in a review. Should this be part of the core team?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 25

Slide 25. Published review or guidelines

Published review or guidelines

A scientist has previously published a review or guideline on the subject of the review. Should this person be the principal investigator?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 26

Slide 26. Published review or guidelines

Published review or guidelines

A scientist has previously published a review or guideline on the subject of the review. Should this person be a co-author?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 27

Slide 27. Scientific advisor or editor

Scientific advisor or editor

A scientist serves on an advisory or editorial board without financial compensation on a related topic to the review. Should this person be the principal investigator?

  1. Yes.
  2. Yes, but with restrictions.
  3. No.
  4. It depends.

Slide 28

Slide 28. Scientific advisor or editor

Scientific advisor or editor

A scientist serves on an advisory or editorial board without financial compensation on a related topic to the review. Should this person be a co-author?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 29

Slide 29. Scientific advisor or editor

Scientific advisor or editor

A scientist serves on an advisory or editorial board without financial compensation on a related topic to the review. Should this person be on the technical expert panel?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.

Slide 30

Slide 30. Scientific advisor or editor

Scientific advisor or editor

A scientist serves on an advisory or editorial board without financial compensation on a related topic to the review. Should this person be a peer reviewer?

  1. Yes.
  2. Yes, but balance with other perspectives.
  3. Yes, but with restrictions.
  4. No.
  5. It depends.
Page last reviewed March 2012
Internet Citation: Assembling a Systematic Review Team: : Balancing Expertise and Potential Conflicts of Interest. March 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2011/chang/index.html