Engaging the Public on the Use of Evidence: The Role of Public Deliberation

Select to access the PowerPoint® presentation (310 KB).

Slide 1

Engaging the Public on the Use of Evidence: The Role of Public Deliberation

Kristin L. Carman, PhD
Maureen Maurer, MPH

Slide 2

Session Objectives

• Explain public deliberation and application in health care.
• Identify best practices for successful public deliberation.
• Describe AHRQ's Community Forum deliberative methods experiment.

Slide 3

Reasons For This Work

• 3-year, ARRA-funded initiative of Agency for Healthcare Research and Quality.
• Seeks to:
  1. Expand the evidence base about public deliberation.
  2. Obtain public input on a topic of value to AHRQ: the use of evidence in health care decisionmaking.

Slide 4

Overview of Public Deliberation

Slide 5

What is Public Deliberation?

• Public consultation approach.
• Involves lay members of the public.
• Includes an educational component and a discussion-based (or deliberative) component.
• Suited for ethical or values-based social issues.
• Concerned with the "greater good" or societal interests.

Slide 6

Core Elements of Public Deliberation

Image: A chart shows the following process:

Convene:

• Individuals from a broad range of perspectives gather through live or virtual channels.

Learn:

• Objective overview of background, issues, and options are presented.

Deliberate:

• Participants exchange reasons, perspectives, and values.

Outcome: Impact on individual participants.

Report:

• A summary or transcript of deliberation prepared for decisionmakers.

Outcome: Impact on civic decisionmaking.

Slide 7

How Deliberation Differs From Other Public Consultation Methods

• Educational.
• Participant-based dialogue.
• Reason-based.
• Societal perspective & mutual responsibility.
• Challenging.

Slide 8

Examples of Deliberative Methods

• Citizens' Jury.
• Citizens' Panel /Council.
• Deliberative Focus Group.
• Deliberative Poll®.
• Issues Forum.
• Study Circles.
• Town Hall.
• Hybrid approaches.

Slide 9

How Deliberative Methods Vary

• Group size, participant sample.
• Length, duration.
• Mode (online, in-person).
• Recruitment method.
• Use of educational materials and experts.
• Facilitation.
• Consensus as goal.

Slide 10

What Are the Outcomes Public Deliberation?

• Informed public input to the sponsor:
  • Summary of important themes in participants' views.
  • Can be used to inform policy, programmatic, or other decisions.

Slide 11

What Are the Outcomes Public Deliberation? (cont.)

• Impact on participants:
  • Increased knowledge of the deliberative topic.
  • Change in attitudes on deliberative topic.
  • Increased willingness to participate in civic activities.
  • Adoption of societal concerns / shift from personal preferences.

Slide 12

Application in Health Care

Slide 13

Real-world Applications of Deliberative Methods

• Examples:
  • Developing a fair cost-sharing structure (Ginsburg et al., 2012).
  • Priority-setting social and health interventions (Pesce et al., 2011).

Slide 14

Developing a Fair Cost-sharing Structure

• Issue: Cost-sharing (deductibles, co-payments) can have varying financial impact on patients depending on their health care needs. What is the fairest way to structure cost-sharing when there is diversity of needs?
• Researchers: The California Health Benefit Exchange and the Center for Healthcare Decisions.
• Goals: Learn how future Exchange users prioritize the health care needs that should have greatest consideration for affordable cost-sharing.
• Evaluation: Assess response to participating in the deliberative process and its importance as input to the design of new health insurance programs.

Slide 15

Prioritizing Interventions

• Issue: How would you determine which social or health services to provide to improve health?
• Researchers: National Institutes of Health, Howard University, and D.C. Department of Health.
• Goals: Learn how participants prioritize social or health services to improve health and understand their reasoning.
• Evaluation: Assess deliberative process and whether deliberation affected participants' knowledge on the determinants of health.

Slide 16

Best Practices for Successful Public Deliberation

Slide 17

What Conditions Enable Good Public Deliberation?

• Clear deliberative goals.
• Participant trust in the process and outcomes.
• Accurate, unbiased information (education, experts).
• Reason-giving.
• Diversity.

Slide 18

Design Considerations

• Setting goals.
• Identifying and recruiting participants.
• Selecting deliberative process or method.
• Educating session participants.
• Facilitating sessions.
• Synthesizing output.

Slide 19

Areas for Further Study

• Understanding what works best:
  • Understanding impact on decisionmakers and policy.
  • Which deliberative methods work best.
• How to address inequalities within context of deliberation:
  • Recruitment strategies and incentives.
  • Measuring equal participation.
  • Language considerations.

Slide 20

Check In

• Questions so far...
• Discussion:
  • What issues do you see as appropriate for deliberation?

Slide 21

AHRQ's Community Forum Experiment

Slide 22

Deliberative Methods Experiment: RCT

Image: A chart shows the following process conducted for a randomized controlled trial (RCT):

• Eligible Participants Randomized:
  • Deliberative Methods: N=960:
    • Pre/Post K&A Survey:
      • DM vs. Control, Compare DMs.
    • Deliberative Experiences Survey:
      • Compare DMs to each other.
    • Qualitative Data:
      • Compare DMs to each other.
  • Control: N=336:
    • Pre/Post K&A Survey:
      • DM vs. Control.

76 deliberative groups in 4 U.S. cities in 4 months.

Slide 23

Deliberative Methods Experiment: RCT

Slide 24

Overarching Deliberative Question

• Deliberate on the use of evidence in healthcare decisionmaking:
  • Should individual patients and/or their doctors be able to make any health decisions no matter what the evidence of medical effectiveness shows, or should society ever specify some boundaries for these decisions?
• Will explore 3 "variations" on the theme:
  • Decisionmaking to encourage better health care.
  • Decisionmaking when there are cost implications.
  • Decisionmaking when there are complex societal tradeoffs.

Slide 25

Sub-questions by Variation

Slide 26

Deliberative Questions Across Methods

Slide 27

Overview Community Forum Evaluation

Slide 28

Overall Process For Evaluation

Image: A flowchart shows the following process:

• Determine measurement approach:
  • Develop surveys:
    • Collect survey data & analyze:
      • Combine data for further analysis.
  • Develop codebook for transcripts:
    • Code & analyze transcripts:
      • Combine data for further analysis.
  • Develop debrief forms:
    • Analyze observation data:
      • Combine data for further analysis.

  Slide 29

  

  Quantitative Analysis: Mapping Concepts to Surveys

  Survey	Group	Measure
  Pre/Post Knowledge & Attitude Survey	Deliberative methods and control	Participants' knowledge, attitudes, and beliefs about the deliberative issue
  Post Deliberative Experience Survey	Deliberative methods only	Discourse quality (e.g., equal participation, tolerance of different perspectives, reasoned justification of ideas) Implementation quality (e.g., quality of facilitation, materials, and presentations)

  
  

  Slide 30

  

  Qualitative Analysis: Aims

  1. Summarize values and ethical principles cited by participants.
  2. Assess whether values and ethical principles differ by method.

  Slide 31

  

  Qualitative Analysis: Approach

  Focus of analyses	Data source	Approach
  Thematic analysis: Ethics and values. Acceptable boundaries. Factors.	Transcripts	Coding—descriptive and interpretive coding Summarized for all groups and then used in comparisons between specific methods.
  Quality of process: Aspects affecting nature of deliberation	Transcripts & debrief forms	Coding—descriptive and interpretive coding; counts of occurrences Used in comparisons between specific methods.

  
  

  Slide 32

  

  How the Public Views the Application of Evidence

  • What and how does the public think about applying medical evidence?
  • What matters most to the public?
  • How can public input be used to inform CER?

  Slide 33

  

  Integrating Quantitative and Qualitative Analyses

  Quantitative	Qualitative
  Relatively high or low scores on Deliberative Experience survey for certain methods—or groups	Deliberative quality/ process elements (e.g., equal opportunity to participate)
  Differences in knowledge and attitudes scores on K & A survey for certain methods—or groups	Deliberative content (e.g., shifts, emphasis on educational materials or expert contributions)
  Differences in societal vs. individual perspective based on attitude questions	Deliberative quality and content (e.g., emphasis on societal perspective, articulation of overall question)

  
  

  Slide 34

  

  Next Steps and Resources

  • Next steps:
    • Implement experiment: now—fall 2012.
    • Analysis: summer 2012—summer 2013.
    • Report of findings: fall 2013.
  • Information to look for on the EHC Program Web site:
    • Webinars:
      • Using Deliberative Methods to Engage the Public: How to Design and Implement an Effective Deliberative Session (April 2012).
      • Using Deliberative Methods to Engage Patients, Consumers, and the Public (December 2011).
    • Literature Review (coming soon)
      http://www.effectivehealthcare.ahrq.gov/index.cfm/tools-and-resources/how-to-get-involved-in-the-effective-health-care-program/.

Slide 35

Key Contacts

• American Institutes for Research:
  • Kristin L. Carman, Project Director:
    • kcarman@air.org.
  • Maureen Maurer:
    • mmaurer@air.org.
• AHRQ:
  • Joanna Siegel, Project Officer:
    • Joanna.Siegel@ahrq.hhs.gov, 301-427-1969.

AHRQ's Effective Health Care Program site:
http://www.effectivehealthcare.ahrq.gov/index.cfm/tools-and-resources/

Slide 36

Questions?