How to Get Insightful and Useful Audience Input for User Friendly Tools (Text Version)

Slide presentation from the AHRQ 2009 conference

On September 14, 2009, Maritza E. Matheus made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (662 KB) (Plugin Software Help).


Slide 1

How to Get Insightful and Useful Audience Input for User Friendly Tools

Maritza E. Matheus, Principal
AHRQ Annual Conference
Monday, September 14, 2009 

Slide 2

Audience Exercise

  • Need: paper and pen
  • On middle of page, write "packing suitcase for business trip" and circle it 

Slide 3

What are we discussing today?

  • When is it useful to obtain audience input?
  • Why conduct formal testing?
  • Approaches
  • Testing mechanics
  • Do's and Don'ts
  • Examples of eliciting information
  • Useful tactical tips
  • Audience exercise 

Slide 4

When is it Useful to Obtain Audience Input?

  • Developing a consumer product
    • Planning a product
    • Pre-testing of a prototype
    • Post-testing of a model
  • Testing an existing product
  • Testing messages
  • Developing an advertising campaign 

Slide 5

Qualitative Research

  • To gather an in-depth understanding of human behavior; answers the "why"
  • Provides insights, feedback and information not gleaned from quantitative approaches
  • Key: use gleaned information into actionable items and strategies 

Slide 6

Approaches

  • Focus group
  • Dyad/Triad
  • One on One/In Depth Interviews
  • Telephone
  • Online
  • Blogs 

Slide 7

Approaches

ApproachParticipantsDurationProsCons
Focus Group8-101.5-2 hoursRich in insights, group dynamics$
Dyads/Triads2 or 31 -1.5 hoursProbe deepConduct numerous
One on One Interviews / In Depth Interviews1 person1 hour or lessFace to face, phone Delve deepNo group dynamics Conduct numerous
Telephone1 person1 hour or lessGood for difficult to schedule participantsMiss face to face cues, interaction
OnlineNumerous1 hour or lessGood for difficult to schedule participantsMiss face to face cues, interaction, $

 

Slide 8

Testing Mechanics

  • Clear Objectives
  • Planning!
  • Appropriate Approach or Combination of Approaches
  • Budget
  • Qualitative Researcher or In-house
  • Venue - field facility or other
  • Recruiting (Demographics; Recruiting Guide or Screener)
  • Moderating (Topic Areas; Discussion Guide)
  • Reporting 

Slide 9

Do's

  • Do allow enough time for testing, including approvals
  • Do be inclusive in selecting audience (users, non-users, clinicians, family, educators, etc)
  • Use tools: flip chart, markers, actual products
  • Do honor time limit set with participants
  • Do provide food
  • Do provide incentives - be creative
  • Do audiotape and video tape
  • Do set time limit on topic areas
  • Do pre and post testing 

Slide 10

Don'ts

  • Don't allow friends or family in same groups
  • Don't allow for "seasoned" participants
  • Don't accept all of the recruited participants (do over recruit)
  • Don't ask participants to share their job title or profession unless necessary
  • Don't ask "WHY" - use other ways to elicit information 

Slide 11

Examples of Eliciting Information

  • Flip Chart
    • Easel, markers
  • Design Team
  • Visualization
  • Word Bubbles
  • Picture Decks
  • Mind Map 

Slide 12

Word Bubbles

  • Use: when learning about sensitive, conflicting or politically-charged attitudes
  • Tools: handout with stick figures with bubbles

Two Stick Figures each one with cloud bubble coming from their heads and each figure has a rectangular shape coming from their mouths. The participant is given described a situation in which he/she and another person are face to face. The participant writes in the respective bubble what they are thinking, what they actually say; what they think the person is thinking and what the person actually says.

Slide 13

Picture Decks

  • Use: need to learn about emotional associations about an experience or an issue
  • Tools: deck of images

Photos of a spider in a web and a women's softball team in a huddle. 

Slide 14

Mind Mapping

  • Use: when learning more details or more indepth understanding of behavior, situation, or relationship
  • Tools: paper and markers

Flow chart with the words "Quit Day" in the middle and circled. From it, there are 3 branches. The first branch has "Triggers" in a circle, the second branch has "Bad Moods" in a square and from that, "Eating" in a square and from that "Weight Gain" in a square. These represent thoughts a participant may be having who is thinking about quitting smoking and as "quit day" approaches. 

Slide 15

Mind Map

  • Example of Completed Mind Map

Flow chart with Disney World is in the middle of the page and circled and it has several branches coming from it.

  • Disney World
    • Mickey Mouse, Commercialization, Childhood, Family
    • Florida, Flying, Long lines, Activities, Summer Clothes, Exercise - toning
    • Costly, Save $
    • Family vacation, Fun, Memorable, Kids cranky, No adult time 

Slide 16

Tactical Tips

  • Over-recruit Latinos and African-Americans
  • Pay for transportation for indigent or elderly
  • Mail publications in advance for participants to review before group testing
  • Best not to mix some participants, such as physician specialties or physicians with other clinicians
  • Consider on-site interviews when appropriate
  • Ask colleagues and observers for their insights 

Slide 17

Audience Exercise

  • Need: paper and pen
  • On middle of page, write "packing suitcase for business trip" and circle it
  • Example
  • Take 2 minutes to write thoughts that come to mind when packing a suitcase
  • When done, circle the most significant branch 

Slide 18

Thank You!

  • Questions?
  • Contact:
    • Maritza E. Matheus
    • Matheus Marketing, LLC
    • Oakton, VA 22124
    • Tel 703 758-7595
    • Email matheusmarket@aol.com
Current as of December 2009
Internet Citation: How to Get Insightful and Useful Audience Input for User Friendly Tools (Text Version). December 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2009/matheus/index.html