PROMIS: The Right Place at the Right Time? (Text Version)

Slide presentation from the AHRQ 2009 conference.

On September 15, 2009, David Cella made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (2.4 MB) (Plugin Software Help).


Slide 1

Slide 1. PROMIS: The Right Place at the Right Time?

PROMIS: The Right Place at the Right Time?

David Cella, Ph.D.
Department of Medical Social Sciences
Northwestern University
Chair, PROMIS Steering Committee
Principal Investigator, Statistical Coordinating Center

 

Slide 2

Slide 2. What is PROMIS?

What is PROMIS?

  • A nine-year commitment of NIH to improve and standardize measurement of patient reported outcomes (PROs)
  • 50 million dollar investment (excl. ARRA)
  • An effort to harness new psychometric methods to improve the quality and interpretability of PROs
  • An attack on the PRO "Tower of Babel"

 

Slide 3

Slide 3. PROMIS Roadmap Goal

PROMIS Roadmap Goal:

  • Create highly valid and reliable item banks and associated computerized adaptive testing
  • Encourage wide adoption to improve assessment of
    • Self-reported symptoms
    • Other health-related quality of life domains
      ...across a wide range of chronic diseases

 

Slide 4

Slide 4. PROMIS Network 2004-2009

An image of a map of the United States is shown. The map is marked with the PROMIS Network:

PROMIS Network: 2004-2009

  • UNC -Chapel Hill
  • Duke University
  • Stanford University
  • Stony Brook University
  • University of Pittsburgh
  • University of Washington
  • Northwestern
  • NIH

 

Slide 5

Slide 5. "Item Bank"

"Item Bank"

  • A large collection of items measuring one thing
  • Items in the same bank are linked on a common metric
  • Basis for
    • Computer Adaptive Testing (CAT)
      • Items are selected to maximize precision and retain clinical relevance
    • Customized short forms tailored to the population and research purpose

 

Slide 6

Slide 6. Building a Bank

Building a Bank

  • Items Deposited After Review
  • Testing and Analysis
  • Defined and Calibrated Bank
  • CAT and Multiple Short Forms

Slide 7

Slide 7. "PROMIS-1" Highlights

"PROMIS-1" Highlights

  • Consensus-driven PRO framework
  • Qualitative study of > 1,000 people
  • Quantitative study of > 20,000 people
  • Eleven Item Banks available to collaborators
  • Website and functioning Assessment Center
  • Ongoing clinical validation in chronic disease populations
  • Clear path/agenda for "PROMIS-2" and expanded collaborations

 

Slide 8

Slide 8. PROMIS Domain Framework

PROMIS Domain Framework

An image of a flow chart of the PROMIS Domain Framework is shown.

Slide 9

Slide 9. Item Banking

Item Banking

Item BankLocal IndependenceStatic Forms
Measurement RangeMulti-Stage
Target PopulationCAT

Item banks can support a variety of scale assembly strategies

Slide 10

Slide 10. Advantages of Short-Forms Developed from Item Banks

Advantages of Short-Forms Developed from Item Banks

  • Flexibility in length and content
    • Select items matched to clinical features and severity in the target population
    • Select items known to provide the most information
  • Any form selected or created produces scores on a common metric

 

Slide 11

Slide 11. PROMIS Profile Short Forms

PROMIS Profile Short Forms

Mental

  • Anxiety - 29
  • Depression - 28

Physical

  • Fatigue - 95
  • Pain Impact - 41
  • Sleep Disturbance - 27

Social

  • Physical Function - 86*
  • Social Role - 14

* reduced set (the full bank has 112 items) was used for real data simulation purposes

 

Slide 12

Slide 12. PROMIS Profiles Subscales: Test Information Functions

PROMIS Profile Subscales: Test Information Functions

 

Slide 13

Slide 13. PROMIS Profile Subscales: Correlations with Full Banks

PROMIS Profile Subscales: Correlations with Full Banks

 

Slide 14

Slide 14. PROMIS Profile Subscales: Correlations with Full Banks

Estimated Power to detect a small effect (d=0.2) when the study population is centered on the general population (T-Score = 50)

 

Slide 15

Slide 15. PROMIS Profile Subscales: Correlations with Full Banks

Estimated Power to detect a small effect (d=0.2) when the study population is centered on 1 SD worse than the general population

 

Slide 16

Slide 16. Computerized Adaptive Testing (CAT)

Computerized Adaptive Testing (CAT)

  • Select questions based on a person's response to previously administered questions.
  • Iteratively estimate a person's standing on a domain (e.g., depressive symptoms)
  • Administer most informative items
  • Desired level of precision can be obtained using the minimal possible number of questions.

 

Slide 17

Slide 17. Example of CAT

Beginning of CAT

T-Score = 50
SE = 10
Best Item - I felt depressed

 

Slide 18

Slide 18. Example of CAT

I felt depressed: Rarely

I felt depressed

  1. Never
  2. Rarely
  3. Sometimes
  4. Often
  5. Always

T-Score = 52
SE = 4

Next Best Item-I felt like a failure

 

Slide 19

Slide 19. Example of CAT

I felt depressed: Rarely

I felt like a failure

  1. Never
  2. Rarely
  3. Sometimes
  4. Often
  5. Always

T-Score = 53
SE = 3

Next Best Item-I felt worthless

 

Slide 20

Slide 20. Example of CAT

I felt depressed: Rarely

I felt worthless

  1. Never
  2. Rarely
  3. Sometimes
  4. Often
  5. Always

T-Score = 55
SE = 2

Next Best Item-I felt helpless

 

Slide 21

Slide 21. Example of CAT

I felt depressed: Rarely

I felt that nothing could cheer me up

  1. Never
  2. Rarely
  3. Sometimes
  4. Often
  5. Always

T-Score = 55
SE = 2

 

Slide 22

Slide 22. Fatigue

Fatigue

  • Experience and Impact
    • 95-item bank
  • Legacy Instruments
    • SF-36 Vitality Scale (4 items)
    • FACIT-Fatigue Scale (13 items)

 

Slide 23

Slide 23. Comparison of Measurement Precision

Comparison of Measurement Precision
Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

Standard Error vs. No Fatigue - Severe Fatigue

SE=0.32 (r=0.90)
SE=0.22 (r=0.95)

 

Slide 24

Slide 24. PROMIS Fatigue Short Form

PROMIS Fatigue Short Form

In the past 7days...

  1. How often did you feel tired?
  2. How often did you experience extreme exhaustion?
  3. How often did you run out of energy?
  4. How often did your fatigue limit you at work (include work at home)?
  5. How often were you too tired to take a bath or shower?
  6. How often did you have enough energy to exercise strenuously?

Rate between: Never, Rarely, Sometimes, Often, and Always

 

Slide 25

Slide 25. Comparison of Measurement Precision

Comparison of Measurement Precision
Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

7-item Short-form
7-item CAT
98-item Bank

Standard Error Vs. No Fatigue - Severe Fatigue
SE=0.32 (r=0.90)
SE=0.22 (r=0.95)

 

Slide 26

Slide 26. PROMIS Profile Subscales Vs. CATs

PROMIS Profile Subscales vs CATs:
Correlations with Full Banks

 

Slide 27

Slide 27. PROMIS Assessment Center and Opportunities for Interaction

PROMIS Assessment Center and Opportunities for Interaction

A publicly available, adaptable and sustainable Internet-based system that:

  1. Hosts 11 item banks for precise, valid, and efficient health status assessment via short forms or CAT
  2. Collects and analyze patients' responses
  3. Provides instant health status reports to users to:
    • Enable and enhance research
    • Improve clinical decision-making
    • Facilitate policy-making by health plan and systems and public programs
  4. Supports health services / comparative effectiveness research applications

 

Slide 28

Slide 28. Screen Capture of http://www.nihpromis.org

Assessment Center

A screen shot of http://www.nihpromis.org is shown.

http://www.nihpromis.org

 

Slide 29

Slide 29. PROMIScore - PSF

PROMIScore - PSF

An image of a Score Profile for John Doe is shown.

 

Slide 30

Slide 30. Assessment Center Aims

Assessment Center Aims

Assessment Center supports.

  • Research and clinical applications
  • CAT
  • Custom or off-the-shelf short forms
  • Non-PROMIS measures welcome
  • Study set up, maintenance and storage
  • Scoring, reporting and interpretation

 

Slide 31

Slide 31. 2009 PROMIS Ver 1.0 Instruments

2009 PROMIS Ver 1.0 Instruments

DomainsItems in
bank
Items in
First Short
Form
Emotional Distress- Anger298
Emotional Distress - Anxiety297
Emotional Distress - Depression288
Fatigue957
Pain - Behavior397
Pain - Impact416
Physical Function12410
Satisfaction with Discretionary Social Activities127
Satisfaction with Social Roles147
Sleep Disturbance278
Wake Disturbance168
Global Health 10

 

Slide 32

Slide 32. Thank you.

Thank you

http://www.nihpromis.org

Current as of December 2009
Internet Citation: PROMIS: The Right Place at the Right Time? (Text Version). December 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2009/cella/index.html