PROMIS: The Right Place at the Right Time? (Text Version)

On September 15, 2009, David Cella made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (2.4 MB) (Plugin Software Help).

Slide 1

PROMIS: The Right Place at the Right Time?

David Cella, Ph.D.
Department of Medical Social Sciences
Northwestern University
Chair, PROMIS Steering Committee
Principal Investigator, Statistical Coordinating Center

Slide 2

What is PROMIS?

• A nine-year commitment of NIH to improve and standardize measurement of patient reported outcomes (PROs)
• 50 million dollar investment (excl. ARRA)
• An effort to harness new psychometric methods to improve the quality and interpretability of PROs
• An attack on the PRO "Tower of Babel"

Slide 3

PROMIS Roadmap Goal:

• Create highly valid and reliable item banks and associated computerized adaptive testing
• Encourage wide adoption to improve assessment of
  • Self-reported symptoms
  • Other health-related quality of life domains
    ...across a wide range of chronic diseases

Slide 4

An image of a map of the United States is shown. The map is marked with the PROMIS Network:

PROMIS Network: 2004-2009

• UNC -Chapel Hill
• Duke University
• Stanford University
• Stony Brook University
• University of Pittsburgh
• University of Washington
• Northwestern
• NIH

Slide 5

"Item Bank"

• A large collection of items measuring one thing
• Items in the same bank are linked on a common metric
• Basis for
  • Computer Adaptive Testing (CAT)
    • Items are selected to maximize precision and retain clinical relevance
  • Customized short forms tailored to the population and research purpose

Slide 6

Building a Bank

• Items Deposited After Review
• Testing and Analysis
• Defined and Calibrated Bank
• CAT and Multiple Short Forms

Slide 7

"PROMIS-1" Highlights

• Consensus-driven PRO framework
• Qualitative study of > 1,000 people
• Quantitative study of > 20,000 people
• Eleven Item Banks available to collaborators
• Website and functioning Assessment Center
• Ongoing clinical validation in chronic disease populations
• Clear path/agenda for "PROMIS-2" and expanded collaborations

Slide 8

PROMIS Domain Framework

An image of a flow chart of the PROMIS Domain Framework is shown.

Slide 9

Item Banking

Item banks can support a variety of scale assembly strategies

Slide 10

Advantages of Short-Forms Developed from Item Banks

• Flexibility in length and content
  • Select items matched to clinical features and severity in the target population
  • Select items known to provide the most information
• Any form selected or created produces scores on a common metric

Slide 11

PROMIS Profile Short Forms

Mental

• Anxiety - 29
• Depression - 28

Physical

• Fatigue - 95
• Pain Impact - 41
• Sleep Disturbance - 27

Social

• Physical Function - 86*
• Social Role - 14

* reduced set (the full bank has 112 items) was used for real data simulation purposes

Slide 12

PROMIS Profile Subscales: Test Information Functions

Slide 13

PROMIS Profile Subscales: Correlations with Full Banks

Slide 14

Estimated Power to detect a small effect (d=0.2) when the study population is centered on the general population (T-Score = 50)

Slide 15

Estimated Power to detect a small effect (d=0.2) when the study population is centered on 1 SD worse than the general population

Slide 16

Computerized Adaptive Testing (CAT)

• Select questions based on a person's response to previously administered questions.
• Iteratively estimate a person's standing on a domain (e.g., depressive symptoms)
• Administer most informative items
• Desired level of precision can be obtained using the minimal possible number of questions.

Slide 17

Beginning of CAT

T-Score = 50
SE = 10
Best Item - I felt depressed

Slide 18

I felt depressed: Rarely

I felt depressed

1. Never
2. Rarely
3. Sometimes
4. Often
5. Always

T-Score = 52
SE = 4

Next Best Item-I felt like a failure

Slide 19

I felt depressed: Rarely

I felt like a failure

1. Never
2. Rarely
3. Sometimes
4. Often
5. Always

T-Score = 53
SE = 3

Next Best Item-I felt worthless

Slide 20

I felt depressed: Rarely

I felt worthless

1. Never
2. Rarely
3. Sometimes
4. Often
5. Always

T-Score = 55
SE = 2

Next Best Item-I felt helpless

Slide 21

I felt depressed: Rarely

I felt that nothing could cheer me up

1. Never
2. Rarely
3. Sometimes
4. Often
5. Always

T-Score = 55
SE = 2

Slide 22

Fatigue

• Experience and Impact
  • 95-item bank
• Legacy Instruments
  • SF-36 Vitality Scale (4 items)
  • FACIT-Fatigue Scale (13 items)

Slide 23

Comparison of Measurement Precision
Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

Standard Error vs. No Fatigue - Severe Fatigue

SE=0.32 (r=0.90)
SE=0.22 (r=0.95)

Slide 24

PROMIS Fatigue Short Form

In the past 7days...

1. How often did you feel tired?
2. How often did you experience extreme exhaustion?
3. How often did you run out of energy?
4. How often did your fatigue limit you at work (include work at home)?
5. How often were you too tired to take a bath or shower?
6. How often did you have enough energy to exercise strenuously?

Rate between: Never, Rarely, Sometimes, Often, and Always

Slide 25

Comparison of Measurement Precision
Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

7-item Short-form
7-item CAT
98-item Bank

Standard Error Vs. No Fatigue - Severe Fatigue
SE=0.32 (r=0.90)
SE=0.22 (r=0.95)

Slide 26

PROMIS Profile Subscales vs CATs:
Correlations with Full Banks

Slide 27

PROMIS Assessment Center and Opportunities for Interaction

A publicly available, adaptable and sustainable Internet-based system that:

1. Hosts 11 item banks for precise, valid, and efficient health status assessment via short forms or CAT
2. Collects and analyze patients' responses
3. Provides instant health status reports to users to:
   • Enable and enhance research
   • Improve clinical decision-making
   • Facilitate policy-making by health plan and systems and public programs
4. Supports health services / comparative effectiveness research applications

Slide 28

Assessment Center

A screen shot of http://www.nihpromis.org is shown.

http://www.nihpromis.org

Slide 29

PROMIScore - PSF

An image of a Score Profile for John Doe is shown.

Slide 30

Assessment Center Aims

Assessment Center supports.

• Research and clinical applications
• CAT
• Custom or off-the-shelf short forms
• Non-PROMIS measures welcome
• Study set up, maintenance and storage
• Scoring, reporting and interpretation

Slide 31

2009 PROMIS Ver 1.0 Instruments

Slide 32

Thank you

http://www.nihpromis.org