Measuring High-Priority Outcomes in Autistic Adults: Initial psychometric assessment of the instruments in the AASPIRE Measurement Toolkit

Abstract Background Adult autism services research is hampered by a lack of accessible self-reported outcome measures. The AASPIRE Outcomes Project used a community-based participatory research (CBPR) approach to create and test the AASPIRE Measurement Toolkit, a set of accessible survey instruments for use in real-world settings. The core toolkit contains 12 characteristics modules and 19 outcome measures, each with self-reported and caregiver-reported versions. Methods In a prior phase of the project, we collaboratively adapted, revised, or co-created all instruments. We used our CBPR-nested Delphi process, our collaborative adaptation/creation process, and cognitive interviews to ensure accessibility and content validity. We then conducted a longitudinal survey to validate the 19 outcome measures in a pragmatic sample of 870 autistic adults from two healthcare systems, two disability service systems, and the larger autistic community in the United States. Participants completed surveys at 3 time points over 12-18 months. A 15% random subset completed an additional retest survey 2 weeks after the second time point. We assessed 1) accessibility using completion rates and perceived ease of use; 2) internal consistency using Cronbach’s alphas and omegas; 3) convergent validity using Pearson’s correlations; 4) two-week test-retest reliability using interclass correlation coefficients; and 5) six-month responsiveness to change by comparing self-perceived change with change in scores. Results Over 90% of participants reported the survey items were easy to understand; over 90% of participants who started the survey completed all applicable sections at each time point; and participants answered 99% of items on each instrument. The outcome measures and their pre-determined subscales demonstrated strong accessibility, content validity, internal consistency reliability, test-retest reliability, convergent and discriminant validity, and responsiveness to change. Conclusion The AASPIRE Measurement Toolkit is accessible and includes 19 outcome measures with strong initial psychometric properties. We will report in-depth assessments of construct and structural validity separately for each measure. All instruments are available for free and can help clinicians, service providers, advocacy organizations, and researchers assess the effectiveness of interventions and follow changes in outcomes over time. Community Brief Why is this an important issue? Autistic adults have worse health and social outcomes than the general population. To learn how to improve outcomes, we need an accurate way to measure them. Most survey measures that exist are hard for autistic people to use, not tested with autistic people, or can’t measure if the outcome has changed. What was the purpose of this study? The AASPIRE Measurement Toolkit is a set of accessible questionnaires (also known as “survey instruments”) that can be used to measure the outcomes most important to autistic adults. The purpose of this study was to test how well these survey instruments work. What did the researchers do? We used a community-based participatory research (CBPR) approach, where scientists and autistic community members worked together as equal partners. We collaboratively selected, adapted, and created the instruments in the Measurement Toolkit. We made a version for autistic people and a version for caregivers. The 19 outcome measures focused on quality of life, health outcomes (overall health, flourishing, depression, anxiety, autistic burnout), social outcomes (community participation, self-determination, choices and decisions, barriers to communication, social support, employment), and services (healthcare services, disability services, and adequacy of supports and services). We used the instruments with a diverse group of 870 autistic adults and caregivers. They took the survey three times over about a year. We used statistics and feedback to test how well the instruments worked. What were the results and conclusions of the study? The survey measures tested very well: 1) They were accessible: people gave spontaneous praise for the survey’s accessibility, calling it “thoughtfully-designed,” “non-ambiguous,” and “fabulous.” Almost everyone answered all the survey questions on the survey and said it was easy to use. 2) They were “internally consistent” - that means people gave similar responses to different questions about the same idea. 3) They had good “test-retest reliability” - that means, as expected, responses didn’t change much over a two-week period. 4) They had good “convergent validity” - that means that they related to each other in expected ways. 5) They were “responsive to change” - changes in scores over 6 months matched participants’ own perceptions of change. Researchers and service providers should use our measures. We feel our measures work well because autistic people were involved at every step. What is new or controversial about these findings? Previously, there weren’t studies to see if measures for these outcomes both worked for autistic adults and could detect change. This study furthers what we know about adapting and co-creating accessible, accurate measures for autistic adults using CBPR. What are potential weaknesses in the study? Not enough caregivers took part to test their version of the measures. We think these measures would work for other people too, but we only tested them with autistic adults. How will these findings help autistic adults now or in the future? These measures will help researchers and service providers know if what they are doing to improve outcomes is working. That will help them create services that can do a better job at improving autistic peoples’ lives.