Exemplifying a measurement validation strategy for rare- and ultra-rare diseases: measuring what matters in alpha-mannosidosis

Outcomes assessment in rare and ultra-rare diseases can be hampered by the paucity of condition-specific patient-reported outcome (PRO) measures that are known to be reliable, valid, and fit for purpose. It would be helpful to develop a strategy for measuring what matters to patients/caregivers in rare and ultra-rare diseases and for validating this measurement approach even with the constraint of very small samples. The present work aims to propose a measurement strategy that builds on a conceptual measurement model of core concerns in the context of an ultra-rare lysosomal storage disorder, namely Alpha-Mannosidosis (AM). Beginning with a comprehensive literature review of what is known about AM, we identified key domains to be assessed. We drafted items for use in a video-conferenced semi-structured interview to assess symptoms in all of the relevant domains. We then identified generic, validated proxy-reported evaluative tools to assess as many of these domains as possible for use in a web-based survey. Using data collected from caregivers of individuals with AM, we then evaluated aspects of validity of the interview and survey modes of data collection. The study sample included 16 AM caregivers reporting on the disease experience of 20 individuals with AM ranging in age from 4 to 49 years old. The new assessment package’s “validation” relied on a more qualitative or visual summary of the many aspects of validity, including content validity, criterion-related validity, known-groups validity, convergent and divergent validity, and longitudinal construct validity (i.e. responsiveness). It integrated information from causal (symptom measures) and effect (evaluative PRO measures) indicators to describe what quality of life concerns are linked to the AM symptom experience. Our approach emphasized effect sizes over p-values. This study provides a roadmap for creating meaningful outcome assessment in the context of rare and ultra-rare disease. This measurement strategy yielded an empirically-based description of the multidimensional impact of AM and characterized treatment benefits more comprehensively. The new assessment package might be used in longitudinal cohort or registry studies to capture the natural history of this ultra-rare disease, could inform future clinical trials, and capture real-world efficacy outcomes of targeted treatments.