Sample Size Parameters for Vaccination Coverage Cluster Surveys—Insights from Pakistan

Abstract Since the 1970s the World Health Organization (WHO) has promoted household surveys with a cluster sample design as an important method to estimate vaccination coverage in low- and middle-income countries. When calculating the survey sample size, planners begin with an educated guess at the outcome proportion and a target precision for the estimate and calculate an effective sample size that would achieve that precision if using a simple random sample. Then they account for correlated outcomes in the cluster sample using an estimated design effect (DEFF) that depends, among other things, on the unknown rate of homogeneity of the outcome as parameterized with an intracluster correlation coefficient (ICC). From the 1970s until 2015, WHO materials suggested that vaccination coverage surveys could be self-weighting and their ICCs would rarely exceed 1/6 or 0.1667. Recommendations drafted in 2015 and finalized in 2018 suggested using a probability sample with survey weights that vary and using a more conservative ICC value of 1/3 = 0.3333 for routine immunization surveys if there was no recent evidence-based estimate available. In this manuscript, we examine variability in survey weights and ICCs for 18 binary outcomes from two recent nationally representative vaccination coverage surveys in Pakistan with 152 districts serving as strata in the sample design, and either 49 or 64 clusters per stratum. The observed values of these parameters are summarized and used to (i) help planners use appropriately small values of ICC when the outcome is expected to have extreme values and (ii) suggest how WHO recommendations on ICC and weights might be refined in a forthcoming update.