Effect of specificity of health expenditure questions in the measurement of out-of-pocket health expenditure: Evidence from field experimental study in Ghana

Background: The effect of number of health items on OOPs has been identified as a source of bias in measuring OOPs. Evidence comes mostly from cross-sectional comparison of different survey instruments to collect data on OOPs. Very few studies have attempted to validate these questionnaires, or distinguish bias arising from the comprehensiveness of the OOPs list versus specificity of OOPs questions.

Objectives: This study aims to estimate biases arising from the specificity of OOPs questions by comparing provider and household’s information.

Methods: A generic questionnaire to collect data on household’s OOPs was developed following the nomenclature proposed in division 6 of the COICOP2018. The four categories within such division are used to set the comprehensiveness of the OOPs list, the specificity within each category was tailored to the design of the nationally representative living standard survey in Ghana where a field experiment was conducted to test the validity of different versions. Households were randomized to 11, 44 or 56 health items. Using data from provider records as the gold standard, we compared the mean positive OOPs, and estimated the mean ratio and variability in the ratio of household expenditures to provider data for the individual households using the Bland-Altman method of assessing agreement.

Findings: We found evidence of a difference in the overall mean ratio in the specificity for OOPs in inpatient care and medications. Within each of these two categories, a more detailed disaggregation yielded lower OOPs estimates than less detailed ones. The level of agreement between household and provider OOPs also decreased with increasing specificity of health items.

Conclusion: Our findings suggest that, for inpatient care and medications, systematically decomposing OOPs categories into finer sub-classes tend to produce lower OOPs estimates. Less detailed items produced more accurate and reliable OOPs estimates in the context of a rural setting.

Methods
This study uses health provider records as the ‘gold standard’ to compare to household reported OOPs, recognizing that the provider records are not a perfect gold standard in the absence of a pre-existing formal recording system in place. Two sets of data were collected in this study. The first set of data was captured from households in a cross-sectional field survey conducted between May 2017 and December 2017 and the second set of data was obtained from health provider records within the same period.

Households were randomized to one of three versions of a household questionnaire on consumption expenditure for face-to-face interview. All three versions of the questionnaire were fielded during the same time period and included questions on out-of-pocket health spending for inpatient care services, preventive care services, other outpatient care services, other health services, medicines and health products. They however differed in the level of specificity within each one of these main OOPs categories. The versions were labeled; version-1 for the questionnaire with 11 items covering the 6 health categories previously mentioned version-2 and 3 for the questionnaire versions with a maximum of 44 and 56 items respectively. All three versions of the instrument used similar recall periods of 4 weeks for items listed under the categories medicines, other outpatient services, other health services; 12 months for inpatient care services and other health products; and 6 months for OOPs related to preventive care services. 

We created a database of provider records to validate the household reported OOPs. Both private and public health care providers within the study area were engaged by the project team to extract records covering a 13 month period which in some cases (mostly for private providers) also required improving the recording of provider records.  Health expenditures reported by any member of the household were tracked and a corresponding health provider record obtained to create a matched sample for validation. A detailed description of the matching procedure can be seen in Supplementary material one under the section Matching. Matched sample in this study refers to one-on-one paired records households reported OOPs to heath provider recorded OOPs for the same reason of incurring the health cost. Unmatched households in this study also refers to all OOPs reported by households without pairing such expenditures to corresponding health provider records.  Household heads were the most common respondents for the household survey but in some cases, other individuals within the household were nominated by the household head to provide responses.