Health Results Based Financing Impact Evaluation 2018, Health Facility Endline Survey - Tajikistan

Abstract
---------------------------

The 2018 endline survey of the impact evaluation (IE) for Health Performance-Based Financing (PBF) in Tajikistan sought to ascertain: (i) the impact and cost-effectiveness of the PBF model implemented in Tajikistan; and (ii) whether PBF is more effective or cost-effective if implemented in conjunction with additional low-cost interventions (Collaborative Quality Improvement, Citizen Report Cards). The results from the IE will help inform the Ministry of Health on whether PBF should be scaled-up to additional PHC level institutions in other regions.

The Collaborative Quality Improvement intervention responds to policy concerns that performance incentives may not produce the desired improvements if providers lack the necessary competencies to inform decisions and knowledge. The Citizen Report Card attempts to improve the effectiveness of PBF by strengthening the 'short route' of accountability (e.g., by increasing accountability of health facilities to their local constituents). Since PBF, collaborative quality improvement (CQI), and citizen report cards (CRC) have never been implemented on a large scale in Tajikistan, it is to be expected that the results from the IE will be useful for designing national PHC policy in Tajikistan, and that they will also contribute to the larger body of knowledge on these interventions.

The IE employs both difference-in-difference and experimental approaches to identify the impact of the different combinations of interventions. Assignment to PBF was not random. Three districts in the Sughd region and four districts in the Khatlon region were selected to implement the program. All Rural Health Centers (RHCs) in these seven districts are covered by the program. Nine additional districts (two in Sughd and seven in Khatlon) were selected as control districts. The selection of the control districts was guided by geographical proximity to treatment districts and similarity in terms of number of health facilities and doctors per capita. The districts were also selected such that the number of RHCs in treatment and control groups in each region would be similar.

Within the chosen 16 districts (treatment and control districts), clusters consisting of an RHC and its subsidiary Health Houses were randomly assigned to implement Collaborative Quality Improvement, Citizen Score Cards, or neither of these two interventions. The randomization was blocked by district. In sum, RHCs were assigned into six study arms.

The goal of the facility-based survey is to measure multiple dimensions of quality of care and collect detailed information on key aspects of facility functioning. Household surveys are primarily used to measure health service coverage at the population level as well as select health outcome indicators measured through anthropometry or tests. The surveys also collect broader data on the health of the households, health seeking behaviors and barriers to use of health services. In addition, PBF and other administrative data would be used to track outcomes over time in the treatment groups 1-3 (the ones receiving performance-based payments). The endline (follow-up) survey took place three years after project implementation. The survey is largely based on the HRITF instruments that were modified to the Tajik and project context.

Geographic coverage
---------------------------

Three districts in the Sughd region and four districts in the Khatlon region were selected to implement the program. All Rural Health Centers in these seven districts are covered by the program. Nine additional districts (two in Sughd and seven in Khatlon) were selected as control districts. The selection of the control districts was guided by geographical proximity to treatment districts and similarity in terms of number of health facilities and doctors per capita. The districts were also selected such that the number of RHCs in treatment and control groups in each region would be similar.

Analysis unit
---------------------------

Health centers,
Health workers,
Patients (adults & children)
Patient household

Kind of data
---------------------------

Sample survey data [ssd]

Sampling procedure
---------------------------

The major features of the sampling procedure include the following steps (they are discussed in more detail in a copy of the study's report located in "External Resources"):

Health Facilities:
1. Table 6-4 in the study's report presents the number of RHCs selected for the sample for each district. Of the 216 RHC selected for the sample (after randomly excluding some RHCs when the total was not divisible by three), 151 have subsidiary HHs. Forty-three HHs were selected of the sample in Sughd and 107 in Khatlon.
2. While some Rural Health Centers have one or more subsidiary Health Houses in their catchment areas, others do not have any. One Health House from each RHC with subsidiary HHs was to be included in the sample. The selection was random with each health house within a cluster having an identical probability of being chosen. Non-selected health houses were ranked to serve as replacements if the survey cannot be implemented in the selected HHs.

Households:
1. The evaluation relies on two samples of households. As the primary focus of the PBF intervention is on Maternal and Child Health (MCH) services, the main household sample is of households with women who experienced a recent pregnancy. This sample would not be appropriate to study the impact on the coverage of services related to Non-Communicable Diseases (NCD). Therefore, a second sample consists of households with individuals over the age of 40. The household samples are clustered according to the catchment area of each Rural Health Center (and its affiliated health houses).
2. The resulting targeted primary household sample size is of 4,320 households, with 20 in each of the 216 clusters in the six study arms. To be eligible to be included in the household survey sample, households must have had at least one woman aged 15-49 years who has had a child in the preceding three years. The same villages were covered for both the baseline and followed up survey and eligibility was determined at each round by a listing exercise.
3. The resulting targeted sample size for the secondary household sample is 1,584 households, with 22 in each of 72 clusters in two of the six study arms. Eligibility for this sample is determined by an individual over the age of 40 in the household. Eligibility for the two samples is determined by a common listing of households in selected villages. Households which satisfy both eligibility criteria can be randomly selected to count towards the sample size requirements for both.
4. A two-stage cluster sampling methodology was employed to identify random samples. First, villages were randomly selected out of a list of the villages served by each facility. The list was obtained from the MoH. RHCs have either single or multiple villages in their catchment areas while HHs typically serve a single village. If an RHC has at least one affiliated HH, then two villages were selected. One village was directly served by the RHC while the other included in the sub-catchment area of the HH. In each village, 100 households were listed. If the village had over 100 households, a random walk method was used to select the target number. A short questionnaire was conducted at each household to determine households' eligibility for the two samples. From all eligible households, the target sample for each catchment area was selected. In catchment areas in which two villages were included in the sample, half of the households were to be selected from each village.

Mode of data collection
---------------------------

Computer Assisted Personal Interview [capi]

Research instrument
---------------------------

The Tajikistan Health Results Based Financing Impact Evaluation 2018 - Health Facility Endline Survey includes the following 7 questionnaires.

Facility-Based Surveys:
1. Health facility assessment module
2. Health worker interview module
3. Observation of patient-provider interaction module
4. Patient exit interview modules

Household Survey:
5. Main household questionnaire
6. Women of reproductive age interview questionnaire
7. Adults over 40 years old questionnaire

Response rate
---------------------------

Health Facilities:
Of 216 RHCs selected for the impact evaluation, 210 were evaluated at both baseline and follow-up. Six RHCs evaluated at baseline were ineligible for selection at follow-up due to closure or re-registration (either upgraded to a district health center or downgraded to health house). These six RHCs and their respective health house and household enumeration areas were replaced before the start of the follow-up survey. A total of 151 health houses were assessed at baseline, and 150 at follow-up. Eleven health houses were close or re-registered as RHCs. Our analyses treat RHCs and health houses as panel data, where it is assumed the observed facility is measured at both time points. Therefore, both the original units which have been replaced and the replacement are excluded in the subsequent difference-in-difference and cross-sectional analyses.

Health Workers:
A total of 1,574 health workers were surveyed in the RHCs included in the analysis sample, 767 at baseline and 807 at follow-up. The average number of health workers fell slightly below the 4 per RHC target, as more remote RHCs did not have four staff members available. In health houses, the two staff per HH was achieved in the baseline sample but narrowly missed in the follow-up survey. Health workers who worked in both the rural health center and health house were treated as RHC employees.

Households:
A total of 10,599 households were surveyed across 230 villages in 210 RHC catchment areas, 4910 at baseline and 5689 during follow-up covering 83,803 household members. Within the two targeted populations, 7048 women 15-49 years of age with a pregnancy in the past three years, and 17,583 adults 40 years or older were surveyed.

摘要
---------------------------
塔吉克斯坦2018年健康绩效基础融资（PBF）影响评估的终期调查旨在明确：（i）塔吉克斯坦实施PBF模型的影响和成本效益；（ii）如果与额外的低成本干预措施（协同质量改进、公民报告卡）结合实施，PBF是否更有效或更具成本效益。影响评估的结果将有助于卫生部门决定是否将PBF扩展至其他地区的初级卫生保健机构。
协同质量改进干预措施旨在回应政策上的担忧，即如果提供者缺乏做出决策和知识所需的必要能力，绩效激励可能无法产生预期的改进。公民报告卡旨在通过加强问责制的‘短途’（例如，通过增加医疗机构对其当地居民的问责制）来提高PBF的有效性。由于PBF、协同质量改进（CQI）和公民报告卡（CRC）在塔吉克斯坦从未大规模实施，因此可以预期，影响评估的结果将对设计塔吉克斯坦国家初级卫生保健政策具有参考价值，并且它们还将为这些干预措施的更广泛知识体系做出贡献。
影响评估采用了双重差分法和实验方法来识别不同干预措施组合的影响。PBF的分配并非随机。在苏格德地区的三个区和卡特隆地区的四个区被选为实施项目。这七个区的所有农村卫生中心（RHC）都纳入了项目。另外九个区（苏格德两个和卡特隆七个）被选为控制区。控制区的选择以治疗区的地理邻近性和人均医疗机构和医生数量的相似性为指导。此外，还选择了这些地区，以便每个地区治疗组和控制组中的RHC数量相似。
在所选的16个区（治疗区和控制区）内，由一个RHC及其下属卫生屋组成的集群被随机分配以实施协同质量改进、公民评分卡或这两项干预措施中的任何一项。随机化以区为单元进行。总的来说，RHC被分配到六个研究臂。
基于设施的调查旨在衡量护理质量的多个维度，并收集有关设施运作关键方面的详细信息。家庭调查主要用于衡量人口层面的卫生服务覆盖率以及通过人体测量学或测试测量的健康结果指标。调查还收集了关于家庭健康、健康寻求行为和利用卫生服务障碍的更广泛数据。此外，PBF和其他行政数据将用于追踪治疗组1-3（接受基于绩效支付的组）的结果。终期（后续）调查在项目实施三年后进行。该调查主要基于HRITF工具，这些工具已被修改以适应塔吉克斯坦和项目环境。
地理覆盖范围
---------------------------
苏格德地区的三个区和卡特隆地区的四个区被选为实施项目。这七个区的所有农村卫生中心都纳入了项目。另外九个区（苏格德两个和卡特隆七个）被选为控制区。控制区的选择以治疗区的地理邻近性和人均医疗机构和医生数量的相似性为指导。此外，还选择了这些地区，以便每个地区治疗组和控制组中的RHC数量相似。
分析单元
---------------------------
卫生中心、
卫生工作者、
患者（成人及儿童）、
患者家庭
数据类型
---------------------------
样本调查数据 [ssd]
抽样程序
---------------------------
抽样程序的主要特点包括以下步骤（它们在研究报告的副本中讨论得更详细，该副本位于“外部资源”部分）：
卫生设施：
1. 研究报告的第6-4表展示了每个区选定的样本RHC数量。在216个选定的样本RHC（在总数不能被三整除时随机排除了一些RHC）中，有151个有下属卫生屋。苏格德地区选择了43个卫生屋，卡特隆地区选择了107个。
2. 虽然一些RHC在其服务范围内有一个或多个下属卫生屋，但其他RHC则没有。每个有下属卫生屋的RHC将包括一个卫生屋在内的样本。选择是随机的，每个集群内的卫生屋被赋予相同的选择概率。未选中的卫生屋被排名，以便在无法在选定的卫生屋中进行调查时作为替代。
家庭：
1. 评估依赖于两个家庭样本。由于PBF干预措施的主要重点是母婴健康（MCH）服务，因此主要家庭样本是最近经历过怀孕的妇女的家庭。这个样本不适合研究对非传染性疾病（NCD）相关服务覆盖率的影响。因此，第二个样本由40岁以上的个人家庭组成。家庭样本根据每个农村卫生中心（及其附属卫生屋）的服务范围进行集群。
2. 结果，针对目标初级家庭样本的大小为4,320户，每个六个研究臂的216个集群中有20户。要符合家庭调查样本的资格，家庭必须至少有一位在过去的三年中生过孩子的15-49岁的妇女。基线和后续调查覆盖了相同的村庄，资格在每一轮通过清单作业确定。
3. 结果，针对二级家庭样本的目标样本大小为1,584户，每个六个研究臂中的两个72个集群中有22户。这个样本的资格由家庭中的40岁以上的个人确定。两个样本的资格由选定村庄的家庭的共同清单确定。满足两个资格标准的家庭可以随机选择，以计入两个样本的样本量要求。
4. 采用两阶段集群抽样方法来确定随机样本。首先，从每个设施服务的村庄列表中随机选择村庄。该列表由卫生部获得。RHC在其服务范围内可能有一个或多个村庄，而HH通常服务一个村庄。如果RHC至少有一个附属HH，则选择两个村庄。一个村庄直接由RHC服务，另一个位于HH的子服务范围内。在每个村庄中，列出100户家庭。如果村庄有超过100户家庭，则使用随机漫步法选择目标数量。在每个家庭进行简短的调查问卷，以确定家庭的两个样本的资格。
数据收集方式
---------------------------
计算机辅助个人访谈 [capi]
研究工具
---------------------------
塔吉克斯坦健康结果基于融资影响评估2018年 - 卫生设施终期调查包括以下7份问卷。
基于设施的调查：
1. 卫生设施评估模块
2. 卫生工作者访谈模块
3. 患者提供者互动观察模块
4. 患者出院访谈模块
家庭调查：
5. 主要家庭问卷
6. 生育年龄妇女访谈问卷
7. 40岁以上成年人问卷
应答率
---------------------------
卫生设施：
在分析样本中包含的216个RHC中，有210个在基线和后续阶段进行了评估。由于关闭或重新注册（升级为区卫生中心或降级为卫生屋），基线评估的六个RHC在后续评估中不符合选择条件。在后续调查开始之前，这六个RHC及其相应的卫生屋和家庭统计区域被替换。基线评估了151个卫生屋，后续评估了150个。有11个卫生屋被关闭或重新注册为RHC。
我们的分析将RHC和卫生屋视为面板数据，其中假设观察到的设施在两个时间点都被测量。因此，被替换的原始单位和替换单位都在后续的双重差分和横断面分析中被排除。
卫生工作者：
在分析样本中包含的RHC中，共有1,574名卫生工作者接受了调查，基线阶段767人，后续阶段807人。卫生工作者的平均数量略低于每个RHC的4人目标，因为更偏远的RHC没有四个工作人员可用。在卫生屋中，基线样本中每个HH实现了两个工作人员的目标，但在后续调查中仅勉强达到。
卫生工作者在RHC和卫生屋中工作被视为RHC的雇员。
家庭：
在210个RHC的210个村庄中，共调查了10,599户家庭，基线阶段4910户，后续阶段5689户，覆盖了83,803户家庭成员。在两个目标人群中，有7048名15-49岁的妇女在过去三年中怀孕，有17,583名40岁或以上的成年人接受了调查。