Impact Evaluation of PNPM Generasi Program 2009-2010, Third Wave - Indonesia

Abstract
---------------------------

PNPM Generasi known as the National Community Empowerment Program - Healthy and Smart Generation (Program National Pemberdayaan Masyarakat-Generasi Sehat dan Cerdas) began in mid-2007 in rural areas of five Indonesian provinces selected by the government: West Java, East Java, North Sulawesi, Gorontalo, and Nusa Tenggara Timur. The Generasi project is implemented by Indonesia’s Ministry of Home Affairs, and is funded through government of Indonesia resources, loans from the World Bank and grants from several bilateral donors.

In Generasi, all participating villages receive a block grant each year to improve education, and maternal and child health in their villages. The village block grants ranged from an average of $8,500 in 2007 up to an average of $18,200 in 2009.

In order to evaluate the overall impact of Generasi, as well as to separately identify the impact of Generasi's performance incentives, program locations were selected by lottery to form a randomized, controlled field experiment. Each location was randomly allocated to one of two versions of the program: an "incentivized" treatment with the pay-for-performance component (treatment A) and an otherwise identical "non-incentivized" treatment without the pay-for-performance incentives (treatment B).

The impact evaluation project was conducted from 2007 until 2010, in three waves. Documented here is the third round, carried out from October 2009 to January 2010.

The sample for the research covered each of the 300 subdistricts that were included in the original Generasi randomization. In each subdistrict, eight villages were randomly selected (unless the subdistrict had fewer than eight villages, in which case all were selected). Overall, 2,313 villages were sampled in each of the three survey waves.

The data for the impact analysis was gathered from surveys of households, mothers, health service providers, and school and village officials.

Geographic coverage
---------------------------

West Java, East Java, North Sulawesi, Gorontalo, and Nusa Tenggara Timur provinces

Analysis unit
---------------------------

- Households,
- Married women, age 16-49,
- Children age 6-15, respondent: mother of the child,
- Children age < 3, respondent: mother of the child,
- Villages,
- Community health centers (Puskesmas),
- Village midwives,
- Village health posts,
- Junior secondary schools.

Kind of data
---------------------------

Sample survey data [ssd]

Sampling procedure
---------------------------

In order to evaluate the overall impact of Generasi, as well as to separately identify the impact of Generasi's performance incentives, Generasi locations were selected by lottery to form a randomized, controlled field experiment.

The Generasi randomization was conducted at the subdistrict (kecamatan) level, so that all villages within the subdistrict either received the same treatment of Generasi (treatment A or treatment B) or were in the control group. Randomizing at the subdistrict level is important since many health and education services, such as community health centers (puskesmas) and junior secondary schools, provide services to multiple villages within a subdistrict.

The Generasi locations were selected through the following procedure. First, 300 target subdistricts were identified, targeting poor, rural areas that had an existing community-driven development infrastructure.

Each subdistrict was then randomly assigned by computer into one of three equal-sized groups: treatment A, incentivized (100 subdistricts); treatment B, non-incentivized (100 subdistricts); or control (100 subdistricts). Within a subdistrict, all villages received the same treatment. The randomization was stratified by district (kabupaten), to ensure a balanced randomization across the 20 different districts in the study.

The sampling design for the household component of the Generasi surveys was chosen to ensure adequate coverage in the key Generasi demographic groups: mothers who recently were pregnant or gave birth, children under age three, and children of school age. Within each village, one hamlet (dusun) was randomly selected, and a list of all households was obtained from the head of the hamlet. Five households were randomly sampled from that list to be interviewed. These households were stratified so that two selected households had at least one child under age two, two selected households had a child under age 15 but no children under age two, and one household had no children under age 15.

Mode of data collection
---------------------------

Face-to-face [f2f]

摘要
---------------------------
PNPM Generasi，即国家社区赋能计划——健康智慧一代（Program National Pemberdayaan Masyarakat-Generasi Sehat dan Cerdas），自2007年中期起在印尼五个由政府选定的农村省份实施，这些省份分别为：西爪哇、东爪哇、北苏拉威西、戈朗岛和努沙登加拉省。Generasi项目由印尼内政部实施，资金来源于印尼政府资源、世界银行的贷款以及多个双边捐助者的拨款。

在Generasi项目中，所有参与村庄每年均获得一笔专项拨款，用于改善其村庄的教育、孕产妇及儿童健康。村庄专项拨款从2007年的平均8500美元增至2009年的平均18200美元。

为了评估Generasi的整体影响，并单独识别Generasi绩效激励的影响，项目地点通过抽签方式选定，以形成一项随机对照的实地实验。每个地点被随机分配到项目的两个版本之一：带有绩效付费成分的激励性治疗方案（方案A）和不带绩效付费激励的类似治疗方案（方案B）。

影响评估项目于2007年至2010年间分三个阶段进行。本文记录的是第三轮评估，于2009年10月至2010年1月进行。

研究样本涵盖了原始Generasi随机分配中的300个次级行政区。在每个次级行政区，随机选取八个村庄（除非该次级行政区村庄数量少于八个，此时选取所有村庄）。总体而言，每个调查波次中均选取了2313个村庄。

影响分析所需数据来源于对家庭、母亲、医疗服务提供者、学校及村庄官员的问卷调查。

地理覆盖范围
---------------------------
西爪哇、东爪哇、北苏拉威西、戈朗岛和努沙登加拉省
分析单位
---------------------------
- 家庭,
- 婚姻妇女，16-49岁,
- 6-15岁儿童，受访者：儿童的母亲,
- 3岁以下儿童，受访者：儿童的母亲,
- 村庄,
- 社区卫生中心（Puskesmas）,
- 村庄接生婆,
- 村庄卫生站,
- 初级中学。
数据类型
---------------------------
样本调查数据 [ssd]
抽样程序
---------------------------
为了评估Generasi的整体影响，并单独识别Generasi绩效激励的影响，Generasi地点通过抽签方式选定，以形成一项随机对照的实地实验。

Generasi的随机化是在次级行政区（kecamatan）层面进行的，确保次级行政区内的所有村庄要么接受相同的Generasi治疗方案（方案A或方案B），要么属于对照组。在次级行政区层面进行随机化非常重要，因为许多健康和教育服务，如社区卫生中心（puskesmas）和初级中学，为次级行政区内的多个村庄提供服务。

Generasi地点的选定通过以下程序进行。首先，确定了300个目标次级行政区，目标是贫困农村地区，这些地区已拥有现有的社区驱动型发展基础设施。

然后，每个次级行政区通过计算机随机分配到三个大小相等的组之一：方案A，激励性（100个次级行政区）；方案B，非激励性（100个次级行政区）；或对照组（100个次级行政区）。在一个次级行政区内，所有村庄接受相同的治疗方案。随机化按区（kabupaten）分层进行，以确保在研究中的20个不同区之间实现平衡的随机化。

Generasi调查的家庭部分抽样设计旨在确保在Generasi关键人口群体中覆盖充分：近期怀孕或分娩的母亲、3岁以下儿童以及学龄儿童。在每个村庄内，随机选择一个村寨（dusun），并从村寨首领处获得所有家庭的名单。从该名单中随机抽取五个家庭进行访谈。这些家庭按以下方式进行分层：两个选定的家庭中至少有一个2岁以下的儿童，两个选定的家庭中有一个15岁以下但无2岁以下儿童，一个家庭没有15岁以下的儿童。

数据收集方式
---------------------------
面对面 [f2f]