Odisha Rural Livelihoods Evaluation - Baseline Household and Village Data, 2011

Impact Evaluation Framework and Results: Odisha Rural Livelihoods Project

This is a baseline panel survey of households from rural Odisha collected to evaluate the impact of the TRIPTI Livelihoods project using a quasi-RDD design. The data were also used, after merging it with information on rainfall patterns, to assess the impact of TRIPTI on mitigating the effects of Hurricane Phailin. Collaboration: World Bank Social Observatory in collaboration with the Government of Odisha.

EVALUATION DESIGN
This evaluation was designed in 2011 after the project areas were chosen, which meant that a “gold standard” impact evaluation with a Randomized Control Trial method was not feasible. The selection rule for project areas however allowed for the next best available tool of Regression Discontinuity Design (See below). Using this design, the difference in difference approach, which measures the change in outcomes between project or “treatment” and comparable non-project or “control” areas over the evaluation time period can be used to evaluate the impact the project.
Selection of TRIPTI Blocks
- In each TRIPTI district, 4 blocks were to be chosen for project ""treatment"" using a ""backwardness"" selecton rule
- All blocks were given a score that gave weightage to block level development indices (Ghadei Committee Index), SHG coverage, total population and SC/ST Populations
- Program blocks then ranked in descending order of scores, and the 4 blocks with highest backwardness score wee chosen for the program
Selection of Evaluation Blocks
- In each district, the non- program or ""control"" block was chosen to the block that had the closest score to the last of the 4 program blocks
- A pair of blocks- one program or “treatment” block, and non- program or “control” blocks) were chosen to be part of the evaluation sample in every district
Selection of GPs, Villages and Households
- Treatment is universal at the level of the block, which implies that at sub-block units, or Gram Panchayats (GPs) receive TRIPTIs interventions.
> 4 GPs randomly chosen in each block
> 2 vilages randomly chosen in each GP
- All targeted households in a TRIPTI GP are eligible TRIPTI interventions
> 15 households randonly chosen in each village
> Oversampling of SC/ST housholds to proxy for target housholds

EVALUATION DATA
The data used in this evaluation come from the first (baseline) of two surveys commissioned by TRIPTI with technical assistance from the World Bank. An independent survey firm implemented both surveys. The baseline survey was completed before the initiation of TRIPTI in the evaluation sample area, between September-November 2011; and the follow up survey was implemented over the same month in 2014. This data therefore covers a 3-year period during which TRIPTI was in operation.
The data collected focused on four modules. A general household module collected data on household consumption expenditures (following the same format as India’s National Sample Surveys that are used to measure poverty); and detailed information on the livelihoods portfolio and debt profile of households. A woman’s module was also administered to an adult married woman in each household. This module measured different metrics of women’s empowerment; and included questions on decision-making within the household, and on women’s participation in local public action. Two focus group discussions with the village in general, and women in the villages separately were also implemented in order to understand key elements related to local politics and civic action. In addition, a GPLF survey module- that covered 58 project Gram Panchayats - was implemented during the follow up survey.
As part of this evaluation, data was to be collected from a sample of 3000 households selected at random from these 160 villages twice: once before the launch of project interventions in these 80 GPs at baseline (2011), and once at the end of the project. Due to some missing data, the baseline survey included in the end a total of 2875 households and the end line survey included a total of 2,874 households. The working sample is the total set of these households with reliable data. In each round of the survey, each household is linked to village-level data from that round.

This evaluation report is an output of the Social Observatory Team of the World Bank and the Orissa Rural Livelihoods Project (TRIPTI), and it was financed by the South Asia Food and Nutrition Security Initiative (SAFANSI). There are two parts to this repot- an executive summary, and a technical paper that is authored by Shareen Joshi (Georgetown University), Nethra Palaniswamy (World Bank), and Vijayendra Rao (World Bank). Discussions with the TRIPTI project team led by the Additional Project Director Babita Mohapatra, and the World Bank task team led by Samik Das were critical to the design of this evaluation. Support from Arvind Padhee and DV Swamy who served as Project Directors of TRIPTI; from Sitaramachandra Machiraju from the World Bank; and from Radu Ban during the initial phase of this evaluation is gratefully acknowledged. We also thank Shantanu Bhoi and Suchismita Padhi from TRIPTI for support during survey implementation, and SUTRA consultancy for implementing the survey. Urmila Chatterjee, Neeraja Gupta, Vasudha Jain, Madhulika Khanna, Nishtha Kochhar, and Smriti Sakhamuri from the SO supported this survey at various times. The findings, interpretations and conclusions of this article should not be attributed to the World Bank Group, it’s executive directors, or its member countries.