Jeevika Livelihoods Project Phase 2 Evaluation (RCT), Bihar, India - Baseline and Endline Household And Village Data 2011-2014

Poverty and empowerment impacts of the Bihar Rural Livelihoods Project: Evidence from a Mixed-Methods Cluster-Randomized Trial Jeevika is a World Bank assisted project focussed (now under the umbrella of the NRLM) on building networks of women's self-help credit and savings groups,and then using them as a base of other \"vertical\" interventions. This houshold and village survey data was collected over two rounds to conduct an impact evaluation of Phase 2 of the project with random assignment of the project over a two year period. Collaboration: World Bank Social Observatory team with Government of Bihar. Evaluation design, methods and implementation In order to evaluate the impacts of Jeevika, 180 panchayats were randomly selected from within 16 blocks in seven districts where scale-up of the project was planned but had not yet occurred. Some of these blocks were in districts relatively far from Patna, which had not yet been entered by the project (Madhepura, Saharsa, Supaul), while others were within the larger districts within which Jeevika was already operating (Gaya, Nalanda, Madhubani, Muzaffarpur). The project had already entered these districts in Phase 1, but had not yet expanded to all blocks due to (project) capacity constraints. Within each of the study villages, hamlets (tolas) in which the majority of the population belonged to a scheduled caste or scheduled tribe were identified. This was the same procedure as used by Jeevika to identify the target population (of poor women) for mobilization into the project. Tolas were identified through a focus group discussion held in each village, along with the population of target castes (SC/STs) within each. In Bihar, tola boundaries are easily distinguishable. Field teams would enter the tola at a random point, determine the skip pattern based on the population size and target sample size, and select households through a random walk. Survey staff aimed to include 70% SC/ST households, and 30% households from other castes in each village, in order to ensure variation in socio-economic status within the sample. If the households in selected tolas included fewer SC/ST households than this, households from nearby non-SC/ST majority tolas were also included in the sample. Interviews for the quantitative study were conducted using a structured paper survey form. Baseline and follow up surveys included detailed questions on debt, asset holdings, consumption expenditures, livelihood activities, and women’s mobility, role in household decisions, and aspirations. In addition, in each village, a focus group discussion was conducted, through which data were collected on village level attributes such as local sources of credit, interest rates from each source, local wage rates, and the presence of or distance to markets and other institutions and amenities. Respondents were not compensated for their time. If a respondent was unavailable during initial field visit, the supervisor recorded contact details and returned with interviewers at a later date. As long as the survey team was in that district, repeat visits were undertaken, keeping attrition to a minimum. If a household could not be re-surveyed at endline, it was replaced with another household in the same village. Short re-surveys containing a subset of questions from the main survey were conducted by supervisors for 10% of the sample. Staff from the project also conducted occasional visits after the survey was completed in a village to confirm that all modules had been covered by survey staff. Data was entered in duplicate using CSPro and any discrepancies were corrected based on the paper form. Following the baseline survey, panchayats were stratified on the 16 administrative blocks in the sample and the panchayat-level mean of outstanding high cost (monthly interest rate of 4% or higher) debt held by households at baseline. They were then randomly assigned to an early rollout group or a late rollout group using the random number generator within the Stata statistical analysis software package. The baseline survey was administered to 8988 households across 333 villages in 179 panchayats. The target number of households per panchayat was 50, but there was some variation around this in reality. The lowest number of households in a given panchayat was 49 (9 panchayats), and the largest number was 53 households (3 panchayats). To ensure that control panchayats were not entered by the project, Jeevika held a quarterly \"\"evaluation panchayat\"\" meeting, which block project managers of the 16 blocks were required to attend. At these meetings the project M&E team checked whether any village in a control panchayat had been entered, and received an update on progress in treatment panchayats. This procedure was successful in maintaining adherence to randomized treatment assignment throughout the evaluation period. Of the 4,472 households in the sample across 89 panchayats allocated to receive the SHG intervention, 2,722 reported that one of their members belonged to an SHG by endline, constituting 61% of the sample. Since SHG membership was optional, approximately 38% of households in treatment group panchayats had no member in an SHG by endline. The remaining 56 households (across 39 panchayats) did not answer this question or were lost to follow-up (only one such household was not replaced). Although it was possible for those residing in control areas to join (non-Jeevika) SHGs, the proportion of households group in this area containing SHG members remained minimal at endline, with only 460 households (just over 10% of the total sample) reporting SHG membership. Attrition (and replacement) were similar in control and treatment arms, with 132 treatment group baseline households not reached for a follow-up interview and all but one of these replaced, and 128 not reached and thus replaced in the control group. The qualitative evaluation draws on data collected from 2011 to early 2015 in six villages, two where Jeevika had been operating since 2006, two it entered during Phase II, and two where it had not yet intervened by the end of data collection. The Phase I treatment villages were selected at random from the set of previously entered villages in two different districts – Muzaffarpur and Madhubani. Each treatment village was then matched with a set of control villages using propensity score matching methods (Imbens and Rubin 2015) on the basis of village level data from the 2001 government census on literacy, caste composition, landlessness, levels of outmigration, and the availability of infrastructure. In order to find the closest treatment-control match, field investigators then visited the set of possible controls for two days for visual inspection and qualitative assessment. This combined quantitative and qualitative matching method yielded three matched pairs of phase I treatment, phase II treatment, and control villages, with each pair located within the same district. This method of sample selection allows comparison of villages receiving the intervention at each stage with their statistical clones that received it at a different stage or had not received it at all, allowing us to draw causal inferences about the effects induced by Jeevika during the different phases of its expansion. For the purpose of keeping their identity anonymous, we refer to the villages in Madhubani district as Ramganj (Phase I treatment), Nauganj (Phase II treatment) and Virganj (control) and the villages in Muzaffarpur district Saifpur (Phase I treatment), Raipur (Phase II treatment) and Bhimpur (Control). Villages in Madhubani are divided into segregated and caste-homogenous tolas. Brahmins are a majority in these villages, and their tolas are located close to the main resources of the village: the temple, pond and school. All other tolas extend southwards in decreasing order of status in the caste hierarchy, with the Schedule Caste (SC) communities being located farthest south. Each of these communities is also spatially segregated. The SC communities of these villages are mainly comprised of Musahar, Pasi, Ram, and Dhobi subcastes, and the other backward caste communities are comprised of Yadav, Mandal, Badhai, Hajaam, and Teli subcastes. The only big difference between Ramganj and Virganj is that the former has a sizeable Muslim population, comprising Sheikhs, Ansaris, Nutts and Pamariyas, while in the latter, there is only one Muslim (Sheikh) family in the entire village. Inhabitants of these villages primarily depend on agriculture and related activities for their livelihood. The villages in Muzaffarpur district are largely similar to the ones in Madhubani with the important differences being that they are primarily bazaar (market)-centric and the dominant caste is the Chaudhury, who belong to the business community. In each of these villages, first, preliminary studies were conducted using several participatory rural appraisal methods to gain an understanding of the layout of the village. Following this, a team of four field investigators (recruited from a local research-based NGO) accompanied by one of the three principal researchers would visit the villages every three to four months for a cycle of data collection (11 in total over the study period). During every cycle, the ethnographers would enter a different tola in the village for a week (there are roughly 10 tolas in each village). The ethnographers spoke to as many respondents as possible across the village and also returned to the first few respondents in the concluding cycles of data collection. These repeat interviews allowed us to see how respondents reflected on changes experienced as a result of the project [or otherwise] over the four-year period. The first set of participants was selected to be representative of different socioeconomic strata in the village, and subsequent participants were selected via a mixture of purposive and snowball sampling. We interviewed women who were members of JEEViKA, their husbands, and key informants in the village such as religious heads, village council members, moneylenders, subsidized food shop dealers, landlords, and public officials. Qualitative data were collected in the villages through a variety of methods: a) personal interviews (open-ended structured and unstructured) and conversations with program participants and non-participants; b) focus group discussions with participants and non-participants; c) passive observation of group meetings, trainings, workshops, mobilization drives and interactions at several levels (village, block, district); d) structured interviews with Jeevika staff at all levels in all villages; and finally e) interviews and focus groups with men and other key stakeholders in the village (religious heads, village council members, moneylenders, subsidized food shop dealers, landlords, and other public officials). The interviews, observations and focus group discussions were guided by a set of themes that were modified throughout the data collection. The interviews were conducted in the local language (Hindi and Maithili) by researchers, transcribed in English, and coded in QSR NVivo (a qualitative data analysis software). During the coding, some themes were preselected to match the themes of the questions asked, but we also allowed themes to emerge from the data in an inductive mode. These multiple cycles of data, coupled with the matched experimental design, allow us to understand cause-effect relations and the mechanisms of change over time. We are able to study social processes as they unfold in the villages with the evolution of Jeevika, rather than being solely reliant on informant recall. In addition, having a comparison across districts allows us to capture variation in processes that occurred in similar rural landscapes. In addition, the qualitative nature of the study permits us to incorporate the participants’ own evaluative metrics and to understand why the women prioritize certain transformations over others. The interviews were combined with direct observation of project activities and focused on understanding how the project and its frontline workers were responsible for the changes experienced. Interviews took one to two hours. They were conducted in the local language (Hindi or Maithili), simultaneously recorded, and then transcribed verbatim into English. In total, over 2000 interviews were conducted (250 with men and the rest with women). Transcripts were coded in NVivo, after which the data were analyzed inductively. In the first step, we tried to understand what kind of changes men and women talked about within treatment villages and to what extent they were attributed to JEEViKA exclusively. After this, data were grouped by emerging themes. Six themes emerged as salient: physical mobility, husband’s reaction, dignity of borrowing, information on village-credit network, perception of government and collectivization. This study documents the impact of a government-sponsored livelihoods project using a mixed methods approach within a cluster-randomized trial. Key features of the project were the formation of women’s self-help groups, and the provision of low-cost credit through these groups. The intervention led to a dramatic increase in self-help group membership and take-up of credit through these groups, and a corresponding decline in the use of informal credit. A reduction in average informal lending interest rates was also observed. Two years after initiation of the program, significant positive impacts on asset ownership among landless households were apparent. Impacts on various indicators of women’s empowerment were mixed, and showed no clear direction when aggregated, nor was there any impact on consumption value. Given the reduction in debt service costs achieved both directly through substitution into lower-cost sources of credit and the reduction in informal interest rates, impacts of the intervention on household welfare are expected to continue to accrue over time.