Using a participatory impact assessment framework to evaluate a community-led mangrove and fisheries conservation approach in West Kalimantan, Indonesia

Community-based conservation (CBC) has been identified as a solution to biodiversity loss, climate change, and the reduction of rural poverty. The heterogeneity in social and economic inequalities often acts as a barrier to community engagement in resource management and further inhibits the distributional equity of social and ecological outcomes.
This study presents a participatory impact assessment (PIA) framework that evaluated the outcomes of a cross-sector community-led conservation initiative. Community members involved in the program identified activities and outcomes for the Conservation Cooperative (CC), ranking the influence of the former on the latter as well as their daily life through multiple focus group discussions (FGDs). Participants were asked to rank the impact of activities on outcomes and the scale of the outcome which was totaled to identify the most impactful program activities and outcomes during the project period.
Community members reported improved income, health, education and the creation of a locally-led natural resource management system. Members also reported improved crab harvest rates and reduced mangrove deforestation. Environmental outcomes identified by community members through the PIA were verified through a secondary spatial analysis and mud-crab independent fisheries monitoring.
The results support the hypothesis that environmental NGOs need to consider a multi-dimensional view of human well-being, and that cross-sector integrated interventions may be effective at improving multiple outcomes.
Future steps should focus on spatial replication of the CC program which will provide further insights by testing for differences in outcomes between villages, how those are impacted by preexisting social and ecological systems, and comparing outcomes between control sites that did not receive interventions.

Methods
The PIA was adopted from the PRISM Conservation Evaluation toolkit (Dickson et al. 2017). The framework was adapted to include a nested ranking system to evaluate indirect changes identified by community members, changes that can be directly attributed to the project, and the impact these changes have made in people’s lives. In addition, the PIA was designed to follow the nine principles presented in Woodhouse et al. (2015), to evaluate the impact of conservation interventions on human well-being. The PIA method focused on project beneficiaries and hinges on basic knowledge of the project activities by the members of the FGD. Within the first step of the PIA FGD, members created an activity list based upon their knowledge of program progress to date. They were then asked to score each of the activities for each of the following criteria.

The importance of an activity as a motivator to engage in the conservation cooperative.
The importance of the activity for your daily life.
The time expended by the community in implementing this activity.

Scores were on a scale of one to four, where one is low importance / labour and four is high importance / labour.

The second step of the PIA was to create an Influence Matrix based upon the results and activities in step one.  Activities from step one were discussed and facilitators asked community members if there were similar activities that could be combined. If members proposed combinations, similar activities were consolidated into large umbrella activities. FGD participants identified changes that have occurred in the community. These were then consolidated by participants into six areas of most significant change since the start of the program in August 2017. Participants scored the level of influence of each activity on the observed change, with 0 = No Influence, 1 = Weak Influence, 2 = Moderate Influence and 3 = Strong Influence. Results were discussed with all FGD participants to understand the reasoning behind rankings and the FGD decided upon the final scores.

The PIA was conducted twice, on two different FGD’s. First with a mixture of men and women involved in the program, and second with village leaders, cooperative leaders, and important village figures. The first FGD was conducted with 7 men and 5 women in May 2019. Program participants were informed three days prior to the FGD and on the day of the FGD we capped the group at the first 12 individuals who arrived. We asked the important figures to not join the first FGD to prevent leaders from dominating the discussion and over-rule community members in nested scoring. The second FGD was conducted in June 2019 and consisted of 5 men and 2 women, all important figures within the village. FGDs were not repeated with members of villagers who did not enroll in the Conservation Cooperative, the PIA asked program participants to reflect upon activities and outcomes not applicable to non-program members. FGDs discussions lasted 3-4 hours with a lunch break included.

All respondents were told upon the start of the FGD the objective of the PIA and that the results would be shared with FGD members at the end of the session. We asked members to be truthful in their responses and informed individuals that their feedback would not harm the program or individuals themselves, but would be used to improve the program in future months.  The methodology, process, and intended use of the PIA was clearly explained to members before the FGD and all individuals were given a choice to voluntarily join if they were interested. Verbal consent was required by each individual before the start of the FGD. Throughout the FGD participants were allowed to leave at any point and the facilitation team fully respected all rights and well-being of participants. All names of members were kept confidential and individuals were given a chance to opt-out if they felt uncomfortable.

2.4 Fisheries Independent Data

To add complementary data to support PIA findings the success of the TMR to improve mud crab harvest, fisheries independent data was collected. Sampling occurred seven days before the closure began in November 2018 and seven days before the TMR rivers re-opened to fisheries in February 2019. A minimum of 20 crab pots were randomly deployed in each river. All three TMR rivers were sampled and effort was distributed across them to account for natural variation. The four rivers that were sampled as controls were immediately adjacent to the TMR rivers, two to the north and two to the south, to eliminate geographical variation as a factor. Mean carapace width of crabs in centimetres, number of crabs and number of pots were recorded. Catch per unit effort (CPUE) was calculated by dividing the number of crabs by the number of pots for each zone in each sampling period.

Mean carapace width and CPUE were compared between two zones (open and TMR rivers) in two sampling periods (November 2018, February 2019), for four total sampling units. Zones were compared against each other within each sampling period, and each zone was compared between sampling periods. All statistical tests were conducted in R studio (version  1.2.1335). The mean carapace width comparisons were conducted using two sample independent t-tests, and when the assumption of normality was not satisfied, transformations were explored. If these did not satisfy this assumption, a Wilcoxon rank sum test with continuity correction was conducted. CPUE comparisons were conducted using a generalised linear model with a quasi poisson distribution, where x values were discrete and represented the sampling unit.

2.5 Mangrove Forest Cover and Disturbance

To test that indicators of tree cover loss observed from the PIA had been achieved, a simple spatial analysis was conducted to detect the occurrence of mangrove clearing during the project period. We analyzed tree cover data and mangrove forest coverage using data from the Global Forest Watch platform from the World Resource Institute (Hansen et al. 2016, 2017, 2018). To test for disturbance, tree cover loss within the 3054-hectare project area was analysed using Global Land Analysis and Discovery (GLAD) alerts and Terra Alerts (Hansen et. al. 2016, 2017, 2018). There was a limitation to the Global Forest Watch platform at calculating fine-scale forest disturbance which therefore restricted our analysis to mangrove forests within the area using Global Mangrove Watch (v2.0, Bunting et. al. 2018). We analyzed tree cover loss from intact forests (defined by having greater than 75% canopy density) from January 1, 2001 to December 31, 2018, and GLAD alerts from January 1, 2015 to April 30, 2019. According to Global Forest Watch, tree cover is defined as all vegetation greater than five meters in height and may take the form of natural forests or plantations across a range of canopy densities. Tree cover loss is defined as “stand replacement disturbance,” or the complete removal of tree cover canopy at the Landsat pixel scale (30 meters by 30 meters). A GLAD alert is an observation of tree cover loss on a per pixel basis by a supervised learning algorithm, therefore one GLAD alert is equal to a tree cover loss of 30m by 30m (ref). The clear land surface observations in the satellite images were assembled and a supervised learning algorithm was applied to identify per pixel tree cover loss.