Content Analysis of U.S. Local Adaptation Plans.xlsx

Purpose of the data:

This
data represents the content and quality of 44 local climate change adaptation
plans in the United States. Climate change already affects local communities
across the country, and these impacts are projected to become more severe and
intense in the future. In response, many local
communities are investing in adaptation – actions to limit the negative
consequences of climate change – and, in many cases, creating climate
adaptation plans. These plans detail how climate change is projected to impact
the community and what actions should be taken to prepare.
Adaptation planning represents systematic attention to climate change and, as a
result, is expected to help prepare communities and lower the cost of
climate-related impacts. This data represents how adaptation plans align with
agreed-upon criteria of plan quality and allows users to identify specific strengths
and areas for improvement.

How this data was created:

This
data was created using a content analysis based plan evaluation. First, we
developed a set of criteria corresponding
to the plan quality characteristics
based on existing planning guidance documents. Second, we created a sample of
44 local adaptation plans in the U.S. Third, we systematically read plans to identify whether they fulfill the specified
criteria. Assessing the presence/absence of criteria allows the conversion of text to
a quantitative measurement of plan quality, which eases comparisons between
plans and permits statistical analyses. Finally,
we used a grounded theory approach to analyze the text originally coded as
strategies to ensure an accurate representation of adaptation strategies
proposed by local governments in the sample.

Researchers
have used plan evaluation to analyze plans from multiple domains, including
hazard mitigation, affordable housing, and sustainability planning. As plan quality evaluation has become more
common, consensus has emerged on the core characteristics, or principles, of
plan quality. Among these core characteristics are goals, fact base, policies,
public participation in the planning process, and details concerning
implementation and monitoring. These characteristics are applicable across
planning domains and scales (e.g. municipal, county, state). When used
effectively, plan quality evaluations can identify specific strengths and
review the effectiveness of planning processes, providing the opportunity improve
planning in the future. It is important to note that plan quality evaluation
and plan quality guidance focus on plans themselves and not the outcomes plans
produce.

Coding protocol

            We developed a coding protocol to assess seven principles
of plan quality: 1) goals, 2) fact base, 3) strategies, 4) public participation in plan
creation, 5) inter-organizational coordination, 6) details regarding
implementation and monitoring, and 7)
how plans deal with uncertainty. To ensure that the protocol captured the most
current theory on adaptation, metrics for each principle were based on an
analysis of nine adaptation guidance documents published by international,
federal, state, and non-governmental organizations. Despite the diversity in
guidance documents considered, there was a high level of agreement on adaptation
processes and factors that should be considered in climate adaptation planning.
From this analysis, we extracted processes and considerations that are present
across multiple adaptation guidance instruments and therefore could serve as
evaluation metrics. In
total, we developed a protocol with 124 metrics across the seven principles. A
description of each metric and how it was applied in the coding process are
provided in column B and C, respectively.

The
metrics pre-tested on eight local adaptation plans from Europe and Australia.
The pre-testing allowed us to train the three coders and refine the metrics and
coding instructions so that they captured the concepts intended.

Sample Selection

We
selected plans to include in our sample based on three criteria: (1) the
central topic of the plan was adaptation, resilience, or preparedness; (2) the
plan was written by or for a U.S. city or county government; and (3) the plan
took a comprehensive approach to adaptation by focusing on more than just one
or two topics (i.e., we exclude sector-based adaptation plans). These criteria
excluded plans that integrate adaptation components but do not focus entirely
on adaptation (e.g., climate action plans and sustainability plans that
dedicate a chapter to adaptation), plans that are written by regional entities
(e.g., the Southeast Florida Regional Climate Change Compact), and plans that
are written without local government involvement (e.g., plans written by local
environmental non-profits).

We
evaluated all U.S. plans that we could find that met these criteria and were released
between 2007, when Keene, NH published the first adaptation plan in the U.S.,
and 2014. We developed the sample based on a search of three adaptation
clearinghouse websites: the Georgetown Climate Center, the Climate Adaptation
Knowledge Exchange (CakeX), and the Center for Climate and Energy Solutions. In
addition, we collected plans through three 100-page Google searches for the
terms “local adaptation plan,” “local resilience plan,” and “local preparedness
plan.”

Coding Procedures

Each
plan was coded independently by two of the trained coders in line with
recommendations from the communications literature on content analysis and
recommendations from the plan evaluation literature on methodology. Before
coding plans in the sample, we calculated inter-coder reliability to ensure
that the coders fell within an appropriate range of inter-coder agreement (0.80
or greater).

            Coders used the NVivo
version 10 qualitative analysis software package to link coding items with the content of plans. After the coders
completed a plan, we compared their quantitative data to identify disagreements
on a metric-by-metric basis. All disagreements were discussed and reconciled by
referring to the qualitative plan content, and the final, agreed-upon codes
were integrated into a master dataset.

We
calculated inter-coder reliability scores for each plan and code using two
measures: percent
agreement and Krippendorff’s alpha reported in column AY for each code and row
149 for each plan. To achieve equal
weighting of the codes for each principle, we calculated index scores for each
plan principle by summing the reconciled scores for the principle and dividing
by the number of codes in that principle. We calculated total plan quality by
averaging index scores for each plan principle. These results were presented in
Woodruff and Stults (2016).

Strategies

Once
all 44 plans had been analyzed, we extracted all of the text related to the
type of adaptation strategy and re-analyzed it using a modified Grounded Theory Method. Each author independently reviewed and classified
strategies, moving strategies to more
appropriate types and creating new types of strategies, if necessary, to more
accurately reflect what the local plans were proposing. To be consistent,
we coded all strategies based on the action being proposed, not the intent of
the action. This meant that if a strategy was a policy change that would
incentivize more resilient building codes, we coded it as being a policy
strategy and not a building code strategy. By doing this, we were able to code
the strategies as presented by the plan authors, avoiding the need to interpret
the plan authors’ intent. All differences were
collaboratively reconciled by referring back to the adaptation plan and looking
for similarities between the strategy in question and other strategies. In the
end, we classified each adaptation activity as one of seventeen types. When
appropriate, adaptation strategies were co-tagged as multiple types. For
example, Baltimore MD’s strategy to “encourage the development of integrated
flood protection systems that use structural (engineering) and non-structural
(wetlands) measures” was double tagged as being both a physical infrastructure
strategy and a green infrastructure strategy. Because coding of the strategies
was based on a more qualitative and iterative process, no inter-coder agreement
scores are reported for the strategy-type metrics.

Time and date of creation:

Plans were collected and analyzed
in 2014 – 2015. No plans released after December 31, 2014 were included in our
sample. Results of this work was published in Nature Climate Change in May 2016 and has lead to several
subsequent articles.

For additional details and analysis of this data please see:

Woodruff, S. C., & Stults, M. (2016). Numerous
strategies but limited implementation guidance in US local adaptation plans. Nature
Climate Change. http://doi.org/10.1038/nclimate3012