Global Education Policy Dashboard 2022 - Sierra Leone

Abstract
---------------------------

The dashboard project collects new data in each country using three new instruments: a School Survey, a Policy Survey, and a Survey of Public Officials. Data collection involves school visits, classroom observations, legislative reviews, teacher and student assessments, and interviews with teachers, principals, and public officials. In addition, the project draws on some existing data sources to complement the new data it collects. A major objective of the GEPD project was to develop focused, cost-effective instruments and data-collection procedures, so that the dashboard can be inexpensive enough to be applied (and re-applied) in many countries. The team achieved this by streamlining and simplifying existing instruments, and thereby reducing the time required for data collection and training of enumerators.

Geographic coverage
---------------------------

National

Analysis unit
---------------------------

Schools, teachers, students, public officials

Kind of data
---------------------------

Sample survey data [ssd]

Sampling procedure
---------------------------

The aim of the Global Education Policy Dashboard school survey is to produce nationally representative estimates, which will be able to detect changes in the indicators over time at a minimum power of 80% and with a 0.05 significance level. We also wish to detect differences by urban/rural location. For our school survey, we will employ a two-stage random sample design, where in the first stage a sample of typically around 200 schools, based on local conditions, is drawn, chosen in advance by the Bank staff. In the second stage, a sample of teachers and students will be drawn to answer questions from our survey modules, chosen in the field. A total of 10 teachers will be sampled for absenteeism. Five teachers will be interviewed and given a content knowledge exam. Three 1st grade students will be assessed at random, and a classroom of 4th grade students will be assessed at random. Stratification will be based on the school’s urban/rural classification and based on region. When stratifying by region, we will work with our partners within the country to make sure we include all relevant geographical divisions. For our Survey of Public Officials, we will sample a total of 200 public officials. Roughly 60 officials are typically surveyed at the federal level, while 140 officials will be surveyed at the regional/district level. For selection of officials at the regional and district level, we will employ a cluster sampling strategy, where roughly 10 regional offices (or whatever the secondary administrative unit is called) are chosen at random from among the regions in which schools were sampled. Then among these 10 regions, we also typically select around 10 districts (tertiary administrative level units) from among the districts in which schools werer sampled. The result of this sampling approach is that for 10 clusters we will have links from the school to the district office to the regional office to the central office. Within the regions/districts, five or six officials will be sampled, including the head of organization, HR director, two division directors from finance and planning, and one or two randomly selected professional employees among the finance, planning, and one other service related department chosen at random. At the federal level, we will interview the HR director, finance director, planning director, and three randomly selected service focused departments. In addition to the directors of each of these departments, a sample of 9 professional employees will be chosen in each department at random on the day of the interview.

Sampling deviation
---------------------------

The sample for the Global Education Policy Dashboard in SLE was based in part on a previous sample of 260 schools which were part of an early EGRA study. Details from the sampling for that study are quoted below. An additional booster sample of 40 schools was chosen to be representative of smaller schools of less than 30 learners.

EGRA Details:

"The sampling frame began with the 2019 Annual School Census (ASC) list of primary schools as provided by UNICEF/MBSSE where the sample of 260 schools for this study were obtained from an initial list of 7,154 primary schools. Only schools that meet a pre-defined selection criteria were eligible for sampling.

To achieve the recommended sample size of 10 learners per grade, schools that had an enrolment of at least 30 learners in Grade 2 in 2019 were considered. To achieve a high level of confidence in the findings and generate enough data for analysis, the selection criteria only considered schools that:
• had an enrolment of at least 30 learners in grade 1; and
• had an active grade 4 in 2019 (enrolment not zero)

The sample was taken from a population of 4,597 primary schools that met the eligibility criteria above, representing 64.3% of all the 7,154 primary schools in Sierra Leone (as per the 2019 school census). Schools with higher numbers of learners were purposefully selected to ensure the sample size could be met in each site.

As a result, a sample of 260 schools were drawn using proportional to size allocation with simple random sampling without replacement in each stratum. In the population, there were 16 districts and five school ownership categories (community, government, mission/religious, private and others). A total of 63 strata were made by forming combinations of the 16 districts and school ownership categories. In each stratum, a sample size was computed proportional to the total population and samples were drawn randomly without replacement. Drawing from other EGRA/EGMA studies conducted by Montrose in the past, a backup sample of up to 78 schools (30% of the sample population) with which enumerator teams can replace sample schools was also be drawn.

In the distribution of sampled schools by ownership, majority of the sampled schools are owned by mission/religious group (62.7%, n=163) followed by the government owned schools at 18.5% (n=48). Additionally, in school distribution by district, majority of the sampled schools (54%) were found in Bo, Kambia, Kenema, Kono, Port Loko and Kailahun districts. Refer to annex 9. for details on the population and sample distribution by district."

Because of the restriction that at least 30 learners were available in Grade 2, we chose to add an additional 40 schools to the sample from among smaller schools, with between 3 and 30 grade 2 students. The objective of this supplement was to make the sample more nationally representative, as the restriction reduced the sampling frame for the EGRA/EGMA sample by over 1,500 schools from 7,154 to 4,597.

The 40 schools were chosen in a manner consistent with the original set of EGRA/EGMA schools. The 16 districts formed the strata. In each stratum, the number of schools selected were proportional to the total population of the stratum, and within stratum schools were chosen with probability proportional to size.

Mode of data collection
---------------------------

Computer Assisted Personal Interview [capi]

Research instrument
---------------------------

The dashboard project collects new data in each country using three new instruments: a School Survey, a Policy Survey, and a Survey of Public Officials. Data collection involves school visits, classroom observations, legislative reviews, teacher and student assessments, and interviews with teachers, principals, and public officials. In addition, the project draws on some existing data sources to complement the new data it collects. A major objective of the GEPD project was to develop focused, cost-effective instruments and data-collection procedures, so that the dashboard can be inexpensive enough to be applied (and re-applied) in many countries. The team achieved this by streamlining and simplifying existing instruments, and thereby reducing the time required for data collection and training of enumerators.

More information pertaining to each of the three instruments can be found below:
- School Survey: The School Survey collects data primarily on practices (the quality of service delivery in schools), but also on some de facto policy indicators. It consists of streamlined versions of existing instruments—including Service Delivery Surveys on teachers and inputs/infrastructure, Teach on pedagogical practice, Global Early Child Development Database (GECDD) on school readiness of young children, and the Development World Management Survey (DWMS) on management quality—together with new questions to fill gaps in those instruments. Though the number of modules is similar to the full version of the Service Delivery Indicators (SDI) Survey, the number of items and the complexity of the questions within each module is significantly lower. The School Survey includes 8 short modules: School Information, Teacher Presence, Teacher Survey, Classroom Observation, Teacher Assessment, Early Learner Direct Assessment, School Management Survey, and 4th-grade Student Assessment. For a team of two enumerators, it takes on average about 4 hours to collect all information in a given school. For more information, refer to the Frequently Asked Questions.

- Policy Survey: The Policy Survey collects information to feed into the policy de jure indicators. This survey is filled out by key informants in each country, drawing on their knowledge to identify key elements of the policy framework (as in the SABER approach to policy-data collection that the Bank has used over the past 7 years). The survey includes questions on policies related to teachers, school management, inputs and infrastructure, and learners. In total, there are 52 questions in the survey as of June 2020. The key informant is expected to spend 2-3 days gathering and analyzing the relavant information to answer the survey questions.

- Survey of Public Officials: The Survey of Public Officials collects information about the capacity and orientation of the bureaucracy, as well as political factors affecting education outcomes. This survey is a streamlined and education-focused version of the civil-servant surveys that the Bureaucracy Lab (a joint initiative of the Governance Global Practice and the Development Impact Evaluation unit of the World Bank) has implemented in several countries. The survey includes questions about technical and leadership skills, work environment, stakeholder engagement, impartial decision-making, and attitudes and behaviors. The survey takes 30-45 minutes per public official and is used to interview Ministry of Education officials working at the central, regional, and district levels in each country.

Cleaning operations
---------------------------

Data quality control was performed in R and Stata.

Code to calculate all indicators can be found on github here:
https://github.com/worldbank/GEPD/blob/master/Countries/Sierra Leone/2022/School/01_data/03_school_data_cleaner.Rmd

Sampling error estimates
---------------------------

The aim of the Global Education Policy Dashboard school survey is to produce nationally representative estimates, which will be able to detect changes in the indicators over time at a minimum power of 80% and with a 0.05 significance level.

摘要
---------------------------
该仪表盘项目采用三种新型工具在各个国家收集新数据：学校调查、政策调查和公职人员调查。数据收集涉及学校访问、课堂观察、立法审查、教师和学生评估，以及与教师、校长和公职人员的访谈。此外，该项目还利用一些现有数据源来补充其收集的新数据。GEPD项目的主要目标是开发聚焦、成本效益高的工具和数据收集程序，以便仪表盘能够以较低的成本应用于（并重新应用于）许多国家。团队通过简化现有工具并减少数据收集和培训统计人员所需的时间，实现了这一目标。
地理覆盖范围
---------------------------
全国
分析单元
---------------------------
学校、教师、学生、公职人员
数据类型
---------------------------
样本调查数据 [ssd]
抽样程序
---------------------------
全球教育政策仪表盘学校调查的目的是产生全国代表性估计值，这些估计值能够在至少80%的最低功效和0.05的显著性水平下检测到指标随时间的变化。我们还想检测城市/农村位置的差异。对于我们的学校调查，我们将采用两阶段随机抽样设计，第一阶段根据当地条件抽取约200所学校样本，由银行工作人员提前选定。在第二阶段，将抽取教师和学生样本以回答我们的调查模块中的问题，选择在实地进行。总共将抽取10名教师以评估缺勤情况。将访谈5名教师并对其进行内容知识考试。将随机评估3名一年级学生和随机评估一个四年级学生班级。分层将基于学校的城市/农村分类和基于地区。在按地区分层时，我们将与国内合作伙伴合作，确保包括所有相关的地理分区。对于我们的公职人员调查，我们将抽取总计200名公职人员。通常在联邦层面调查约60名官员，而在地区/区级层面调查140名官员。在地区和区级官员的选择中，我们将采用聚类抽样策略，从抽样学校所在的地区中随机选择约10个地区办公室（或称二级行政单位）。然后在这些10个地区中，我们还通常从抽样学校所在的地区中选择约10个区（三级行政单位）。这种抽样方法的结果是，对于10个集群，我们将从学校到区办公室到地区办公室到中央办公室建立联系。在地区/区内，将抽取五或六名官员，包括机构负责人、人力资源总监、财务和规划部门的两位部门总监，以及从财务、规划和随机选择的另一个服务相关部门随机选择的1至2名专业员工。在联邦层面，我们将访谈人力资源总监、财务总监、规划总监和三个随机选择的服务导向部门。除了这些部门的负责人外，在面试当天，每个部门将随机选择9名专业员工。
抽样偏差
---------------------------
SLE的全球教育政策仪表盘样本基于部分基于早期EGRA研究中260所学校的先前样本。以下引用了该研究的抽样细节。还选择了一个额外的40所学校样本，以代表学习者数量少于30人的小型学校。
EGRA详情：
“抽样框架始于联合国儿童基金会/MBSSE提供的2019年年度学校普查（ASC）清单中的小学名单，该研究从7,154所小学的初始清单中获得了260所学校的样本。只有符合预先定义的选拔标准的学校才有资格进行抽样。
为了实现每级推荐样本量为10名学习者的样本量，2019年在二年级至少有30名学习者的学校被认为是合适的。为了确保调查结果的置信度高并产生足够的数据进行分析，选拔标准仅考虑了以下学校：
• 在2019年一年级至少有30名学习者；
• 2019年有活跃的四年级（注册人数非零）。
样本是从符合上述资格的4,597所小学的总人口中抽取的，代表了塞拉利昂7,154所小学的64.3%（根据2019年学校普查）。有更多学习者的学校被有意选择，以确保每个地点的样本量可以满足。
因此，使用按比例分配的简单随机抽样无替换在每个层中进行，抽取了260所学校的样本。在总人口中，有16个区和五个学校所有权类别（社区、政府、使命/宗教、私立和其他）。通过组合16个区和学校所有权类别，形成了63个层。在每个层中，样本量是根据总人口按比例计算的，并从每个层中随机抽取样本。从Montrose过去进行的其他EGRA/EGMA研究中抽取的样本中，还抽取了一个最多78所学校（样本总人口的30%）的备用样本，以便调查员团队可以替换样本学校。
在抽样学校的所有权分布中，大多数抽样学校由使命/宗教团体拥有（62.7%，n=163），其次是政府拥有的学校，占18.5%（n=48）。此外，在学校按区的分布中，大多数抽样学校（54%）分布在波、卡比亚、肯ema、科诺、波洛科和凯拉胡恩区。有关人口和按区样本分布的详细信息，请参阅附件9。
由于至少在二年级有30名学习者的限制，我们选择从3至30名二年级学生的小型学校中添加额外的40所学校样本。补充的目标是使样本更具全国代表性，因为该限制将EGRA/EGMA样本的抽样框架从7,154所减少到4,597所。
40所学校的选择方式与原始的EGRA/EGMA学校集合一致。16个区形成层。在每个层中，所选学校的数量与层的总人口按比例，并在层内按比例选择学校。
数据收集方式
---------------------------
计算机辅助个人访谈 [capi]
研究工具
---------------------------
仪表盘项目在每个国家使用三种新型工具收集新数据：学校调查、政策调查和公职人员调查。数据收集涉及学校访问、课堂观察、立法审查、教师和学生评估，以及与教师、校长和公职人员的访谈。此外，该项目还利用一些现有数据源来补充其收集的新数据。GEPD项目的主要目标是开发聚焦、成本效益高的工具和数据收集程序，以便仪表盘能够以较低的成本应用于（并重新应用于）许多国家。团队通过简化现有工具并减少数据收集和培训统计人员所需的时间，实现了这一目标。
关于三个工具的更多信息如下：
- 学校调查：学校调查主要收集关于实践（学校服务提供质量）的数据，但也收集一些事实上的政策指标。它包括现有工具的简化版本，包括教师和投入/基础设施的服务交付调查、教学实践调查、全球幼儿早期发展数据库（GECDD）关于幼儿入学准备情况以及关于管理质量的开发世界管理调查（DWMS），以及一些新问题以填补这些工具中的空白。尽管模块的数量与全面版本的服务交付指标（SDI）调查相似，但每个模块的项目数量和问题的复杂性显著降低。学校调查包括8个简短模块：学校信息、教师出勤、教师调查、课堂观察、教师评估、早期学习者直接评估、学校管理调查和四年级学生评估。对于两名统计人员，平均大约需要4小时收集给定学校中的所有信息。更多信息，请参阅常见问题解答。
- 政策调查：政策调查收集信息以输入政策法定的指标。该调查由每个国家的关键信息提供者填写，利用他们的知识来识别政策框架的关键要素（如银行过去7年使用的SABER方法收集政策数据）。调查包括与教师、学校管理、投入和基础设施以及学习者相关的政策问题。截至2020年6月，调查中有52个问题。关键信息提供者预计将花费2-3天收集和分析相关信息以回答调查问题。
- 公职人员调查：公职人员调查收集有关官僚机构的能力和取向，以及影响教育成果的政治因素的信息。该调查是官僚实验室（世界银行治理全球实践和发展影响评估单位的一个联合倡议）在几个国家实施的国家公务员调查的简化、以教育为重点的版本。调查包括关于技术和领导技能、工作环境、利益相关者参与、不偏不倚的决策以及态度和行为的问题。每位公职人员的调查需要30-45分钟，并用于采访每个国家在中央、地区和区级工作的教育部官员。
数据清洗操作
---------------------------
在R和Stata中进行了数据质量控制。
计算所有指标的代码可以在以下GitHub上找到：
https://github.com/worldbank/GEPD/blob/master/Countries/Sierra Leone/2022/School/01_data/03_school_data_cleaner.Rmd
抽样误差估计
---------------------------
全球教育政策仪表盘学校调查的目的是产生全国代表性估计值，这些估计值能够在至少80%的最低功效和0.05的显著性水平下检测到指标随时间的变化。