Quality for Preschool Impact Evaluation 2016, Midline Survey - Ghana

Abstract
---------------------------

The Quality Preschool for Ghana Impact Evaluation 2016, Midline survey (QP4G-ML 2016) was approved by the Strategic Impact Evaluation Fund (SIEF) of the World Bank on August 2015 in the Great Accra Region of Ghana. The official project name is called "Testing and scaling-up supply- and demand-side interventions to improve kindergarten educational quality in Ghana”, known as “Quality Preschool for Ghana (QP4G)”.

The project seeks to increase the quality of preschool education during the two years of universal Kindergarten (KG) in Ghana through intervening in the supply-side (i.e., teacher in-service training) and the demand side (i.e., increasing parental awareness for developmentally appropriate quality early education).

The primary goal of the impact evaluation is to test the efficacy of a potentially scalable (8-day) in-service teacher training to improve the quality of KG teacher practices and interactions with children and to improve children’s development, school readiness and learning in both private and public preschools in the Greater Accra Region of Ghana. Additional goals of this evaluation are: to test the added value of combining a scalable (low-cost) parental awareness intervention with teacher in-service training; to compare implementation challenges in public and private schools; and to examine several important sources of potential heterogeneity of impact, primarily impacts in public vs. private schools.

The current submission is for the Midline Survey, conducted with 3 types of respondents across two phases – School survey and Caregiver [household] surveys. The school survey was conducted from May to July 2016 and consisted of collecting the following data: (a) direct assessments of children’s school readiness, (b) surveys of KG teachers, (c) direct observation of inventory of facilities within KG classrooms [environmental scan]; videotaping of KG classroom processes, teaching, and learning (not being submitted); as well as video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted). The caregiver survey was conducted via phone from August to September 2016 on primary caregivers of KG children. The caregiver survey sought information on caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD. Overall, the Midline Survey was conducted from May to September 2016 for all respondents.

Geographic coverage
---------------------------

Urban and Peri-Urban Districts, Greater Accra Region

Analysis unit
---------------------------

Units of analysis include individuals (KG teachers, children, caregivers), KG classrooms and preschools.

Universe
---------------------------

The survey universe is 6 poor districts in the Greater Accra Region. We sampled 240 schools, 108 public (Govt.) schools and 132 private schools. The population of interest is KG teachers and children in KG 1 and KG 2 classrooms in these schools, as well as the caregivers of sampled students.

Kind of data
---------------------------

Sample survey data [ssd]

Sampling procedure
---------------------------

This impact evaluation applies a cluster-randomized design. Eligible schools were randomly selected to participate in the study. The eligible population was schools with KG 1 and KG 2 classrooms (the two years of universal preprimary education) in six districts in the Greater Accra Region. In these six districts, we have sampled 240 schools; 108 public schools and 132 private schools in total.

The unit of randomization for this randomized control trial (RCT) is schools, whereby eligible schools (stratified by public and private sector schools) are randomly assigned to: (1) in-service teacher-training program only; (2) in-service teacher-training program plus parental awareness program; or (3) control (current standard operating) condition.

The sampling frame for this study was based on data in the Education Management Information System (EMIS) from the Ghana Education Service. This data was verified in a 'school listing exercise' conducted in May 2015.

Sample selection was done in four stages:
The first stage involved purposive selection of six districts within the region based on two criteria: (a) most disadvantaged (using UNICEF's District League Table scores, out of sixteen total districts); and (b) close proximity to Accra Metropolitan for travel for the training of the KG teachers. The six selected municipals were La Nkwantanang-Madina Municipal, Ga Central Municipal, Ledzokuku-Krowor Municipal, Adentan Municipal, Ga South Municipal and Ga East Municipal.

The second stage involved the selection of public and private schools from each of the selected districts in the Accra region. We found 678 public and private schools (schools with kindergarten) in the EMIS database. Of these 361 schools were sampled randomly (stratified by district and school type) for the school listing exercise, done in May 2015. This was made up of 118 public schools and 243 private schools. The sampling method used for the school listing exercise was based on two approaches depending on the type of school. For the public schools, the full universe of public schools (i.e., 118) were included in the school listing exercise. However, private schools were randomly sampled using probability proportional to the size of the private schools in each district. Specifically, the private schools were sampled in each district proportionate to the total number of district private schools relative to the total number of private schools. In so doing, one school from the Ga South Municipal was removed and added to Ga Central so that all districts have a number of private schools divisible by three. This approach yielded 122 private schools. Additionally, 20 private schools were randomly selected from each of the districts (i.e., based on the remaining list of private schools in each district following from the first selection) to serve as replacement lists. The replacement list was necessary given the potential refusals from the private schools. There were no replacement lists for the public schools since all public schools would automatically qualify for participation.

The third stage involved selecting the final sample for the evaluation using the sampling frame obtained through the listing exercise. A total of 240 schools were randomly selected, distributed by district and sector. Schools were randomized into treatment groups after the first round of baseline data collection was completed.

The survey respondents were sampled using different sampling techniques:
a. KG teachers: The research team sampled two KG teachers from each school; one from KG1 and KG2. KG teachers were sampled using purposive sampling method. In schools where there were more than two KG classes, the KG teachers from the "A" stream were selected. For the treatment schools, all KG teachers were invited to participate in the teacher training program.
b. KG child-caregiver pair: The research team sampled KG children and their respective caregivers using simple random sampling method. Fifteen KG children-caregivers pair were sampled from each school. For schools with less than 15 KG children (8 from KG1, 7 from KG2 where possible), all KG children were included in the survey. KG children were selected from the same class as the selected KG teacher. The survey team used the class register to randomly select KG children who were present on the day of the school visit. Sampling was not stratified by gender or age. The caregivers of these selected child respondents were invited to participate in the survey.
The research team sought informed consent from the school head teacher, caregivers, as well as child respondents.

Mode of data collection
---------------------------

Other [oth]

Research instrument
---------------------------

Data were collected at Midline Survey using structured questionnaires or forms.

Child Direct Assessment: The KG Child Assessment was conducted using the International Development and Early Learning Assessment (IDELA) tool designed by Save the Children. IDELA was adapted based on extensive pre-testing and piloting by different members of the evaluation team. The adapted version measured five indicators of ECD. The indicators were early numeracy skills, language/literacy skills and development, physical well-being and motor development, socio-emotional development, and approaches to learning. IDELA contained 28 items. In addition, one task was added – the Pencil Tap – to assess executive function skills. Apart from the English language, IDELA was translated and administered into three local languages, namely, Twi, Ga, and Ewe. These local language versions had gone through rigorous processes of translation and back translation. The IDELA tool has not been shared as Save the Children have proprietary rights over this.

KG Class Environmental Scan: The KG classroom observation involved taking inventories of the KG classrooms [environmental scan] and conducting video recordings of the classroom processes. The KG Class Environmental Scan tool was designed to take inventories of the facilities in the KG classrooms. The classroom video recordings have not been shared as they contain PIIs.

TIPPS: The video recordings taken during the classroom observations were coded using an early childhood education adapted version of Teacher Instructional Practices and Processes Systems (TIPPS). Seidman, Raza, Kim, and McCoy (2014) of New York University developed the TIPPS instrument. TIPPS observes nineteen key concepts of teacher practices and classroom processes that influence children’s cognitive and social-emotional development. The concept sheet was used to code the kindergarten classroom videos. The TIPPS tool has not been shared as New York University has proprietary rights over this.

KG Teacher survey: The Midline KG Teacher Survey was based on the modification of the Baseline version. The modifications took into account the data need for measuring outcomes of the teacher training intervention as well as concerns about respondent burden/distress, response rates, and costs. Two modules were added: (a) participation in in-service training (i.e., participation in any in-service training including QP4G teacher training, issues of contamination or spill-over effects, and receipt of text-message intervention); and (b) perceptions of early childhood development. The latter module was culled from the Caregiver Survey. Also, KG teachers who took part in the Baseline II survey were excluded from answering the following time-invariant questions: (a) background characteristics such as local languages of teachers/caregivers, the level of proficiency in speaking and writing in English and local languages; teacher’s paternal and maternal educational level, and (b) English reading knowledge.

Caregiver Survey: The Midline Caregiver survey was based on the modification of the Baseline II Caregiver Survey. The modification involved the removal of the food security, tracking, and mobility updates modules as well as the addition of modules on child discipline and parental participation in the parental awareness raising program. The module on child discipline was adopted from the UNICEF’s MICS 2013 Household Questionnaire. A Call Records Form was also designed to track and screen each caregiver before the actual interview was conducted. Four local language dictionaries of keywords and phrases were developed for the Caregiver Survey. The selected languages were Ga, Ewe, Twi, and Hausa. These languages were selected because they were used extensively in interviewing the caregivers at Baseline II survey.

School Attendance Records: The School Attendance Records Form was designed to record school attendance information for the sampled KG teachers and children. The Form captured school-specific attendance details such as the active number of school days, the number of national/school-related holidays per term, and child-specific information such as present/absent from school.

Questionnaires are provided under the Related Materials tab.

Cleaning operations
---------------------------

Data consistency checks namely high-frequency checks and backchecks (audits) were conducted for all surveys remotely. Corrections were made during and after data collection after errors were reconciled.

All checks and cleaning was done using STATA and IPA data management systems. IPA possesses all the relevant code.

Response rate
---------------------------

Schools: Of the 240 randomly sampled schools at baseline, 235 were surveyed at Midline. This represents 97% of the sample schools. Of the remaining 3% (n = 5), three schools dropped out of the study and 2 schools closed down.

KG teachers: 78% of KG teachers were interviewed at Midline survey. Notably, we interviewed 348 of the 444 teachers surveyed at baseline. This is because 96 teachers had either transfered (moved) to other schools (or classes) or had left the teaching profession.

KG children: Of the 3435 children who were assessed at baseline, 2975 were assessed at Midline. This represents 87% coverage rate. Thirteen percent of the children could no be tracked at Midline due largely to change in school [because of family migration outside the catchment districts or region].

Caregivers: Caregiver participation in the QP4G study increased from 2134 at Baseline to 2710 at Midline. This represented 27% increase in the caregiver coverage over the baseline. The increase in the caregiver coverage was largely due to the 576 additional active contact numbers obtained from the caregivers. These caregiver, hitherto, had no contact numbers at baseline.

摘要
---------------------------

2016年加纳质量学前班影响评估中期调查（QP4G-ML 2016）于2015年8月获得世界银行战略影响评估基金（SIEF）的批准，在加纳大阿克拉地区进行。该项目的正式名称为“测试和扩大供应和需求侧干预措施，以提升加纳幼儿园教育质量”，简称“加纳质量学前班（QP4G）”。

该项目旨在通过干预供应侧（即，教师在职培训）和需求侧（即，提高家长对适合儿童发展质量的早期教育的意识）来提高加纳两年普及学前班（KG）教育质量。

影响评估的主要目标是测试一种可能具有可扩展性（8天）的教师在职培训的效力，以改善KG教师的教学实践和与儿童的互动，并提高儿童在加纳大阿克拉地区公立和私立幼儿园中的发展、入学准备和学习质量。该评估的附加目标还包括：测试将可扩展性（低成本）的家长意识干预措施与教师在职培训相结合的增值；比较公立和私立学校的实施挑战；以及检验影响潜在异质性的几个重要来源，主要是公立学校与私立学校之间的影响。

当前提交的是中期调查，该调查在两个阶段进行了3种类型受访者的调查——学校调查和照顾者[家庭]调查。学校调查于2016年5月至7月进行，包括收集以下数据：（a）对儿童入学准备的直接评估，（b）对KG教师的调查，（c）对KG教室内部设施的直接观察[环境扫描]；KG教室过程、教学和学习的录像（不提交）；以及使用教师教学实践和过程系统（未提交的工具）对KG教室视频录像进行编码。照顾者调查通过电话于2016年8月至9月对KG儿童的初级照顾者进行。照顾者调查旨在收集照顾者的背景、贫困状况、参与或参与学校和家庭活动的情况，以及对早期儿童发展（ECD）的看法。总的来说，中期调查于2016年5月至9月对所有受访者进行。

地理覆盖范围
---------------------------

大阿克拉地区的城市和郊区

分析单位
---------------------------

分析单位包括个人（KG教师、儿童、照顾者）、KG教室和幼儿园。

总体
---------------------------

调查总体是大阿克拉地区6个贫困区。我们抽样了240所学校，包括108所公立（政府）学校和132所私立学校。感兴趣的群体是这些学校KG 1和KG 2教室中的KG教师和儿童，以及抽样学生的照顾者。

数据类型
---------------------------

样本调查数据 [ssd]

抽样程序
---------------------------

本影响评估采用集群随机设计。符合条件的学校被随机选中参与研究。符合条件的群体是在大阿克拉地区六个区的KG 1和KG 2教室（两年普及学前教育）的学校。在这六个区中，我们抽样了240所学校；总共108所公立学校和132所私立学校。

本随机对照试验（RCT）的随机化单位是学校，符合条件的学校（按公立和私营部门学校分层）被随机分配到：（1）仅在职教师培训计划；（2）在职教师培训计划加家长意识计划；或（3）控制（当前标准操作）条件。

本研究的研究框架基于加纳教育服务（Ghana Education Service）的教育管理信息系统（EMIS）中的数据。这些数据在2015年5月进行的“学校清单作业”中得到了验证。

样本选择分为四个阶段：
第一阶段涉及根据两个标准在区域内有目的地选择六个区：（a）最不发达的（使用联合国儿童基金会（UNICEF）的区联赛表得分，共十六个区）；以及（b）靠近阿克拉大都会，以便为KG教师的培训旅行。根据这两个标准，选出的六个市政分别是拉恩克瓦塔南-马达纳市政、加中央市政、莱德佐库-克罗沃市政、阿丹坦市政、加南部市政和加东部市政。

第二阶段涉及从每个选定的阿克拉地区区选择公立和私立学校。我们在EMIS数据库中发现了678所公立和私立学校（有幼儿园的学校）。其中361所学校被随机抽样（按区和学生类型分层）进行学校清单作业，于2015年5月进行。这包括118所公立学校和243所私立学校。学校清单作业中使用的抽样方法基于两种方法，具体取决于学校的类型。对于公立学校，包括在清单作业中的公立学校（即118所）。然而，私立学校是按每个区的私立学校规模的比例随机抽样的。具体来说，私立学校在每个区的抽样比例与该区私立学校的总数相对于私营学校总数的比例成比例。这样，从加南部市政抽走了一所学校，并加到了加中央，以确保所有地区都有可被3整除的私立学校数量。这种方法产生了122所私立学校。此外，从每个区随机选择了20所私立学校（即，基于每个区的剩余私立学校名单，从第一次选择中得出），作为备选名单。备选名单是必要的，因为考虑到私立学校可能有拒绝的情况。公立学校没有备选名单，因为所有公立学校都会自动有资格参加。