Southern and Eastern Africa Consortium for Monitoring Educational Quality 2000 - Tanzania

Abstract
---------------------------

In 1991 the International Institute for Educational Planning (IIEP) and a number of Ministries of Education in Southern and Eastern Africa began to work together in order to address training and research needs in Education. The focus for this work was on establishing long-term strategies for building the capacity of educational planners to monitor and evaluate the quality of their basic education systems. The first two educational policy research projects undertaken by SACMEQ (widely known as "SACMEQ I" and "SACMEQ II") were designed to provide detailed information that could be used to guide planning decisions aimed at improving the quality of education in primary school systems.

During 1995-1998 seven Ministries of Education participated in the SACMEQ I Project. The SACMEQ II Project commenced in 1998 and the surveys of schools, involving 14 Ministries of Education, took place between 2000 and 2004. The survey was undertaken in schools in Botswana, Kenya, Lesotho, Malawi, Mauritius, Mozambique, Namibia, Seychelles, South Africa, Swaziland, Tanzania, Uganda, Zambia and Zanzibar.

Moving from the SACMEQ I Project (covering around 1100 schools and 20,000 pupils) to the SACMEQ II Project (covering around 2500 schools and 45,000 pupils) resulted in a major increase in the scale and complexity of SACMEQ's research and training programmes.

SACMEQ's mission is to:
a) Expand opportunities for educational planners to gain the technical skills required to monitor and evaluate the quality of their education systems; and
b) Generate information that can be used by decision-makers to plan and improve the quality of education.

Geographic coverage
---------------------------

The survey covered Mainland Tanzania.

Analysis unit
---------------------------

- Pupils
- Teachers
- Schools

Universe
---------------------------

The target population for SACMEQ's Initial Project was defined as "all pupils at the Grade 6 level in 1995 who were attending registered government or non-government schools". Grade 6 was chosen because it was the grade level where the basics of reading literacy were expected to have been acquired.

Kind of data
---------------------------

Sample survey data [ssd]

Sampling procedure
---------------------------

The sample designs used in the SACMEQ II Project were selected so as to meet the standards set down by the International Association for the Evaluation of Educational Achievement. These standards required that sample estimates of important pupil population parameters should have sampling accuracy that was at least equivalent to a simple random sample of 400 pupils (thereby guaranteeing 95 percent confidence limits for sample means of plus or minus one tenth of a pupil standard deviation unit).

Some Constraints on Sample Design
Sample designs in the field of education are usually prepared amid a network of competing constraints. These designs need to adhere to established survey sampling theory and, at the same time, give due recognition to the financial, administrative, and socio-political settings in which they are to be applied. The "best" sample design for a particular project is one that provides levels of sampling accuracy that are acceptable in terms of the main aims of the project, while simultaneously limiting cost, logistic, and procedural demands to manageable levels. The major constraints that were established prior to the preparation of the sample designs for the SACMEQ II Project have been listed below.

Target Population: The target population definitions should focus on Grade 6 pupils attending registered mainstream government or non-government schools. In addition, the defined target population should be constructed by excluding no more than 5 percent of pupils from the desired target population.

Bias Control: The sampling should conform to the accepted rules of scientific probability sampling. That is, the members of the defined target population should have a known and non-zero probability of selection into the sample so that any potential for bias in sample estimates due to variations from "epsem sampling" (equal probability of selection method) may be addressed through the use of appropriate sampling weights (Kish, 1965).

Sampling Errors: The sample estimates for the main criterion variables should conform to the sampling accuracy requirements set down by the International Association for the Evaluation of Educational Achievement (Ross, 1991). That is, the standard error of sampling for the pupil tests should be of a magnitude that is equal to, or smaller than, what would be achieved by employing a simple random sample of 400 pupils (Ross, 1985).

Response Rates: Each SACMEQ country should aim to achieve an overall response rate for pupils of 80 percent. This figure was based on the wish to achieve or exceed a response rate of 90 percent for schools and a response rate of 90 percent for pupils within schools.

Administrative and Financial Costs: The number of schools selected in each country should recognize limitations in the administrative and financial resources available for data collection.

Other Constraints: The number of pupils selected to participate in the data collection in each selected school should be set at a level that will maximize validity of the within-school data collection for the pupil reading and mathematics tests.
The Specification of the Target Population
For Tanzania the desired target population was all pupils enrolled in Standard 6 in the ninth month of the school year (i.e., in the first week of December 2000). A decision was made to exclude pupils in special schools and those in schools which had fewer than 20 Standard 6 pupils which led to the establishment of the defined target population.

Tanzania there were 10,786 schools having 529,296 Standard 6 pupils. The excluded population was 17,942 pupils from 1,270 schools which was 3.3 percent of all pupils. The defined population from which a sample had to be drawn consisted of 511,354 pupils from 9,516 schools.

Note: Detailed descriptions of the sample design, sample selection, and sample evaluation procedures have been presented in the "Tanzania Working Report".

Mode of data collection
---------------------------

Face-to-face [f2f]

Research instrument
---------------------------

The data collection for SACMEQ’s Initial Project took place in October 1995 and involved the administration of questionnaires to pupils, teachers, and school heads.
The pupil questionnaire contained questions about the pupils’ home backgrounds and their school life; the teacher questionnaire asked about classrooms, teaching practices, working conditions, and teacher housing; and the school head questionnaire collected information about teachers, enrolments, buildings, facilities, and management. A reading literacy test was also given to the pupils. The test was based on items that were selected after a trial-testing programme had been completed.

Cleaning operations
---------------------------

Data Entry and Data Cleaning
Six persons from the National Examination Council of Tanzania (NECTA) and MOEC were appointed and trained in the use of WINDEM, a special data entry package to be used in SACMEQ. NECTA and MOEC computers were used for Data entry and data cleaning. The process was facilitated by written instructions and follow- up support from IIEP staff in the basic steps mainly via the internet and permitted the NRCs to:
(i) identify major errors in the sequence of identification numbers,
(ii) cross-check identification numbers across files (for example, to ensure that all pupils were linked with their own reading and mathematics teachers),
(iii) ensure that all schools listed on the original sampling frame also had valid data collection instruments and vice-versa,
(iv) check for "wild codes" that occurred when some variables had values that fell outside pre-specified reasonable limits, and validate that variables used as linkage devices in later file merges were available and accurate.

The volume of information required to be entered and cleaned in the code sheets was immense despite the user friendliness of the software thus perseverance and experience of keyboard operation was required. The following can help one imagine the volume of work entered. Data collection instruments contained the follows information to be coded: school form: 58; pupil name form: 51; pupil questionnaire: 150; pupil reading test: 85; pupil mathematics test: 65; teacher questionnaire: 587; teacher reading test: 51; teacher mathematics test: 43; and school head questionnaire: 319. All the data entered were sent to the IIEP for checking in order to ensure that there were no errors such as inconsistencies or wild values. The IIEP then sent back the data to Tanzania for cleaning, after which the Ministry sent it back to IIEP for further checks. This process continued until the data was absolutely clean, and it took 21 months to complete (March 2001and November 2002).

Response rate
---------------------------

Response rates for pupils and schools respectively were 77% and 98%.

Sampling error estimates
---------------------------

The sample designs employed in the SACMEQ Projects departed markedly from the usual "textbook model" of simple random sampling. This departure demanded that special steps be taken in order to calculate "sampling errors" (that is, measures of the stability of sample estimates of population characteristics).

摘要
---------------------------
1991年，国际教育规划研究所（IIEP）与南部和东部非洲的多个教育部合作，旨在解决教育领域的培训和科研需求。该工作的重点在于制定长期战略，以增强教育规划者的能力，监控和评估其基础教育体系的质量。SACMEQ（广为人知的“SACMEQ I”和“SACMEQ II”）开展的前两个教育政策研究项目旨在提供详细的信息，这些信息可用于指导旨在提高小学教育体系质量的教育规划决策。

1995年至1998年间，七个教育部参与了SACMEQ I项目。SACMEQ II项目于1998年开始，涉及14个教育部的学校调查在2000年至2004年之间进行。调查在博茨瓦纳、肯尼亚、莱索托、马拉维、毛里求斯、莫桑比克、纳米比亚、塞舌尔、南非、斯威士兰、坦桑尼亚、乌干达、赞比亚和桑给巴尔进行。

从涵盖约1100所学校和20000名学生的SACMEQ I项目转向涵盖约2500所学校和45000名学生的SACMEQ II项目，导致SACMEQ的研究和培训计划在规模和复杂性上有了显著增加。

SACMEQ的使命是：
a) 扩大教育规划者获取监控和评估其教育体系质量所需的技术技能的机会；
b) 生成信息，供决策者用于规划和提高教育质量。

地理覆盖范围
---------------------------
调查覆盖了坦桑尼亚大陆。
分析单元
---------------------------
- 学生
- 教师
- 学校

总体
---------------------------
SACMEQ的初始项目针对的目标人群被定义为“1995年所有在注册政府或非政府学校就读的六年级学生”。六年级被选中，因为在这一年级，学生应已掌握阅读素养的基本知识。

数据类型
---------------------------
样本调查数据 [ssd]

抽样程序
---------------------------
SACMEQ II项目中使用的样本设计是为了满足国际教育成就评估协会设定的标准。这些标准要求对重要学生人群参数的样本估计应具有至少等同于400名学生的简单随机样本的抽样精度（从而保证样本均值加减0.1个学生标准差单位的95%置信区间）。

一些样本设计的限制条件
教育领域的样本设计通常在一系列相互竞争的约束条件下准备。这些设计需要遵守既定的调查抽样理论，同时也要充分考虑到它们将要应用的财务、行政和社会政治环境。对于特定项目，“最佳”的样本设计是那种在满足项目主要目标的前提下，同时将成本、物流和程序需求限制在可管理水平的样本设计。在为SACMEQ II项目准备样本设计之前确立的主要限制条件如下。

目标人群：目标人群的定义应集中于就读于注册主流政府或非政府学校的六年级学生。此外，定义的目标人群应通过排除不超过5%的学生从理想的目标人群来构建。

偏差控制：抽样应遵循科学概率抽样的公认规则。也就是说，定义的目标人群中的成员应具有已知的非零概率被选入样本，以便通过使用适当的抽样权重（Kish，1965年）来解决因“等概率选择法”（epsem sampling）而产生的样本估计偏差的潜在可能性。

抽样误差：主要标准变量的样本估计应符合国际教育成就评估协会设定的抽样精度要求。也就是说，学生测试的抽样标准误差应与采用400名学生的简单随机样本所能达到的量相当或更小（Ross，1985年）。

应答率：每个SACMEQ国家应努力实现学生整体应答率为80%。这个数字是基于实现或超过90%的学校应答率和学校内学生的90%应答率的愿望。

行政和财务成本：每个国家选定的学校数量应认识到数据收集可用的行政和财务资源的限制。

其他限制条件：在每个选定的学校中选定的参与数据收集的学生数量应设定在最大化校内数据收集（学生阅读和数学测试）有效性的水平。

目标人群的指定
对于坦桑尼亚，希望的目标人群是所有在学年第九个月（即2000年12月第一周）注册的六年级学生。决定排除特殊学校的学生和在六年级学生少于20人的学校的学生，这导致了定义的目标人群的建立。

坦桑尼亚有10786所学校，有529296名六年级学生。排除的人口为来自1270所学校的17942名学生，占所有学生的3.3%。从其中抽取样本的指定人口由来自9516所学校的511354名学生组成。

注：样本设计、样本选择和样本评估程序的详细描述已在“坦桑尼亚工作报告”中提供。

数据收集方式
---------------------------
面对面 [f2f]

研究工具
---------------------------
SACMEQ的初始项目数据收集于1995年10月，包括向学生、教师和校长发放问卷。
学生问卷包含有关学生家庭背景和学校生活的问题；教师问卷询问有关教室、教学实践、工作条件和教师住房；校长问卷收集有关教师、招生、建筑、设施和管理的信息。还对学生进行阅读素养测试。测试基于在完成试验测试计划后选定的项目。

数据清理操作
---------------------------
数据录入和数据清理
来自坦桑尼亚国家考试委员会（NECTA）和MOEC的六人被任命并接受培训，以使用WINDEM，这是用于SACMEQ的特殊数据录入软件包。NECTA和MOEC的计算机用于数据录入和数据清理。通过书面指示和来自IIEP工作人员的基本步骤的后续支持（主要通过互联网），该过程得以顺利进行，允许NRCs：
(i) 识别识别号码序列中的主要错误，
(ii) 在文件之间交叉检查识别号码（例如，确保所有学生都与自己的阅读和数学教师相联系），
(iii) 确保所有列在原始抽样框架上的学校都拥有有效的数据收集工具，反之亦然，
(iv) 检查“野码”，当某些变量的值低于预先指定的合理范围时，并验证在后续文件合并中用作链接设备的变量是可用和准确的。

尽管软件的用户友好性很高，但需要录入和清理的代码表中的信息量是巨大的，因此需要毅力和键盘操作的技能。以下可以帮助想象录入的工作量。数据收集工具包含以下信息以进行编码：学校表格：58；学生姓名表格：51；学生问卷：150；学生阅读测试：85；学生数学测试：65；教师问卷：587；教师阅读测试：51；教师数学测试：43；以及校长问卷：319。所有录入的数据都发送到IIEP进行检查，以确保没有错误，如不一致或野值。然后，IIEP将数据发送回坦桑尼亚进行清理，之后教育部将其发送回IIEP进行进一步检查。这个过程一直持续到数据绝对干净，共花费了21个月（2001年3月至2002年11月）。

应答率
---------------------------
学生和学校的应答率分别为77%和98%。

抽样误差估计
---------------------------
SACMEQ项目中采用的样本设计与通常的“教科书模型”简单随机抽样有很大不同。这种偏离要求在计算“抽样误差”（即样本估计的总体特征的稳定性度量）时采取特殊步骤。