EMIS, drop-in and dropout in Eritrea

Abstract

Purpose

Many African countries grapple with school dropout, partly due to weak mechanisms for retaining children in schools, inadequate investment in education or concerns about the relevance of education to the current and future needs of children and their countries. Education Management Information Systems (EMIS) are expected to inform planning and decision-making processes for education service delivery to alleviate such problems. Yet the role and relevance of EMIS remain contested even in the best of circumstances. This paper explores the relationship among dropout, drop-in and EMIS in Eritrea, noting challenges confronting EMIS and arguing for increased focus on drop-in in EMIS.

Design/methodology/approach

The overarching methodology was influenced by interpretivism and constructionism. This research explored the following questions: What is the status of risk of dropout among Middle school students in Eritrea? What else can the Eritrean EMIS identify and measure? The first author conducted a descriptive analysis of existing quantitative EMIS data and interview data with students, teachers and government officials.

Findings

The key finding was the significant risk of dropout among middle school students in Eritrea. Secondly, whereas EMIS documented dropout rates annually, the system was not adept at identifying, documenting and analysing risks among students who drop in. There were tensions among diverse stakeholders about EMIS ownership, validity and reliability of the data and its management.

Research limitations/implications

The research was limited by inability to access raw EMIS data from the Ministry or from schools.

Practical implications

Nonetheless, the research highlighted the dilemma of using existing EMIS data for reporting on dropout and drop-in and recommended updating Eritrea’s outdated EMIS tools that were developed in 1993.

Originality/value

Relatively little research has focused on Eritrea’s education system. Although EMIS is widely considered a standard tool, this research found it problematic regarding its influence to improve educational access.

Introduction

This paper seeks to demonstrate the relationship between EMIS, drop-in and dropout. It is based on the findings of the doctoral research conducted by Emmanuel Kamuli in two middle schools in Eritrea: Beles and N’Hafash (both pseudonyms). It presents a statistical layperson’s engagement with the country’s Education Management Information System (EMIS) and shares the dilemmas of working with national information systems that are prone to competing tensions. Whereas the public may assume that the compilation of EMIS is a straightforward exercise that supports objective decision making, the reality is that EMIS is a negotiated product that accommodates sometimes contradictory imperatives.

In educational planning, EMIS is expected to play a critical role in illuminating what goes on in the education space of a country. However, as we argue in this paper, that expectation is mostly a theoretical ideal. Key stakeholders at the school and community levels, and those with limited statistical expertise in Ministries, NGOs and development agencies are either excluded or play peripheral roles and the outcomes that form EMIS are not entirely free of biases or influence.

The paper is organised as follows. First, it provides a brief overview of EMIS, then it examines the risks contributing to dropout and why they have EMIS relevance. Section two defines risk in educational contexts and the concept of drop-in as applied in this research and relate it later to EMIS. Section three presents the methodology, followed by section four on findings on EMIS data analysis and discussion in relation to drop-in and dropout. Section five concludes the paper.

Education management information systems

Countries maintain databases on different aspects of education, ranging from enrolment, teachers, classrooms and scholastic materials to highly sophisticated systems encompassing a wide array of aspects of education (Abdul-Hamid, 2017; Trucano, 2006). The databases are generically called education management information systems (EMIS). Whereas there is no standard definition of EMIS or a shared vision of what EMIS should entail (Cassidy, 2006), UNESCO presents EMIS as “an organized group of information and documentation service that collects, stores, processes, analyses and disseminates information for educational planning and management” (UNESCO, 2012, p. 5). Powell (2006, p. 6) concurs that EMIS is an information centre that is “responsible for the use of information for policy, planning and implementation, decision making, monitoring and evaluation of the education system”.

According to Villanueva, “The main purpose of an EMIS is to integrate information related to the management of educational activities, and to make it available in comprehensive yet succinct ways to a variety of users” (UNESCO, 2012, p. 5). The users range from community and school level stakeholders like students, teachers and parents, to national level policy makers and global stakeholders like the Global Monitoring Report. However, not all users possess the necessary expertise to engage with EMIS (Villanueva, 2003; UNESCO, 2012). Similarly, while The World Bank (2011) argues that the value of EMIS lies in its capacity to empower decision makers to make smart spending decisions that contribute to learning, EMIS can also be manipulated to disempower objective decision making (Carr-Hill, Hopkins, Riddell, & Lintott, 1999). Makwati, Audinos, and Laurez (2003) also caution that the real value of EMIS depends on the quality of statistics.

Arguably, a good EMIS can and should “fuel progress toward … improved student learning, increased equity, and stronger accountability relationships among policymakers, school administrators, teachers, parents, and students” (Custer, King, Manuelyan, Read, & Sethi, 2018, p. 4). However, the diverse contexts of EMIS are neither homogenous (Mainguet & Baye, 2006) nor unambiguous (Lewin, 2007). In low-income countries, a range of constraints challenge the development and management of EMIS (UNESCO, 2012). These include resource limitations, quality, limited capacity across all levels of stakeholders, limited accessibility and negative user perceptions. Moreover, as pointed out by Villanueva, “the reality is that in most developing nations serious limitations exist in terms of the availability, quality and costs of the range of practical and affordable technologies” (Villanueva, 2003, p. 35).

Concerns about EMIS persist. Apart from the seven ways highlighted by Carr-Hill et al. (1999) in which data can be manipulated to influence public sector managerial behaviour persist (“tunnel visioning”; sub-optimization; concentration on short term issues; anxieties about being exposed as an outlier; disinclination to experiment; altering behaviour and misrepresentation or outright fraud), UNESCO (2012) also reported that “Some Ministers of Education know that data collection does not function properly and thus they do not trust it. Thus, as Custer et.al. noted, there is little indication that decision-makers are systematically using education data and analysis to inform their policies or decisions” (Custer et al., 2018, p. 9). The limitations beg the question whether EMIS in poor countries supports the pursuit of learning outcomes or whether EMIS inadvertently contributes to the learning crisis.

This paper seeks to demonstrate the relationship between EMIS, drop-in and dropout. It argues that paying equal attention to drop-in and dropout in EMIS will improve accountability for students in education systems.

Defining risk and drop-in

Risk: There is considerable literature on risks confronting children in and out of school. Studies by individuals (Ananga, 2011; Palme, 1998), research teams (Hammond, Linton, Smink, & Drew, 2007; Ngware, Oketch, Ezeh, & Mudege, 2009), consortia such as RECOUP and CREATE (Reddy & Sinha, 2010) and global organisations (UNESCO, UNICEF, the World Bank, FCDO and others) identify various risks. The manifestation of risks and their effects differ (Hammond et al., 2007). Some contexts aggravate risks while others mitigate them through social protection and policy provisions (Fine, 1986; Reddy & Sinha, 2010; Unterhalter et al., 2014).

Hammond et al. (2007) identified and categorised the risk factors into individual, family, school and community domains. That categorisation is replicated in similar researches, notably the Out-of-School Children Initiative by the United Nations Children’s Fund (UNICEF and UNESCO (2015). The five dimensions of exclusion identified in OOSCI and the categories of risk by Rumberger and Lim (2008) referenced the work of Hammond et al. (2007).

The meta-analysis by Rumberger and Lim (2008) characterised risk factors into individual and institutional categories thus clustering Hammond et.al.’s (2007) three domains of family, school and community under “institutional factors”. Whereas most of the literature on risk is from high-income country contexts, there is growing literature based in low-income countries (Ananga, 2011; Reddy & Sinha, 2010; Abuya, Oketch, Mutisya, Ngware, & Ciera, 2012; Abuya, Mutisya, Onsomu, Ngware, & Oketch, 2019; Nyambedha & Aagaard-Hansen, 2010) whose themes and categories resonate with those identified in high-income country researches. Thus, existing literature provides evidence of the capacity for analytical frameworks to travel across spatial and sociocultural boundaries in high-income and low-income contexts.

Drop-in: Globally, and especially in low-income countries of sub-Saharan Africa, the phenomenon of children who enrol but fail to complete a given cycle of education continues to challenge their right to education. Some scholars have attempted to explain the causes and impacts of dropout (Ackers and Nzomo, 2001; Colclough & Lewin, 1993; Hammond et al., 2007; Hunt, 2008; Lewin, 2007). There is growing recognition that dropout is not an event but a protracted process (Álvares & Estêvão, 2014; Hunt, 2008). However, there is also a nascent recognition that some students who may have dropped out return to school (Hunt, 2008) and that students who are at risk of dropping out of school hang in and eventually complete a given education cycle (Warrington, Kiragu, & Fentiman, 2012). Álvares and Estêvão (2014, p. 30) describe the “abandonment-permanency” continuum whereby students who are in school may be in the critical phases of the multi-stage process of dropping out of school.

The OOSC Initiative, which aimed to “make a significant and sustainable reduction in the number of children who are out of school” (UNICEF & UNESCO, 2015, p. 36), identified five dimensions of exclusion (5DE) as central to the understanding of the phenomenon of out of school children (Figure 1).

Figure 1

The five dimensions of exclusion (5DE) extracted from the out of school children’s initiative by UNICEF and UNESCO (2015)

The 5DE model presents a modified lens for characterising children in and out-of-school from the perspective of exclusion. Thus, the model enriches knowledge about the protracted process of dropout.

Methodological approach and limitations

Originality: Relatively little research has focused on Eritrea’s education system. Although EMIS is widely considered a standard tool, this research found it problematic regarding its influence to improve educational access.

Approach: The study followed a two-step sequential process of a descriptive analysis of secondary quantitative EMIS data from twelve annual reports of the Eritrea Ministry of Education 2003–2015 to analyse dropout trends. The research explored the following questions:

RQ1.What is the status of risk of dropout among Middle school students in Eritrea? What else can EMIS capture?

Analysed EMIS data for 2003–2015 showed that Amiche district in the Northern Region consistently had the weakest indicators of wastage, survival and transition rates. Therefore, Amiche district was purposively sampled as a case study. The research based on existing literature to develop research tools to identify individual, family, community and school factors that contributed to dropout and drop-in. Twenty-eight participants were purposively sampled to participate in a total of 13 interviews and four focus group discussions. They included six teachers who were directly involved in the pastoral care for at-risk students and 21 students identified by schools as at-risk. A ministry official participated by virtue of office.

Research limitations

The research was limited by inability to access raw EMIS data from the Ministry or from schools. Resorting to secondary data meant no researcher control over what went into the official analysis or its output.

Whereas EMIS data were to be analysed using the (UNICEF & UNESCO, 2015) Out of School Children Initiative Framework, qualitative data analysis was done through transcription, coding and derivation of analytic memos whose emerging themes were related to theoretical concepts in existing literature (Fine, 1986; Schoon, 2006; Hammond et al., 2007; Rumberger & Lim, 2008; Ngware et al., 2009; Reddy & Sinha, 2010; Unterhalter et al., 2014). Data were triangulated within and across four different categories (individual, family, school, school community). The data were further triangulated with official EMIS data and the Eritrean demographic survey reports on education (EDHS, 2002; EPHS, 2010). At all times the research adhered to the ethical guidelines of the British Educational Research Association (2011) and UNICEF’s Procedure for Ethical Standards in Research, Evaluation, Data Collection and Analysis (UNICEF, 2021).

The EMIS questionnaires of the Eritrean Ministry of Education (MOE) collected data annually on the causes of dropout. Questionnaire Number 9 (Q9) had a preselected list of 12 risk factors plus an “other” category, which schools could use to report on causes that did not fall within the list of 12 (Figure 2).

Figure 2

Pre-selected risk factors (Figure extracted by authors from Eritrea EMIS Questionnaire 9, 2014/15)

However, it was not possible for schools to explain what the “other” causes were. The quantitative data were analysed by level of education (elementary, middle school, etc.), teaching staff and internal efficiency of the system. The section on internal efficiency provided data on flow rates. The data on promotion, dropout and repetition for 12 academic years 2003/4 to 2014/15 (Table 1) were consolidated and analysed to develop an understanding of their link to the risk of dropout (Mizunoya & Zaw, 2017; UNESCO, 2012) and their likely contribution to the learning crisis.

Table 1

Combined flow rates (percentage) for middle school students in Government of the State of Eritrea (2003/4–2014/15)

Year	Grade	Dropout (percent)			Repetition (percent)			Promoted (percent)
		Boys	Girls	Total	Boys	Girls	Total	Boys	Girls	Total
2003–4	6	7.3	6.5	7	17.7	16.8	17.3	75	76.6	75.7
2004–5	6	8.3	6.6	7.7	17.9	14.8	16.7	73.7	78.5	75.6
2005–6	6	16.5	8	13.1	17.4	15.8	16.8	66.1	76.2	70.1
2006–7	6	8.4	5.3	7.2	19.7	18.1	19.1	71.9	76.6	73.8
2007–8	6	7.9	4.9	6.6	21.2	18.7	20.2	70.9	76.4	73.2
2008–9	6	10.1	5.1	8	14.1	11.3	12.9	75.8	83.6	79.1
2009–10	6	7.7	4	6.1	16.1	12.1	14.4	76.2	83.9	79.6
2010–11	6	6.98	4.4	5.87	15.84	11.9	14.14	77.18	83.71	79.99
2011–12	6	7.6	4.8	6.4	19.5	14.3	17.3	72.9	80.9	76.3
2012–13	6	7.5	4.1	6.1	19.3	13	16.6	73.2	82.9	77.3
2013–14	6	4.8	10.1	7.8	14.1	20.1	17.5	81.1	69.8	74.6
2014–15	6	9.6	4.8	7.5	22.2	14.6	18.9	68.2	80.6	73.6
2003–4	7	6.5	5.9	6.3	12.9	10.3	11.8	80.6	83.8	81.9
2004–5	7	8.7	7	8	21.3	15	18.8	70	78	73.2
2005–6	7	20.3	8.7	15.8	12.8	11.3	12.2	66.9	80	71.9
2006–7	7	7.6	5.2	6.6	15.5	13.6	14.8	76.9	81.2	78.7
2007–8	7	6.3	4.8	5.7	12.9	10.6	12	80.8	84.5	82.3
2008–9	7	9.4	5.1	7.6	9.8	7.6	8.9	80.8	87.3	83.6
2009–10	7	6.6	4	5.4	10.5	8	9.4	82.9	88	85.2
2010–11	7	6.31	3.9	5.24	9.96	7.36	8.8	83.73	88.74	85.97
2011–12	7	7	4.3	5.8	12.4	9.6	11.2	80.7	86.2	83.1
2012–13	7	7	3.8	5.6	11.6	7.4	9.8	81.4	88.8	84.6
2013–14	7	4.9	10.5	8.1	8.9	13.5	11.5	86.1	76	80.5
2014–15	7	9.1	4.4	7	15.3	8.8	12.3	75.6	86.7	80.7
2003–4	8	7.9	6.4	7.3	16.9	10.7	14.5	75.2	82.8	78.1
2004–5	8	10.2	7.4	9.1	19.9	11.2	16.5	69.9	81.4	74.4
2005–6	8	27.3	11.1	20.4	5.8	5.1	5.5	66.9	83.8	74.1
2006–7	8	7	7.8	7.4	11.1	9.4	10.3	81.9	82.7	82.3
2007–8	8	5.6	6.2	5.9	12	9	10.7	82.4	84.8	83.4
2008–9	8	9.5	6.6	8.2	10.2	6.4	8.6	80.3	87	83.2
2009–10	8	6.9	4.2	5.7	10.7	6.3	8.7	82.4	89.5	85.6
2010–11	8	5.64	4.82	5.27	11.35	6.83	9.29	83.02	88.35	85.45
2011–12	8	7.1	5.6	6.4	15.5	9	12.4	77.4	85.5	81.2
2012–13	8	7	5.4	6.2	18.9	10.6	15.1	74.1	84	78.7
2013–14	8	7.1	12.2	9.8	11.9	20.8	16.6	81	67	73.5
2014–15	8	9.7	6.1	7.9	20.8	10.6	15.9	69.5	83.3	76.2

Source(s): MOE EMIS, Asmara (with authors’ italic)

Compiling the totals for each year yielded summary data on dropout, repetition and promotion, as shown in Table 2.

Table 2

Eritrea – middle school annual flow rates

Year	Dropout (percent)			Repetition (percent)			Promotion (percent)
	Boys	Girls	Total	Boys	Girls	Total	Boys	Girls	Total
2003–4	7.2	6.3	6.8	15.9	13.1	14.8	76.9	80.6	78.4
2004–5	9	7	8.2	19.6	13.8	17.3	71.4	79.2	74.5
2005–6	20.7	9.1	16.1	12.7	11.2	12.1	66.6	79.7	71.8
2006–7	7.8	6.1	7.1	15.9	14	15	76.4	80	77.9
2007–8	6.7	5.3	6.1	16	13.3	14.9	77.3	81.4	79
2008–9	9.7	5.5	7.9	11.6	8.6	10.4	78.7	85.8	81.7
2009–10	7.1	4.1	5.7	12.7	9	11.1	80.2	86.9	83.2
2010–11	6.3	4.4	5.5	12.6	8.8	10.9	81.1	86.9	83.7
2011–12	7.2	4.9	6.2	15.9	10.8	13.6	76.9	84.3	80.2
2012–13	7.2	4.5	6	16.9	10.4	14	75.9	85	80
2013–14	5.7	10.9	8.6	11.8	18.4	15.5	82.5	70.6	75.9
2014–15	9.5	5.1	7.5	19.7	11.6	16	70.8	83.3	76.5

Source(s): MOE EMIS, Asmara (with authors’ italic)

From the analysis, it was evident that dropout and repetition were proxy indicators of risk. There was no evidence in EMIS reports that the MOE considered or analysed the 12 and “other” factors collected using Q9.

Dropout statistics

Within the period of 2003–2014, Eritrea experienced the highest dropout rate of 16.1% at middle school level in 2005/06 (see Table 2).

Disaggregated data showed that boys were the most vulnerable, given 20.7% dropped out during the 2005/06 academic year compared to 9.1% girls.

While this information helped in establishing the extent of the problem, it left information gaps on what factors led to dropout, particularly in years like 2005/06 and 2013/14 where spikes were experienced (i.e. 16.1% and 8.6% dropouts respectively) or why some years had favourable rates. The data did not explain why particular grades were more prone to dropout than others (e.g. 20.4% of grade 8 students dropped out in 2005/6 compared to 13.1% of grade 6 students in the same year). Neither did the data explain the significant differences among different sexes, including whether there were any specific strategies that enabled students to drop-in and persist with schooling.

Repetition of grades

All the Eritrea EMIS reports cited grade repetition as a major contributor to wastage. Consistent with UNESCO (2012), any Eritrean student who was not promoted to the next grade or did not complete an educational programme and remained in the same grade the following school year was called a repeater. UNESCO (2012) notes that often, repetition results from academic failure, unsatisfactory progress, insufficient examination marks to advance to the next level of instruction, age of a student, poor attendance or lack of local educational opportunities. Much as repetition is sometimes considered to be a pragmatic solution if learning objectives for that grade level are not achieved (UNESCO, 2012), it is nevertheless globally acknowledged as a sign of system wastage. From analysing the Eritrea EMIS data, it was noted that in all the 12 years, repetition rates at Middle school level in Eritrea were above 10%. As Table 2 shows, the worst years in descending order were 2004/5 (17.3%), 2014/15 (16%), 2013/14 (15.5%) and 2006/7 (15%). The years 2007/8 and 2003/4 had close to 15%.

As with dropout, boys were more likely than girls to repeat grades. In three out of the 12 years analysed, girls’ repetition rates fell below 10%; the highest ever recorded was 18.4% in 2013/14. For boys, the lowest ever recorded was 11.6% in 2008/9 and the highest was 19.7% in 2014/15. Much as the research did not deliberately apply a gender lens when analysing the data, it was evident that gender mattered. Overall, average repetition rates across the 12 years were 11.8% (12.8% for males and 10.3% for females). This was significantly higher than the sub-Saharan regional trends for 2000, 2005 and 2010 reported by UNESCO (2012), which were consistently below 6%, making Eritrea’s repetition situation remarkable within the sub-Sahara region.

Despite the stipulation of official policy that grade repetition “is a prime concern of the MOE [and] that students should not and need not repeat classes” (MOE, 2003, p. 10), EMIS reported on the phenomenon regularly without providing any insights or qualitative explanations as to why students were repeating grades and why certain years returned significantly higher repetition rates than others. There was no reference to the MOE policy (2003) regarding repeating. Arguably, repetition was contributing to the learning crisis.

Other risks of dropout

In its questionnaire, the Eritrean EMIS acknowledged that there were a range of causes of dropout from schooling.

An examination of Q9 templates across the years noted that ever since the MOE developed the list of 12 causes in 1993 (Figure 2), it had not revised them by 2015. There was no analytical report on the 12 items.

Findings on EMIS data analysis and discussion

The named first author of this paper was positioned as an insider/outsider and this vantage location complemented the processes of data collection and analysis (Day, 1998; Mullings, 1999). Similarly, the researcher’s professional experiences informed the entire research process (Bryman, 2012; Gray, 2014). The insider status enabled access to vital information while carrying out duties at UNICEF, through meetings, correspondences, anecdotal observation and fugitive literature (Dowling & Brown, 2010). For example, while attending an international meeting on EMIS (outside Eritrea), the researcher witnessed an animated discussion between high-ranking Eritrean MOE officials and UNESCO representatives regarding the data provided by the United Nations Population Division (UNPD). The MOE delegation protested that UNPD was publishing inflated population estimates for Eritrea which led to negative exaggerations in education indicators. This exchange roused curiosity and led to a comparison of the OOSC reports by UNESCO Institute of Statistics (UIS) with OOSC data from Eritrea EMIS. Table 3 was the result of that inquiry:

Table 3

Annual middle school dropout rates 2003/4–2014/4

Year	Dropout
	Boys	Girls	Total
2003–4	7.2	6.3	6.8
2004–5	9	7	8.2
2005–6	20.7	9.1	16.1
2006–7	7.8	6.1	7.1
2007–8	6.7	5.3	6.1
2008–9	9.7	5.5	7.9
2009–10	7.1	4.1	5.7
2010–11	6.3	4.4	5.5
2011–12	7.2	4.9	6.2
2012–13	7.2	4.5	6
2013–14	5.7	10.9	8.6
2014–15	9.5	5.1	7.5

Source(s): MOE EMIS, Asmara

What emerged was that the UIS data on OOSC grew exponentially while the EMIS data shrank for progressive years. Secondly, UIS figures were more than twice the figures of MOE. If indeed the population of Eritrea was about three million, then every year at least 10% of the population was out of school. However, by 2016 UIS had ceased reporting on Eritrea altogether. Nevertheless, it is conceded that the insider/outsider status challenged the researcher’s ability to remain objective and maintain theoretical sensitivity (Oktay, 2012).

Three important issues emerged about EMIS and we explore them further below.

Inconsistent population figures

The first issue was inconsistency in population figures. Carr-Hill (2012) argues that reliable census data is one of the pillars of a sound EMIS. However, he notes that in low-income countries, vital demographic registration systems are not fully functional. Donors and multilateral agencies have a stake in national data systems (UNESCO, 2012; Carr-Hill, 2012). And indeed, the international community had an interest in determining the population of Eritrea but also contributed to the lack of consensus about the population estimate. The population of Eritrea was critical to this research, since the population figure is vital in computing education indicators.

Broeck, Argeseanu Cunningham, Eeckels, and Herbst (2005) highlight challenges like strange patterns, inconsistencies or sheer implausibility that can impact the quality of data. For Eritrea, available records indicated that between 2001 and 2009 Eritrea’s population grew from 3.2 million to 3.9 million and then in 2010 it reverted to 3.2 million (Table 4). Population estimates cited in the Eritrea EMIS publications indicated an average annual increase of 110,000 persons between 2000 and 2009, and then a sudden decline of over 700,000 persons in 2010. The researchers considered this to be an illustration of a strange pattern (Broeck et al., 2005).

Table 4

Annual repetition rates for middle school level

Year	Repetition
	Boys	Girls	Total
2004–5	19.6	13.8	17.3
2014–15	19.7	11.6	16
2013–14	11.8	18.4	15.5
2006–7	15.9	14	15
2007–8	16	13.3	14.9
2003–4	15.9	13.1	14.8
2012–13	16.9	10.4	14
2011–12	15.9	10.8	13.6
2005–6	12.7	11.2	12.1
2009–10	12.7	9	11.1
2010–11	12.6	8.8	10.9
2008–9	11.6	8.6	10.4

Source(s): MOE EMIS, Asmara (with authors’ italic)

Notably, the population estimate by the NSO was the same for 2001 and 2010. An analysis of the annual EMIS reports revealed that starting with the 2010/2011 academic year the MOE discontinued the practice of giving population figures as the basis for EMIS calculations. The EMIS report of 2011 did not explain the discontinuation. However, from Table 46 of the annual EMIS booklet (2010/11), which gives detailed information on age-specific enrolment, it was possible to prorate the school aged population of 2,155,951 to arrive at a population figure of 3,142,228 for that year. This was evidence of an inconsistency as described by Broeck et al. (2005). The change represented a drastic fall in the population estimate from that of 3,984,162 in the previous academic year.

The declining population estimates had attracted the interest of Woldemicael (2005, 2008), an Eritrean scholar who hypothesised that the declines were indicative of long-term fertility transition or prolonged spacing of births, cessation of further childbearing, delayed age at marriage and possibly the effects of conflicts and economic hardships. Woldemicael’s studies raised concerns about the estimations of Eritrea’s population. However, the dilemma of unstable population estimates was not unique to the MOE. A scan of other sources indicated that there were diverse “credible” sources of Eritrean population estimates and that this was partly due to the absence of a national census (Randall, 2015) given that by 2016 Eritrea had not conducted any national population census since 1938 according to the European Asylum Support Office (EASO, 2015) or since 1984 when Eritrea was still part of Ethiopia, according to the Ethiopia Population and Housing Census Commission (1991). Thus, for all her life as an independent country, Eritrea had never conducted a population census. For the EMIS the NSO provided national population estimates. Figure 3 shows the discrepancies in population estimates for 2011 from various “authoritative sources” on Eritrea:

Figure 3

Different population estimates by source. Compiled by the authors

What emerged from the analysis was that there was no single reliable population estimate for Eritrea. Triangulation of figures from the various sources confirmed the absence of a reliable estimate.

This was not an indictment on Eritrea’s EMIS but rather an exemplification of what Broeck et al. (2005) identified as the idiopathic anomalies relating to population data which is used as a basis for key decisions and studies.

Observed mismatches for OOSC trends

The second issue about EMIS was mismatched trends. Reviewing the EMIS data revealed a change in the historical figures for out-of-school children (OOSC). As illustrated in Figures 4 and 5 which were extracted from Tables 15 of the 2003/04 and 2010/11 EMIS reports, there was a significant change in the values of OOSC. The change affected the period from 1991 to 2010. Thus, whereas the 2003/04 report indicated that there were 250,252 children aged 7–11 who were out of school, the 2010/11 report reduced the figure for 2003/04 by 45% to 112,785 OOSC. The 2010/11 EMIS report did not provide the rationale for revising historical OOSC rates.

Figure 4

Extract from EMIS report showing original flow rates (with authors’ highlight)

Figure 5

Extract from EMIS report showing adjusted flow rates (with authors’ highlight)

Mismatch of age of children enrolled in school

The third issue was over-enrolment. The EMIS publication of 2012/13 showed that 65,916 eight-year-old children had enrolled into grade two, which was greater than the estimated total population of 65,042 eight-year-old children in the country (Figure 6).

Figure 6

Extract from EMIS report showing age 8 enrolment anomalies (authors’ highlight)

There had not been any reported influx of refugees. Barring the possibility that invisible children (Carr-Hill, 2012; Randall, 2015) had been enrolled, the over-enrolment against the estimated total population was logically and biologically implausible (Broeck et al., 2005).

Yet, the abiding sense was that the three discrepancies were not necessarily cases of ineptitude among the processors of EMIS data. Rather, they presented a dilemma to the research given the data were from secondary sources. Broeck et al. (2005) suggest that screening, diagnosing and editing the data is an iterative process that can begin anywhere. Broeck et al. (2005) advise that in case of problematic data observations, the researcher must decide whether to correct them, delete them or leave them unchanged. However, since the first author did not personally collect the raw data, he decided to bring his observations to the attention of the MOE and NSO who were the original owners and custodians of EMIS, to get clarification before running the data in the OOSCI excel spreadsheet for the five dimensions of exclusion (5DE). Below are the outcomes of those consultations.

Official response about anomalies and possible remedies

The three key issues were the population figures which formed the basis for the EMIS computations by MOE; the rationale for recalculating the historical out-of-school statistics from 1991 to 2010 and the possibility that more children enrolled into Grade Two than the entire population estimate for that age group.

Inconsistent population estimates

Analysed data from interviews with MOE and NSO officials indicated that they considered information on population statistics in the country to be a highly sensitive issue. An official from NSO cautioned,

No other authority than the NSO whether in the country or elsewhere has the mandate to determine the population of Eritrea. … Partners working in Eritrea should be wary of regurgitating wrong figures purported to be from so-called reliable sources (interviewed 7 December 2015).

First author recalled the animated discussion between MOE and UNESCO officials regarding UNPD figures. However, he also concluded that the different figures cited by various departments of the government emanated from the understanding that no changes could be made to the population estimates unless such changes were communicated by the NSO. If a department did not need to publish routine data the way EMIS does with education data, then possibly they did not take the initiative to seek updates from the NSO. Further analysis of data from the interviews with the MOE EMIS officials revealed that the MOE used different mathematical models to calculate the population growth figures and this may have contributed to the higher population size.

The NSO acknowledged having asked the MOE to discontinue the practice of citing their population denominator in the EMIS publications. During the interview with an official of the NSO, first author asked whether the changes in population estimates could have been arbitrary, for example from 3.2 million to 2.9 million for 2010. The official explained, “Such changes are normal since data cleaning processes are continuous and no database is ever exact” (interviewed 7 December 2015). That explanation was consistent with literature (Broeck et al., 2005). He also explained that the NSO used the cohort component model which considered factors like fertility, mortality and migration, and concluded, “Based on the recalculations, we found that the MOE had erroneously exaggerated the out-of-school statistics and that was why in 2010 they had to adjust historical data for 1991–2009” (interviewed 7 December 2015).

Our analyses showed that the problem of different population figures was not unique to Eritrea government data sources only. United Nations agencies working in Eritrea had different figures much as they depended on the same UNPD for their respective population estimates. In a nutshell, there were different estimates of the national population of Eritrea depending on the source.

Changes in OOSC trends

The second observation was about a reduction in the OOSC population across the years. The change had resulted in a 45% reduction in the OOSC rates. The MOE explained that the change was a result of adjusting the national intake age from seven years to six years. It was not immediately clear why the adjustment stretched to 1991. In the interview discussion with the NSO, the officer explained that because the MOE had consistently used inappropriate mathematical models to calculate population growth figures, they accumulated errors due to exaggerated population size. That was why the EMIS unit had to revise their historical population “projections” and to recalculate the flow rates. He concluded, “That was why the NSO asked the Ministry to discontinue the practice of citing their population denominator in the EMIS publications” (Interviewed 7 December 2015).

Higher enrolment than entire age-cohort

In the analysis of the quantitative data the report of 2012/2013 showed that 66,245 eight-year-old children were enrolled whereas the total estimated population of eight-year-old children that year was 65,042 (Figure 6). In other words, the enrolment exceeded the entire population of eight-year-olds. Analysed data from interviews with the MOE and NSO officials revealed that they considered such anomalies to be the result of age-heaping during national population surveys when respondents round off their ages to the nearest zero or five. The NSO official said, “Here it is common for an eleven-year-old to claim to be 10 years old while a fourteen-year-old may claim to be 15” (interview on 7 December 2015). Literature confirms that age-heaping is a typical practice during population surveys and EMIS data collection processes (A’Hearn, Delfino, & Nuvolari, 2016; Lyons-Amos & Stone, 2017). Lyons-Amos and Stone (2017) report that in sub-Saharan Africa five per cent of respondents misreport their age. Indeed, the EPHS (2010, p. 12) reported, “noticeable heaping at ages ending with 0 and 5 for both sexes” and concluded that “Ages ending with 1 and 9 are underreported.” Thus, the NSO reported that they used statistical techniques to smooth the zigzags before prorating the population by age groups.

School-level efforts to capture qualitative information on students

A key finding related to the way schools documented student risks and how their innovative efforts were neither harnessed nor acknowledged by the EMIS. At both schools, the first author probed how qualitative information on risks was collected and stored and whether such school-level information fed into the annual EMIS reports.

The two schools had different ways of collecting the information. In N’Hafash the Principal had an incident book in which teachers on duty reported any concerns they noted during the week. It was not confidential. In Beles, they maintained a confidential logbook on incidents affecting students’ participation. The principal and deputy principal were the only custodians.

Curiously, the information on risks that schools gathered was not compiled and escalated to the MOE to be woven into the narrative of EMIS. Thus, EMIS reports did not benefit from school level insights, particularly how teachers and students converted their risk cases into drop-in and completion cases. That would have contributed to analysing the “other” category of Q9, the risks they faced, what actions they had taken, the challenges encountered or what lessons they had drawn from dealing with the risks to foster drop-in.

Secondly, given that EMIS data are collected once a year, the absence of a feedback mechanism denies the process a potential source of comprehensive data that would enrich analyses and conclusions of EMIS reports. A teacher at Beles asked if the researchers had experienced such a system of school level inputs to the national EMIS elsewhere in Africa. None of the researchers had seen such a practice, thus conceding that this could be a general gap with traditional EMIS systems. Our view is that teachers and school communities would benefit from MOE guidance on how to systematically capture, document, analyse, report and utilise the locally generated qualitative information on the risks of dropout and more importantly on how schools could collate their experiences of drop-in and convert them into important lessons for the education system and its multiple stakeholders.

There was a procedural omission regarding who accessed the EMIS reports once they were published. While it was laudable that the MOE published the annual EMIS reports within a year of collecting the data, circulation of the reports was limited to MOE and regional education offices. Schools never received copies of the EMIS reports whose data they helped to compile. That inadvertently excluded schools from tapping into the findings of the EMIS questionnaires. Similarly, the language in the EMIS reports is very technical, the graphs and formulae are complex and the underlying assumptions need to be simplified for the wider audience of diverse stakeholders.

Conclusion

The key finding from the foregoing procedures was to highlight the dilemma of using existing EMIS data for reporting on dropout, drop-in or even learning outcomes. We believe that wastage in the school system, directly relates to the way information on schools is gathered, processed, managed and disseminated through EMIS. Our analysis has contributed to a diagnosis on EMIS reporting especially the likelihood of underreporting on OOSC. Additionally, the research identified that capturing information on drop-in has the potential to enrich government actions and accountability for students. More broadly, we also note that EMIS remains a mystery to several levels of stakeholders who may not grasp the technical complexities, iterative steps, critical decisions and what informs the final product. Consumers of EMIS have a stake and the potential to participate in enriching EMIS.

Lastly, whereas the burden of providing accurate EMIS rests with the national government, other nongovernment players have influence on the final outcomes and they can contribute the required financial and technical resources needed to support strong and credible EMIS in sub-Saharan Africa. That way EMIS can meaningfully contribute towards mitigating the learning crisis.

Author Affiliation

References

Abdul-Hamid, H. (2017). Data for learning. Building a smart education data system. In Directions in Development. Washington, DC: World Bank Group.

Abuya, B., Oketch, M., Mutisya, M., Ngware, M., & Ciera, J. (2012). Influence of mothers’ education on children’s maths achievement in Kenya. Education 3-13: International Journal of Primary, Elementary and Early Years Education, 41(6), 633–646. doi: 10.1080/03004279.2011.631562.

Abuya, B. A., Mutisya, M., Onsomu, E. O., Ngware, M., & Oketch, M. (2019). Family structure and child educational attainment in the slums of Nairobi, Kenya. Sage Open, 9(2). doi:10.1177/2158244019855849.

Ackers, J. M. J., & Nzomo, J. (2001). Identifying and addressing the causes of declining participation rates in Kenyan primary schools. International Journal of Educational Development, 21(4), 361–374. doi: 10.1016/s0738-0593(00)00054-7, Available from: www.elsevier.com/locate/ijedudev

Álvares, M., & Estêvão, P. (2014). What do we mean by school dropout? Early school leaving and the shifting of paradigms in school dropout measurement. Portuguese Journal of Social Science., 13(1), 21–32.

Ananga, E. D. (2011). Typology of school dropout: The dimensions and dynamics of dropout in Ghana. International Journal of Educational Development, 31(4), 374–381. doi: 10.1016/j.ijedudev.2011.01.006.

A’Hearn, B., Delfino, A., & Nuvolari, A. (2016). Rethinking age-heaping: A cautionary tale from nineteenth century Italy. Oxford: University of Oxford Discussion Papers in Economic and Social History. Number 148, September 2016.

British Educational Research Association . (2011). Ethical guidelines for educational research. London: BERA.

Broeck, V. J., Argeseanu Cunningham, S., Eeckels, R., & Herbst, K. (2005). Data cleaning: Detecting, diagnosing, and editing data abnormalities. PLoS Medicine, 2(10), e267. doi: 10.1371/journal.pmed.0020267.

Bryman, A. (2012). Social research methods (4th Ed). Oxford: OUP.

Carr-Hill, R. (2012). Finding and then counting out-of-school children. Compare: A Journal of Comparative and International Education, 42(2), 187–212. doi: 10.1080/03057925.2012.652806.

Carr-Hill, R., Hopkins, M., Riddell, A., & Lintott, J. (1999). Monitoring the performance of educational programmes in developing countries. Education Research Paper No. 37, 1999 (p. 190). London: Department for International Development.

Cassidy, T. (2006). Education management information systems (EMIS) in Latin America and the Caribbean: Lessons and challenges. Washington, DC: Inter-American Development Bank Education Network.

Colclough, C., & Lewin, K. M. (1993). Educating all the children: Strategies for primary education in developing countries. Oxford: OUP.

Custer, S., King, M. E., Manuelyan, A. T., Read, L., & Sethi, T. (2018). Toward data-driven education systems: More data and more evidence use. Journal of International Cooperation in Education, 20(2/21), 5–34. doi:10.15027/50521.

Day, C. (1998). Working with the different selves of teachers: Beyond comfortable collaboration. Educational Action Research, 6(2), 255–272. doi: 10.1080/09650799800200055.

Dowling, P. C., & Brown, A. J. (2010). Doing research/reading research: Re-interrogating education (2nd ed.). London: Routledge.

EDHS. (2002). Demography and health survey. Asmara: National Statistics Office.

EPHS. (2010). Population and health survey. Eritrea population and health survey. Asmara: National Statistics Office.

European Asylum Support Office . (2015). EASO country of origin information report – Eritrea Country focus. Luxembourg: Publications Office of the European Union.

Fine, M. (1986). Why urban adolescents drop into and out of public high school. Urban Adolescents, 87(3), 393–409. doi: 10.1177/016146818608700309.

Gray, D. E. (2014). Doing research in the real world. London: SAGE Publications.

Hammond, C., Linton, D., Smink, J., & Drew, S. (2007). Dropout risk factors and exemplary programs. Technical Report. Clemson, SC: National Dropout Prevention Center, Communities in Schools.

Hunt, F. M. (2008). Dropping out from school: a cross country review of literature. University of Sussex, report. Retrieved from https://hdl.handle.net/10779/uos.23313047.v1

Lewin, K. (2007). Improving access, equity and transitions in education: Creating a research agenda. Create Pathways to Access. Research Monograph No. 1. Online Submission.

Lyons-Amos, M. and Stone, T. (2017). Trends in demographic and health survey data quality: An analysis of age heaping over time in 34 countries in Sub Saharan Africa between 1987 and 2015. BMC Research Notes, 10(1), 760. doi: 10.1186/s13104-017-3091.

Mainguet, C., & Baye, A. (2006). Defining a framework of indicators to measure the social outcomes of learning. In Proceedings of the Copenhagen symposium on measuring the effects of education on health and civic engagement. OECD.

Makwati, G., Audinos, B., & Laurez, T. (2003). The role of statistics in improving the quality of basic education in Sub‐Saharan Africa. Working Document, Association for the Development of Education in Africa. Tunis: African Development Bank.

Mizunoya, S. & Zaw, H. T. (2017). Measuring the holes of the ship: Global cost estimations of internal inefficiency in primary education. International Journal of Educational Development, 54(1), 8-17. doi: 10.1016/j.ijedudev.2017.01.006, Available from: https://www.learntechlib.org/p/195734/ (accessed 29 December 2023).

Mullings, B. (1999). Insider or outsider, both or neither: Some dilemmas of interviewing in a cross-cultural setting. Geoforum, 30(4), 337–350. doi: 10.1016/s0016-7185(99)00025-1.

Ngware, M., Oketch, M., Ezeh, A., & Mudege, N. (2009). Do household characteristics matter in schooling decisions in urban Kenya?. Equal Opportunities International, 28(7), 591–608. doi: 10.1108/02610150910996425.

Nyambedha, E. O., & Aagaard-Hansen, J. (2010). Educational consequences of orphanhood and poverty in Western Kenya. Educational Studies, 36(5), 555–567. doi:10.1080/03055691003728981.

Oktay, J. S. (2012). Grounded theory. Oxford: Oxford University Press.

Palme, M. (1998). The meaning of school repetition and dropout in the Mozambican primary school. Sociology of Education and Culture Research Reports (Uppsala University), 16.

Population and Housing Census Commission . (1991). The 1984 population and housing census of Ethiopia analytical report at national level. Addis Ababa: Transitional Government of Ethiopia.

Powell, M. (2006). Rethinking education management information systems: Lessons from and options for less developed countries. Semantic Scholar.

Randall, S. (2015). Where have all the nomads gone? Fifty years of statistical and demographic invisibilities of African mobile pastoralists. Pastoralism: Research, Policy and Practice, 5(1), 22. doi: 10.1186/s13570-015-0042-9.

Reddy, A. N., & Sinha, S. (2010). School dropouts or pushouts? Overcoming barriers for the right to education. (C. R. M. 40. Ed.). National University of Educational Planning and Administration (NUEPA) New Delhi. (CREATE research monographs: pathways to access series, PTAs, 040).

Rumberger, R. W., & Lim, S. A. (2008). Why students drop out of school: A review of 25 years of research. California Dropout Research Project Report #15. Santa Barbara: University of California. Available from: www.lmri.ucsb.edu/dropouts (accessed 19 December 2017).

Schoon, I. (2006). Risk and resilience: Adaptations in changing times (1st ed). Cambridge: Cambridge University Press.

Trucano, M. (2006). Education management information system: A short case study of Mozambique. Working Paper No. 3. Cambridge: Cambridge Education, InfoDev.

UNESCO (2012). Opportunities lost: The impact of grade repetition and early school leaving. Global Education Digest. Paris: UNESCO.

UNICEF. (2021). UNICEF procedure for ethical standards in research, evaluation, data collection and analysis. Document Number: CF/PD/DRP/2015-001. Division of Data, Research and Policy. Retrieved from https://www.unicef.org/media/54796/file

UNICEF and UNESCO (2015). Global out-of-school children initiative – operational manual. New York: UNICEF.

Unterhalter, E., North, A., Arnot, M., Lloyd, C., Moletsane, L., Murphy-Graham, E, … Saito, M. (2014). Interventions to enhance girls’ education and gender equality. In Education Rigorous Literature Review. London: Department for International Development.

Villaneuva, C. C. (2003). Education management information system (EMIS) and the formulation of Education for All (EFA) plan of action, 2002-2015. UNESCO report. Dushanbe: Tajikistan Ministry of Education.

Warrington, M., Kiragu, S., Fentiman, A., & Muhwezi, M. (2012). Factors influencing retention in Kenya and Uganda. In M. MacBeath, & M. Younger (Eds), Millennium goals revisited: a common wealth of learning (pp. 61–71). Oxford: Routledge.

Woldemicael, G. (2005). Evidence of recent fertility decline in Eritrea: An analysis of trends and determinants. MPIDR working paper 2005-026. Rostock: Max Planck Institute for Demographic Research.

Woldemicael, G. (2008). Recent fertility decline in Eritrea: Is it a conflict-led transition?. Demographic Research, 18, 27–58. Retrieved from http://www.demographic-research.org/volumes/vol18/2/18-2.pdf

World Bank . (2011). Learning for all: Investing in people’s knowledge and skills to promote development. World Bank Group education strategy 2020. Washington, DC: The World Bank.