Effective Assessment and
Progress Monitoring
in an Online Environment:

Executive Summary

This study sought to understand in more detail how, during the COVID-19 pandemic, teachers and school leaders in low and middle-income countries approached assessing student progress when students were learning remotely.

The study was undertaken in six countries – Bangladesh, India, Nigeria, Pakistan, Philippines, and South Africa. The main method of data collection was focus group discussions with two held in each country. One focus group discussion involved teachers and the other school leaders. A two-step panelist screening process was implemented, which also provided quantitative data for analysis (see page 39 for more details).

It was evident from the focus group discussions that teachers had faced numerous challenges and obstacles in seeking to maintain learning provision for their students during the pandemic. The issue of how best to assess progress appeared a secondary issue to the key priority of finding ways to reach as many students as possible and then engage them in learning effectively.

Approach to learning

Teacher approach to assessment was heavily interlinked with student access to digital devices and connectivity. Where the infrastructure allowed, many teachers frequently utilised EdTech tools for assessment. Free to use multiple-choice quizzing tools such as Google Forms were widely used and comparatively easy to deploy. Many teachers cited EdTech assessment tools as having a positive impact on student engagement (see page 40 for more details).

Many participants explained how they had to use asynchronous learning approaches, as many students had to share access to devices with their parents and/or siblings. In these instances, teachers often sent resources on WhatsApp or sent links to a third-party quizzing tool. A hybrid approach to assessment was needed when students did not have access to a device. Other students’ parents in their community were relied upon to help students submit their work. It was common for students without access to send pictures of their written answers via a friend/friend’s parents’ phone to the teacher. In instances where a digital-first approach was not possible, schools distributed hard copy resources (typically workbooks) via parental collection or by visiting house to house to distribute the worksheets. Teachers often resorted to seeking one to one contact, either via phone, recorded voice messages, or as a last resort via physical visits. Overwhelmingly, teachers fed back this one-to-one approach was ineffective as it was inefficient and did not provide them with particularly practical information.

For those that could not be reached easily by other means, broadcast radio and TV provided some learning material, but participants in the focus groups were sceptical as to student engagement in the learning and struggled to find an effective means to monitor their learning progress. Governments, broadcasters, and education resource providers should consider how they can include interactive elements into broadcast education content to ensure greater student engagement. Scheduled broadcasts could be published with supporting information regarding the lesson topics and learning objectives covered, as well as interactive tasks, potentially via a partner app. 

Data costs

A complicating factor for both students and teachers was the issue of mobile data costs. The study found that the issue of the cost of data restricts how students can learn and limits how teachers are able to deliver lessons and progress monitor. For example, in many low resource settings, teachers relied on recorded WhatsApp voice messages, asking simple comprehension questions to the student to seek to assess their progress as this form of delivery was most efficient in terms of data costs and suited for asynchronous learning. Governments, NGOs and Telecommunication providers need to be more proactive and creative in seeking to solve issues surrounding data costs for education and consider specific packages for the education system including specifically for teachers based on a future workforce skills agenda.

Approach to assessment

Adapting assessment techniques to remote learning resulted in a range of new challenges emerging. A common theme was concern from teachers regarding the authenticity of assessments, with concern students were sharing answers in WhatsApp groups or searching for answers online. Some participants used features in EdTech tools to mitigate this such as shuffling questions and using timed assessments, but suspicions remained. For education systems that have a culture of high stakes assessment focus, the issue of assessment authenticity is a critical one and EdTech providers should ensure, relevant features to mitigate are offered and well communicated.

As teachers sought to ensure learning continuity via whatever means possible, and by extension utilise a range of approaches to monitor their students’ progress, teachers have deployed extremely inefficient workflows, creating huge workload, stress, and often direct financial costs for themselves. Opportunities exist for new EdTech solutions to combine the low cost/free and low data consumption benefits of tools such as WhatsApp, with a more streamlined approach to communication and learning resource management. Some teachers cited the fact that their personal phones quickly ran out of storage after receiving hundreds of messages and photos of work from students. EdTech solutions, could seek to alleviate this pressure, either by using cloud storage solutions or alternatively using file storage tools that would allow teachers to delete files from their phone after they’ve been simply saved in a safe location elsewhere. In addition, governments and system leaders can undertake simple steps to support teacher workload challenges and reduce the friction around EdTech uptake for example by developing centralised resource libraries of useful EdTech tools (see Appendix B for details of the tools mentioned by participants) and creating central curated platforms where teachers can share digital lesson plans and resources.

During the focus groups discussions teachers and school leaders described how they had experimented with different forms of distance learning. It was curious and concerning, therefore, that there was much less experimentation with effective progress monitoring pedagogy. A combination of factors impacted this, for example the extreme workload faced and the fact that tools offering multiple choice questions were widely available, with preprepared question banks and these tools were typically easy to use, and critically free. The issue of data costs, also restricted format innovation for many. In addition, a lack of dedicated professional development regarding assessment pedagogy took place.  Participant experiences with professional development were very mixed, but where it existed professional development tended to focus on practical usage tips on technology, not reviewing different approaches to assessment. Governments, researchers, system leaders and school leaders, should review if sufficient resources, tools, and professional development support exist to support teachers in developing a range of assessments. For example, testing concept mastery and understanding, as well as knowledge and are they all suited to both online and offline environments?

For governments and school leaders to identify and seek to address learning gaps that have developed, structured data collection systems are required and could provide significant benefits. The focus groups however, highlighted a widespread lack of school or departmental level strategy and activity regarding how to use and store data/learnings gathered from online formative assessments. A small number of individual teachers provided anecdotes on how they collected, stored and analysed student data, typically to support with instructional differentiation efforts and remediation identification. Opportunities exist for data aggregation and analysis solutions that support data collection efforts across multiple apps, helping to develop feedback on student learning progress. In addition, governments, should consider dedicated data standards legislation to ensure governance suited to the specific requirements of the education sector. 

The study identified, that whilst EdTech tools have proven popular with teachers as means of developing student engagement and conducting formative assessment, governments and schools in LMICs face many urgent challenges to ensure effective and consistent progress monitoring is possible whilst learning online. If governments are to address learning gaps that have widened during the pandemic, then ensuring all teachers are able to utilise effective techniques for progress monitoring in an online learning environment will be key.