The Potential of Assessing Children’s Learning by Phone: An Update

Schools around the world have been closed for months due to COVID-19, and while some are reopening their doors, others will remain closed for many months to come. Most countries have rolled out some version of distance learning (Figure 1), but is it working? With social distancing in place and budgets focused on fighting a pandemic, phone-based assessments offer a potential way to evaluate learning at low cost.

Figure 1. Distance learning programs by modality

Image

Source: CGD COVID-19 Education Tracker

In an earlier blog post, we proposed a few lessons from the experience of some members of our team with piloting phone-based assessments in Botswana and from other members with oral assessments from before the pandemic. Now, with more piloting experience and data, we update those lessons in an article published at BMJ Global Health: “Practical Lessons for Phone-Based Assessments of Learning.” Check out the highlights.

The potential

With new data, we compare phone-based assessments of arithmetic for more than 2,000 children in Botswana who were in grades 3-5 before schools closed with a face-to-face assessment of a similar sample of children from roughly the same schools and grades. In both assessments, children were asked to perform a basic operation (addition), and if they did well, they could advance to more complicated operations (all the way through division). As you can see in Figure 2, the phone-based assessments and the face-to-face (ASER) assessments show a broadly similar distribution of skills. This suggests that for basic skills, at least, phone-based assessments can yield useful information. The cost of the phone-based assessments in Botswana is low, about US$4.40 per child. The suggestive data on accuracy and the low cost suggest that the value of phone-based assessments may extend beyond the COVID-19 crisis.

Figure 2. The distribution of student ability as measured through phone-based and face-to-face assessment

Image

Source: Angrist et al. 2020.

What we’ve learned

We share nine lessons from our experience, ranging from the ethical (“Protect children”) to the technical (“Test the reliability and validity of your measures”) (Figure 3).

Figure 3. Nine practical lessons for phone-based assessments of learning

Image

Many of these seem obvious. But for each, we provide practical examples and references to other resources to help policymakers and researchers piloting their own phone-based assessments of learning.

Our team, and the education field in general, is still at the beginning of the road when it comes to learning how to assess students in different subjects and at different levels by phone. But these promising results, together with the uncertain duration of this crisis, suggest that it is worth learning to do this right.

This blog post was updated on July 23, 2020, to change the publication status of the research article from forthcoming to published.