Brazil case study

School-based surveys: Helping to stop Brazilian children from falling behind

Brazil has taken significant steps toward reducing infant mortality and improving the number of young children going to kindergarten and school. However, like many low- and middle-income countries, a Brazilian child’s opportunity to participate and achieve in education still depends on where they are born and their family’s socioeconomic circumstances. The early Human Capability Index (eHCI) tool developed by Telethon Kids Institute researchers is helping to build acceptance around monitoring childhood development in Brazil and demonstrate the benefit when development needs are clearly identified and children’s progress can be reliably monitored across the country.

Significant initiatives are being undertaken across Brazil to improve both opportunities and outcomes for children whose development is impacted by poverty and the lack of access to health and education facilities.

To support this, researchers and policy makers in Brazil recognised that monitoring early childhood development at a population level is an increasing priority in public policy. They want to be able to identify the circumstances of children who are not developmentally on track and to assess the impact of health and educational programs.

However, Brazil has had several attempts to introduce large-scale, school-based monitoring of kindergarten children and it was not until researchers tested the early Human Capability Index (eHCI) that they found a tool that would be widely accepted by teachers and school communities.

There were a couple of key learnings from previous attempts that paved the way for the successful use of eHCI, according to Professor Daniel Domingues dos Santos, from the Economics Department at the University of Sao Paulo, who undertook the research in Brazil. These learnings centred around carefully planning the research and engaging with stakeholders from an early stage.

“First, we brought the teachers to the table to validate the instrument with us, every step…to explain, to discuss and so on,” Prof. Santos said. “This was very important.”

“Secondly, we didn’t choose a city like Rio de Janeiro, which is politically very hard to manage. We chose a medium-sized city in Sao Paulo, to let us do the job without mixing up with the political cycle.”

In selecting an instrument to measure children’s outcomes quickly, easily and inexpensively, at scale, the Brazilian research team identified the eHCI as a quick and simple to use tool.

But would the results be reliable in a Brazilian cultural context?

First, the eHCI was translated into Portuguese and adapted for use in the culture of Brazil. The eHCI was then used for an initial study, involving around 2,200 children, to validate the tool for the Brazilian culture and to build a short, 30-item version for use at scale that would make it even quicker and easier for teachers to complete.

“The average time a teacher took to fill in a form was a little bit more than 10 minutes,” Prof. Santos said. “The typical classroom in Brazil has 25 kids, so you multiply that by 25.”

Following the initial validation tests, the eHCI followed around 240 kindergarten children for 18 months and Brazilian researchers took the opportunity to test eHCI’s performance against another previously-used tool, the Ages and Stages Questionnaire (ASQ-3).

While the eHCI is an open-source tool that is provided free of charge for use across the world, the other tool is a licensed tool that can only be used with payment.

Some 300 teachers were involved in filling out a survey of 29 questions for eHCI and 30 questions for ASQ-3. Both tools were used to assess approximately 2,500 children aged from three to five years old over an 18-month period.
eHCI came out the clear leader.

“Teachers reported (eHCI) was more related to their work, number one (and eHCI) predicted the behavioural tasks better,” Prof. Santos said.

“Almost all of the variance of ASQ-3 was captured by eHCI but the opposite wasn’t true, which means that eHCI had more independent content.”

Using data collected with the eHCI has also enabled researchers to get some sense of how Brazilian children compare with children in other countries using the eHCI.

“When we compare Brazil with China, of course our sample doesn’t represent Brazil and neither does their represent China,” Prof. Santos said.

“But the fact that at the age of three, Brazilian and Chinese children knew pretty much the same content in terms of math, and two years later, a Brazilian kid knows, according to the teacher reports, only 70 per cent of what Chinese children do. What this suggests is that very early in life, we start to fall behind let’s say.”

Professor Sally Brinkman and her team at the Telethon Kids Institute have worked with Brazil and numerous other low- and middle-income countries around the world to help them adapt and apply the eHCI.

“We were delighted the eHCI was so readily accepted by teachers in Brazil and we’ve found in other countries where it’s completed by parents and caregivers, the eHCI has also been found to be simple and easy to use – and most importantly accurate,” Prof. Brinkman said.

“The eHCI will show whether a child is thriving or doing poorly on different aspects of development and can detect developmental change over time, which is a key foundation for both developing and measuring the impact of public policies to improve childhood development.”