PISA 2018: Evidence is the key to turn education fortunes around

By Jennifer Buckingham

The earth has revolved around the sun three times since the last set of results from the OECD’s international student assessments known as PISA – Program for International Student Assessment — were released. Unfortunately, Australia has once again slipped against global standards.

A lot of focus is on the rankings of countries in these assessments. Watching the rise of smaller, previously neglected countries is fascinating. Meanwhile, Australia’s ranking has dropped each time we have participated. We started near the top of the table in the first PISA cycle in 2000 and are now closer to the middle.

Rankings are not the best or only indicator of performance. The number of countries and partner economies participating in PISA has changed each cycle, so any country’s rankings will almost inevitably change irrespective of its scores. There were 79 countries and partner economies in the 2018 PISA assessments, compared with 32 in PISA 2000. The rankings also include a number of individual provinces and city states, with which it is arguably unfair to compare a country like Australia. Furthermore, there are often only small differences between scores, which affect the rankings but are not statistically significant.

It is just as important, if not more so, to consider a country’s average performance and the distribution of scores, and how those have changed over time. In the case of Australia, it has been a significant decline.

From 2000 to 2018, mean reading literacy scores dropped from 528 to 503, the equivalent of a year’s learning.

If Australia had maintained its PISA 2000 reading literacy mean score of 528, we would be ranked third in PISA 2018 – above high performing Macau (China) and Hong Kong.

In mathematical literacy, Australia’s mean score dropped from 524 in 2003 to 491 in 2018, leaving us languishing at the OECD average for the first time. In science, the deterioration has also been significant but over an even shorter period of time, down from 527 in 2006 to 503 in 2018.

The problem is not that the entire bell curve is moving in the wrong direction. The problem is that the distribution of scores is becoming increasingly skewed to the left. Mean scores are decreasing because the proportion of low achievers has grown and the proportion of high achievers has shrunk.

In reading, the proportion of low performers grew from 12% in 2000 to 20% in 2018, while the proportion of high performers fell from 18% in 2000 to 13% in 2018. This pattern is replicated in maths and science. The top ranked combined Chinese provinces of Beijing, Shanghai, Jiangsu and Zheijang (B-S-J-Z China) had 32% of students in the high performing category.

Over the coming months and even years, the enormous amount of data yielded by the PISA assessments will be sliced and diced in multiple ways to attempt to make sense of these trends. The granular data that tabulates student performance against numerous educational, social and economic factors will follow these high level statistics and will keep education analysts and researchers occupied for hours upon hours.

One of the most consistently well-kept secrets from previous PISA reports is the strong positive relationship between teacher-directed instruction and student performance in the tests. The OECD has typically preferred to promote inquiry- or project-based approaches as part of its vision for 21^st Century schools, despite its own findings favouring teaching practices more closely aligned with explicit or direct instruction.

A growing research literature is building toward an understanding that new content and skills should be taught methodically and systematically with student inquiry best introduced after the key concepts have been mastered. Explicit teaching is particularly important for students who have socioeconomic or educational disadvantages – those most at risk of being among the low achievers. This finding is emerging across curriculum areas, yet almost every announcement of a big new investment in policy or school infrastructure seems to be pushing teaching in the opposite direction.

Can Australia turn its educational fortunes around? Yes, eventually – if policy and practice follows rigorous research evidence of what leads to better learning. The price of not doing so will be paid by our future students.

This article was published in The Australian newspaper on 5 December 2019