NAPLAN tests must be fit-for-purpose

Jennifer Buckingham

24 September 2019

NAPLAN has been in the political cross-hairs since it was introduced in 2008. It becomes big news at least four times year – when the tests are held in May, when the preliminary national results are released in September, when the full national report is published in the following March, and when the My School website is updated with the individual school results in April.

Over the past year or so, in addition to these flashpoints there has been widespread discussion about various reviews and inquiries into the NAPLAN assessments and the ways in which the results are reported.

The most recently announced review is a collaboration between the NSW, Victorian, and Queensland governments. The ACT and Queensland have conducted their own separate reviews in the past year. A review of NAPLAN reporting was also commissioned by the Council of Australian Governments (COAG).

The terms of reference for the current review include determining the objectives of standardised testing in schools and how well NAPLAN is placed to meet those objectives. These are important questions because different purposes require different types of assessments. It is essential that assessments be ‘fit-for-purpose’. No single assessment can fulfil all objectives.

The primary objective of NAPLAN is to provide consistent data on school and system performance on the fundamental, non-negotiable aspects of education – literacy and numeracy. As a secondary function, it provides information about individual student performance to parents, which can give them some externally validated reassurance that their child’s school reports are providing an accurate picture of their educational progress and level of achievement, but it cannot provide detailed information. Teachers use a number of other assessments for this purpose.

In order to fulfil these functions, NAPLAN must be a standardised assessment that all participating students take at roughly the same time in the school year. It must be able to be administered to students in groups, it must be able to assess students with a wide range of abilities, and it must be in a format that allows moderation and comparison within year groups (to measure gains for cohorts of students) and over time (to measure performance trends).

In her comments on the reasons for the review, Minister Mitchell has stated on a number of occasions her preference for NAPLAN to be a ‘diagnostic’ test. By this Minister Mitchell seems to mean that the test should be able to provide up-to-date measures of individual students’ achievement or ability and their learning growth. It is indisputably important that teachers use sound assessments to obtain detailed information about the learning progress and needs of individual students, however this is not the purpose of NAPLAN, nor can it be.

There are substantive differences between standardised and diagnostic tests that make them fit-for-purpose for different educational objectives.

Standardised tests

Standardised tests are designed to allow comparisons. Standardised assessments are norm-referenced, meaning that they have been administered to representative samples of students and a profile created to provide information about how a student’s performance on the test compares with their peers. Norms can be created for each item or question in the assessment, for subsections that assess certain abilities, or for the whole assessment.

They have strict administration procedures and can be administered in groups or individually. They provide reliable information but cannot measure small changes in progress and should not be used frequently.

Standardised assessments are useful for research purposes but are not usually helpful for teachers making instructional decisions. If a child performs poorly on a standardised test, they will usually require further assessment — such as a diagnostic test — to determine the appropriate instructional response. Used with careful reference to statistical and measurement limitations, they are suitable for school and system performance reporting and accountability.

Diagnostic tests

Diagnostic assessments are used to identify specific strengths and weaknesses in the abilities of individual students on specific learning areas, such as reading. They assess a range of subskills in order to generate a diagnosis for a student’s learning difficulty.

Diagnostic assessments are necessarily time-consuming. They should usually be used only for students for whom more detailed information is required than can be provided by more general assessments of reading that are appropriate for their age and stage of learning (for example, from curriculum-based assessments and standardised tests). Diagnostic assessments should be used as and when a child demonstrates that they may be in need of specialised support and not administered at a set time in the school calendar.

It is not appropriate to use diagnostic assessments for system performance monitoring and accountability purposes.

NAPLAN cannot be a diagnostic test

 From the above descriptions it is clear why NAPLAN cannot fulfil the dual objectives of measuring and monitoring school and system performance as well as being an individualised diagnostic test that provides detailed information about a students’ abilities and learning needs.

A diagnostic test cannot be implemented across an entire system at the same time because it must be focused on the individual and used at the time when it is needed. The results of diagnostic assessments are not suitable for school and system performance measurement. For this purpose, a broader standardised assessment is required. A student’s scores on NAPLAN may point to the need for a further assessment, including a diagnostic test, but it cannot perform both roles.

Assessment is only useful insofar as it provides reliable and accurate information that can be used to guide decisions on instruction or on policy, and lead to improvements in teaching and learning. It is therefore essential to make assessment decisions based on sound measurement principles.

NAPLAN does need some revisions but should be retained

Given their influence on policy and practice, the NAPLAN tests should be scrutinised, reviewed and revised if necessary. They have some significant weaknesses.

The ‘National Minimum Standard’ (NMS) is a very low benchmark for achievement – much lower than the minimum standard in international assessments – and does not describe a level of proficiency that allows good academic progress across the curriculum.

The writing assessment has been troublesome from the start, with fluctuations in scores from year to year, and a questionable fixation on persuasive over other forms of writing. While a consistent writing form for the test may be more conducive to constructing annual trend data, it has arguably had a restrictive influence on students’ writing development and should be reviewed.

However, arguments against NAPLAN on the basis that it causes excessive and harmful stress to students are unfounded, and there are more parents who value NAPLAN than don’t.

While the initial roll-out of NAPLAN Online had some problems, it is a positive move. With NAPLAN Online, it is hard to see why all children would need to take the assessment at the same time on the same day – it could be administered over a one to two week period – relieving some of the pressure on schools, teachers and students.

NAPLAN is necessary. Without NAPLAN, there would be no way of knowing whether our education systems are improving or not. There would be no way of quantifying the gaps in student achievement associated with socioeconomic and other forms of disadvantage, and therefore no pressure on governments and educational leaders to address them. NAPLAN shines a light on government and system performance and this capacity of the assessment should not be subjugated in order to achieve a different goal – one that is better achieved by different means.