PISA data shows English and Welsh schools’ performance falling behind that of other countries. Or does it? And is importing other nations’ education policies the way to move up the tables? It’s not that simple, says Ian Bauckham.

Global positioning

International comparative data on educational achievement has suddenly become headline news.

Much of the white paper The Importance of Teaching, which applies to England only, is predicated on an analysis of the performance of English students in comparison with those in other countries.

The Prime Minister’s introduction cites falling performance in science, literacy and mathematics in the OECD’s 2006 PISA (Programme for International Student Assessment) report, and the theme is taken up and developed throughout the white paper.

Adopting features of high performing systems – including approaches to organisation and autonomy, the curriculum and especially the English Baccalaureate – are proposed repeatedly as ways of addressing performance.

When the 2009 PISA figures looked even worse, they were greeted by the coalition government in England as a vindication of its radical approach to reform.

In Wales the PISA statistics were met with real concern on the part of politicians and educators as the results show 15 year-olds falling further behind since 2006 and Wales ranked below the OECD average. The Welsh government quickly responded with a 20-point action plan.

So what are the PISA tests, and what do they actually tell us about our relative strength – or weakness?

Focus on reading

PISA is a survey of the achievement of 15-year-olds which has run every three years since 2000 across OECD countries and educational jurisdictions. It tests performance in reading, science and mathematics with the focus area rotating each time.

Schools are picked to take part in the sample in each country in such a way as to provide a statistically robust cross-section across all sectors (including the independent sector).

In 2009, the main focus was performance in reading. England’s score was 494 points, almost the same as in 2006, and one point off the 2009 OECD average of 493. Wales went from 481 to 476.

Concern over the outcome was understandably widespread in Wales, but why all the pessimism in England? The answer is two-fold.

First, in the rank ordering of countries, England has fallen behind, despite having a similar score to last time. As other countries have improved England has stood still. In reading, England has slipped from 17th position to 25th over the three-year period (although the 12 countries ahead of England scored so closely they are not considered statistically significant).

The second cause for alarm, one suspects, is the identity of some of the countries further ahead in the table: not just the predictable Pacific Rim and Scandinavian countries, but also some Eastern European countries such as Poland, and a good number of other Anglophone countries such as Australia, New Zealand and Ireland.

Maths was also tested as a subsidiary area in 2009. England scored 492, which was only very slightly lower than the 2006 score, and again very close to the OECD average. Wales, at 472, dropped 12 points from 2006. However, in rank ordering terms, England went down from 24th to 28th place (although seven are too close to be considered statistically significant), now behind not just the Asian high-performers, but also a whole range of comparable European nations and other English-speaking countries.

In science, the picture is a little more encouraging for England. Its score of 514 was the same as last time, but measurably above the OECD average and in 16th place (down from 14th in 2006). Wales dropped 9 points to 496.

The PISA results reveal other information as well as international rankings and changes in performance over time.

On the gender gap, although girls performed significantly more strongly in England than boys (25 points between them), a gender gap in favour of girls was found in almost every country, and the English gap was less pronounced than in some other high performing countries, such as Australia (37), Canada (34) and Finland, with a surprisingly high 55 point gender gap.

A less encouraging feature of the English profile is the spread of achievement. PISA found 18 per cent of those tested in England performing at the lowest level assessed, which, while not dissimilar to the OECD average, compares unfavourably with Finland’s 7 per cent and Australia’s 14 per cent.

Finally, PISA measures a range of attitudes to learning and schooling. Although close to the average, it is noteworthy that 39 per cent of English 15-year-olds said they did not read at all for pleasure.

Students’ responses to questions about school climate are generally an endorsement of what many English schools aim to achieve. For example, 78 per cent said that most of their teachers were interested in their well-being, much higher than the OECD average of 66 per cent.

Valid outcomes?

Teachers and school leaders might ask how valid PISA outcomes are. It is a legitimate question, and there are reservations in some quarters.

The recent Ofqual progress report International Comparisons in Senior Secondary Assessment notes that PISA presumes objective knowledge of what is best and assumes that the samples really are representative. Furthermore, it reminds us that, like all international testing, PISA is a snapshot and gives us little information about consequences of educational decisions over time.

Nonetheless, despite these possible caveats, we can probably say that PISA outcomes give us some useful pointers.

What does all this tell us about the nature and effectiveness of English education? First and foremost, it is quite hard to argue with the conclusion the government has drawn that while English performance is holding its own in comparison with where it has been historically, other countries are overtaking England, especially in reading and mathematics. In a competitive world market, this clearly matters.

But there are some good features as well, in particular the trust students in England report towards their teachers – far removed from the ‘broken’ system some politicians would paint for us.

But we must surely also ask how it is that, after a decade of unprecedented spending on education, there has apparently been at best standstill in terms of objectively measured performance. This leads us to the irresistible conclusion that something has to be done differently but it is hard to separate out proposals which are irrefutably evidence-based from those which are motivated by a particular political vision.

In his November 2010 report for Cambridge Assessment titled Could Do Better, Tim Oates (now a member of the National Curriculum Review Panel) devotes some time to analysing possible systemic reasons why Finland appears to do so well in international comparisons.

He notes that more superficial analyses of Finnish education focus on “high levels of autonomy to schools; small, homogeneous schools; and the absence of setting or streaming”. Often overlooked are the long history (since 1686) of legally enforced literacy in Finland by requiring it for marriage; a long, post-war period of centrallycontrolled teaching materials; the fact that, alongside the absence of setting and streaming, over 40 per cent of Finnish 15 year-olds move into a lower-status vocational pathway; a strong culture of early family learning and literacy; and the simple written structure of the Finnish language.

Oates concludes that while we must still admire the achievements of high performing systems such as that in Finland, “crude policy borrowing” is less likely to be successful than a measured approach to improving the coherence of the British systems, aligning the curriculum, examinations, teacher training and accountability, and allowing the system to become established and well understood.

One could reflect that our politicians may need to learn to think beyond the five-year window of power the electorate gives them, and, critically, they may need to develop greater cross-party consensus to avoid the potentially damaging pendulum effect at each change of government.

Sampling approach

The coalition is now committed to benchmarking UK educational achievement against international data in a more robust way than hitherto. To this end, it plans to make participation for maintained schools obligatory.

Moreover, the government plans for us to take part in a new international study, TALIS (Teaching and Learning International Survey). This will take place for the first time in the UK in spring 2013 and focuses on areas such as teachers’ professional development, leadership, appraisal, teachers’ pedagogical beliefs and practices and how under-performing teachers are dealt with.

With both PISA and TALIS, it will be important to resist the backwash effect of high-stakes international testing. We must focus on embedding genuine improvement, over time, resist crude policy borrowing and try to ensure that major changes are evidence-based. Reform must be for the genuine good of schools and students, not just designed to improve results in the next international test.

• Ian Bauckham is headteacher of Bennett Memorial School, Kent and chair of ASCL’s International Committee.

This article draws on an analysis of PISA by Lorna Bertrand, international evidence manager at the Department for Education and chair of the PISA governing board.

Designed with IMPACT

© 2024 Association of School and College Leaders | Valid XHTML | Contact us