Grade Expectations

Comparable outcomes were intended to guarantee fairness for students from one year to the next. Now, in light of this year's GCSE English debacle, they appear to have become a way of fixing 'grade inflation'. Sue Kirkham looks at where and why the policy went awry.

During the summer term of 2010, I attended an Ofqual briefi ng about the comparable outcomes process t that was to be applied to the fifi rst A level results for the revised specififi cation. It was presented primarily as a process to ensure that students would get the same grade as they would have received had they sat the examination the previous year. This is very important for A level, as students with 2010 A levels could be competing for higher education places with students who had gained A levels in 2009 or 2011.

ASCL had some concerns about the introduction of this process but, since the main driver appeared to be that of fairness to students, it was diffificult to argue with the intention. It was also the case that awarding bodies were already, individually, using the relationship between GCSE performance and A level outcomes to help ensure comparable outcomes but this had been a self-regulating process as they were using their own versions of the process. The difference from 2010 was that this would be applied consistently by all awarding bodies and regulated by Ofqual.

We were assured that it would not be used to hold down results artifi cially if there was evidence of improved performance by the candidates and that chief examiners would still be able to use their professional judgement. Indeed, GCSEs and A levels in Summer 2012: Our Approach to Setting and Maintaining Standards (Ofqual, July 2012) still states that comparable outcomes should only be applied where “there has been no substantial improvement (or drop) in teaching and learning at a national level” but all concerned know that it is in practice very difficult to provide evidence to substantiate such an improvement or drop.

We recommended to Ofqual that they publish information about the process in order to make it more widely known to and understood by centres, which they did in May 2011, revising it this summer. So the process was applied to A level in 2010 and subsequent years and, as we know, has caused A level results to plateau.

The comparable outcomes approach was applied to the new GCSE specifi cations that were introduced in 2011 and then, in 2012, to the new specifications in English, English language, English literature, maths and ICT. In meetings with senior Ofqual staff in May and June 2012, ASCL offi cers and members expressed their concern about the potential impact of this process on English and maths in view of the high entry and high stakes nature of these qualifi cations.

Our concerns were partly based on the statistical methodology used that bases the GCSE expected outcomes at a national level on Key Stage 2 results, a very different matter from using GCSE results as an indicator of cohort ability at A level. We were also only too aware that there could be unexpected outcomes from even small changes to grade boundaries at the point of awarding in large entry subjects.

Unfortunately, by the time the results were published this summer, ASCL had still not been able to gain access to the full information about the use of Key Stage 2 data so questions remained unanswered. For example, independent schools’ students are not included in Key Stage 2 statistics just as Welsh students who do not take Key Stage 2 tests are excluded, and yet they are taking GCSEs from the whole range of awarding organisations. It is not clear how this incomplete data set has been extrapolated to cover the whole examination cohort.

Ofqual decided not to use the process for the first science awards in 2012 or 2013, following the new specifications, as these syllabuses were designed to be more challenging. This is fully explained in the Ofqual document referred to earlier. Schools were expecting it and should have been able to plan and prepare students accordingly.

Fairness to individuals?

It seems that the initial intention of the comparable outcomes process has already mutated during its short lifetime. Intended as a guardian of fairness to students from year to year, especially in a period of qualification change, it now appears to have become a way of fixing ‘gradeinflation’.

The results for English GCSE in 2012 – and to a lesser but important extent maths – are said to be sound and safe because national statistics show that there was a small decrease in overall attainment in line with the ability of the cohort. What has happened to the concept of fairness to individual students? Surely if this process is to work then it must work for modular assessments taken at different times, just as it should work for whole qualifi cations certifi cated in different years. What was the role for the professional judgement of examiners this summer if they were happy with the June 2011 and January 2012 awards and then suddenly found them to be over-generous by July 2012?

Awarding bodies have considerable experience of modular awarding at both A level and GCSE. When modular A levels were fi rst awarded following the Curriculum 2000 reform, a similar situation arose to the one that arose this summer at GCSE and one might have expected that lessons were learned. Awarding bodies have also been awarding modular GCSEs for a number of years, although not in such large numbers as in 2011 and 2012.

It was known and well documented that the new English and English language specifications would be difficult to award. Surely, given an awareness of the very high stakes nature of this qualification both for individual students and schools, extra care should have been taken, both by awarding bodies and by Ofqual, at each awarding point during 2011 and 2012 to ensure that awards were lining up with expectations.

At the time of writing, ASCL is still fi ghting for a resolution that is fair to all students and all schools. As far as we are concerned, the process of awarding in summer 2012 has gone badly wrong and must be put right. We hope that, by the time you read this article, that will have happened.

Our fears remain for the coming years, given that GCSE replacements will not be in place until 2017. Over the next few years, linear assessments will continue to replace modular assessments; spelling, punctuation and grammar (SPAG) marks will be awarded in some subjects; and the process of ‘tightening’ some subjects to ensure more rigour in the assessment process shall continue.

How are awarding organisations to deal with these constant year-on-year changes while maintaining standards over time? Students now in Year 8 or above will be taking the current suite of GCSEs and they need to understand what standards are expected and to be reassured that they will be rewarded fairly for their efforts.

Joined-up approach

The consultation on the future of GCSEs is now underway and we know that considerable change is planned from 2015. ASCL Council is debating this at its meeting in October and will be responding very fully. We urge all ASCL members to let us know your thoughts by emailing consultations@ascl.org.uk

It will be extremely important for us to debate and address not just the issues in this consultation but also the more detailed questions about the purpose of the qualification and national expectations. If it is intended as a qualifi cation for all 16 year-olds and yet there are expected outcomes, whether produced by norm referencing, comparable outcomes or some other similar statistical device, then there is no point in schools working hard to encourage young people to be ambitious about their future and no point in expecting all schools to strive to be above the latest 'floor targets'.

ASCL opened a debate about this aspect of our qualification system with its conference on 4 October (planned way before the issues raised this summer) and will in awarding and a more joined-up approach to policy.

In its response, ASCL will be reiterating its long-held view that fi rst there must be agreement about the curriculum and educational experiences that young people need. Only then can you design appropriate qualifi cations and assessments. While qualifications remain a political football, to be kicked out as soon as governments or even ministers change, then awarding organisations will not be able to focus on their job, which is to provide fit-for-purpose, reliable and fair assessments. Huge amounts of time and money have been squandered over recent decades on developing new qualifications and tinkering with existing ones only to see them disappear or change again before they have had time to bed in or be properly evaluated.

Proper qualification reform should be able to gain consensus from all those with an interest in it, provide time for assessment experts to test it out and model potential challenges and unexpected consequences, and then be backed up with sufficient resources to ensure that students and teachers are properly prepared.

Our young people deserve better. Their counterparts in other countries – with whom we are frequently compared – are rarely used as guinea pigs in this way. continue to demand transparency

• Sue Kirkham is ASCL's curriculum specialist.
  

Designed with IMPACT

© 2024 Association of School and College Leaders | Valid XHTML | Contact us