Sunday 20th January 2013

This is another question commonly asked by students and their parents. Even assuming that my 'retake' student has been savvy enough actually to look at the record of results sent to them by the examination board (and it is amazing how many don't!) they are usually totally baffled by the figures presented on it. As discussed in a previous blog on HOW UK EXAMINATIONS ARE GRADED it may well be no coincidence that examination results are presented in such an opaque fashion.

The numbers that are presented on the results record by the examination board are what as known as 'UMS' marks. This stands for 'Uniform Mark Scheme'. And, as you might expect from the UK's arcane and over-complex examinaton system, these numbers bear no relationship to the Raw Marks achieved by the student in the actual examination.

The question therefore becomes 'how are the raw marks on the actual paper converted into these UMS marks'?

I would love to be able to give you a straightforward answer to this question. In fact I asked the OCR this question directly some years ago. To their credit they replied promptly - with five pages of abstruse mathematics!

I don't think that any mere mortal would stand much of a chance of understanding the complex calculations involved. This in itself suggests that the system is in urgent need of reform and simplification. In essence, under the modern system of grade reference marking (that replaced normative marking in the UK in 1987) the examiners decide in their infinite wisdom what constitutes a 'grade A' standard answer and the raw mark boundary to achieve a grade A is established at that point. For reasons that I don't fully understand this exercise is repeated only for the grade 'B' and the grade 'E' raw marks boundary for that examination session. Grades 'C' and 'D' seem to be fitted around the figures decided to grades 'A', 'B' and 'E'.

The idea is that if an examination is 'easy' (ie a lot of candidates are getting high marks) then the raw mark boundary to achieve a given UMS mark can be shifted upwards - and indeed vice-versa. This means that the factor that is used to convert a given raw mark into a 'UMS' mark varies from sitting to sitting of a given examination.

In a throwback to 'normative' marking the UMS marks to get a given grade at A-level are fixed permanently at 80% UMS for a grade 'A'; 70% UMS for a grade 'B'; 60% UMS for a grade 'C'; 50% UMS for a grade 'D' and 40% UMS for a grade 'E'. So, to get a grade 'A' you need to be in the top 20% of all UMS marks and so forth.

To further complicate an already over-complicated system UMS marks for a given unit are often quoted out of numbers other than 100. For example the UMS result for unit 2 of the AQA A-level biology examination is always given out of 140 total UMS marks. So, for example, to get a grade 'A' in unit 2 you will need to get 112/140 UMS marks, which calculates out at 80% UMS (as described above). But of course the RAW marks required to get this UMS mark will vary from examination to examination depending on the 'difficulty' of the examination session in question (and, it is reasonable to assume under 'grade reference' marking systems, the whim of the examiner).

To give you an idea of how this system can be manipulated (a cynic might assume for political reasons not unconnected with recent attacks on the examination boards regarding 'dumbing down' - and the need to manipulate the numbers of people going to University, for example) here are a few figures for UMS conversions over the past seven AQA A-level biology sessions.

In January 2009 the raw mark needed to get 80/100 UMS marks (ie a grade 'A') on unit 1 of AQA A-level biology was 62%. In June 2009 this figure rose to 67%. It fell slightly to 63% in January 2010 and rose again to 75% in June 2010. In January 2011 it was 68% and this rose to 75% in June 2011. In January 2012 it rose to an astronomical 80% raw and by June 2012 the raw mark required to get a grade 'A' went even higher to 82%!

So in short a candidate needed only 62% raw marks to get a grade 'A' on paper 1 in January 2009 whereas in June 2012 this figure was 20% higher at 82%. No wonder I had a whole procession of AS-level retake students this September, quite convinced that they were in some way 'stupid' when in fact they were struggling aganst rapidly-moving goalposts.

Am I angry about this on my students' behalves? You bet! The possible reasons that the goalposts have been moved so far and so fast, logically-speaking, are:

1) that the examination is getting to be so stupendously 'easy' that candidates need now to get much higher raw marks to achieve a given grade (presumably in an effort to make their achievement comparable to previous, more 'difficult' sessions).

2) Student are getting to be so much 'cleverer' nowadays (or better taught, according to the teaching unions) that the raw marks needed to get a given 'grade' need to be adjusted upwards - again, presumably in an attempt to make their efforts comparable to previous generations.

3) The examination board has got themselves into a state (via grade reference marking) whereby they are in danger of awarding 'too many' high grades and therefore need to raise the grade boundaries quite arbitrarily.

I honestly do not know which is the correct answer to this conundrum. However, as the reader may have guessed, I have my suspicions! The fact is that in the 19 years of working with AQA and its predecessors prior to the introduction of AS/A2 in 2009 I never knew the raw mark needed to get a grade 'A' in a biology A-level theory paper to rise above about 66% - and more usually it was closer to 62%. This does suggest that if examinations are 'getting easier' or candidates are 'getting brighter' then it has all happened in an astoundingly short period of time! The fact that all of the examination boards have been caught at some time with their hands in the cookie jar arbitrarily adjusting grades downwards to suit their political agendas (more on this later) does suggest that it is not an impossible notion.

Until we get shot of this pernicious grade reference grading system and move to a transparent system of simply quoting raw marks (in the context of the distribution of marks for all candidates taking the examination) then the UK examination system will continue its inexorable slide into irrelevance.