Using Adaptive Comparative Judgement to Assess Mathematics

Ian Jones & Lara Alcock
Loughborough University

Adaptive Comparative Judgement (ACJ) is a method for assessing evidence of student learning that offers an alternative to marking (Pollitt, 2012). It requires no mark schemes, no item scoring and no aggregation of scores into a final grade. Instead, experts are presented with pairs of student work and asked to decide, based on the evidence before them, who has demonstrated the greatest mathematical proficiency. The outcomes of many such pairings are then used to construct a scaled rank order of students from least to most proficient.

ACJ is based on a well-established psychophysical principle, called the Law of Comparative Judgement (Thurstone, 1927), which states that people are far more reliable when comparing one thing with another than when making absolute judgements. The reliability of comparative judgements means “subjective” expertise can be put at the heart of assessment while achieving the sound psychometrics normally associated with “objective” mark schemes.

Until recently comparative judgement was not viable for educational assessment because it is tedious and inefficient. The complete number of required judgements for producing a rank order of (n) scripts is (frac{n^2-n}{2}).However the development of an adaptive algorithm for intelligently pairing scripts as more judgements come in means the number of required judgements has been slashed to around (6n).

Advertisement

Alan Turing and the bullying of Britain’s geeks

[Republished from The Guardian’s The Notherner Blog]

Celebrations of Alan Turing’s life and work reach a peak this week with the centenary of his birth. The chair of the project, Professor S.Barry Cooper, continues his series for the Guardian Northerner with insights on the torment which the bright but unusual can still suffer at school.

John Turing talks in the family’s reminscences about his younger brother Alan, recalling how the future computer genius was noted for:

bad reports, slovenly habits and unconventional behaviour

The ‘neurotypical’ John says that neither he nor his parents “had the faintest idea that this tiresome, eccentric and obstinate small boy was a budding genius.”

It is still very common for geekishly irritating little boys and girls to suffer misunderstanding and routine bullying at school. Nowadays Alan would probably have been diagnosed with Asperger’s syndrome.

A Level Reform Consultation

Ofqual:

This consultation considers proposals for the reform of A levels in England. We are seeking views from higher education, employers, learned societies, colleges, schools and others so that A levels are the best that they can be.

You can download a copy in PDF format, A Level Reform Consultation [PDF, 557KB].

The deadline for responses to this survey is 11th September 2012.

Find out how to respond. To register for one of the events, or to request more details, please email  reform@ofqual.gov.uk:

• 18th July in Birmingham and 5th September in London for schools and colleges 
• 25th July in Birmingham and 24th August in London for the Higher Education community  
• 27th July in Manchester, 10th August in Birmingham and 29th August in London for all interested stakeholders including those who are unable to make the other dates above. 
  

Universities to set A-levels in new qualifications overhaul

Graeme Paton in The Telegraph:

Examiners will be expected to enlist the help of at least 20 British universities when drafting exam syllabuses and test questions as part of a major drive to raise standards, the Telegraph has learned.

All new qualifications will require a formal “sign-off” from universities – particularly leading research institutions – before being sat by sixth-formers.

The reforms, to be outlined on Tuesday by Ofqual, the qualifications regulator, are intended to ensure teenagers have the appropriate levels of subject knowledge and study skills required to get the most out of a degree course.

Read the whole article.

 

31st MATHEMATICS TEACHERS AND ADVISERS CONFERENCE/WORKSHOP

31st MATHEMATICS TEACHERS AND ADVISERS CONFERENCE/WORKSHOP
13.30-16.30, Tuesday 26th June 2012
School of Mathematics, University of Leeds
No registration fee

The School of Mathematics at the University of Leeds has a long history and tradition (over 30 years) of developing and maintaining contact with teachers in schools via this conference, our Sixth Form Conference, Mathematics Lectures in Schools and other promotional activities.

Knowledge of Fractions and Long Division Predicts Long-Term Math Success

Press release from Association for Psychological Sciences:

From factory workers to Wall Street bankers, a reasonable proficiency in math is a crucial requirement for most well-paying jobs in a modern economy. Yet, over the past 30 years, mathematics achievement of U.S. high school students has remained stagnant — and significantly behind many other countries, including China, Japan, Finland, the Netherlands and Canada.

A research team led by Carnegie Mellon University’s Robert Siegler has identified a major source of the gap — U. S. students’ inadequate knowledge of fractions and division. Although fractions and division are taught in elementary school, even many college students have poor knowledge of them. The research team found that fifth graders’ understanding of fractions and division predicted high school students’ knowledge of algebra and overall math achievement, even after statistically controlling for parents’ education and income and for the children’s own age, gender, I.Q., reading comprehension, working memory, and knowledge of whole number addition, subtraction and multiplication. Published in Psychological Science, a journal of the Association for Psychological Science, the findings demonstrate an immediate need to improve teaching and learning of fractions and division.

Mathematical Cultures I

This conference (see  Project Overview) is partially supported by the LMS and will take place in the  De Morgan House on 10-12 September 2012.

Aaron Sloman: Is education research a form of alchemy?

This piece by Aaron Sloman in ALT News Online may be of interest:

First three paragraphs:

Alchemists did masses of data collection, seeking correlations. In the process they learnt a great many useful facts – but lacked deep explanations. Searching for correlations can produce results of limited significance when studying processes with an underlying basis of mechanisms with astronomical generative power. But this correlation-seeking approach characterises much educational research.

Accelerated progress in chemistry came from developing a deep explanatory theory about the hidden structure of matter and the processes such structure could support (atoms, subatomic particles, valence, constraints on chemical reactions, etc.). Thus deep research requires (among other things) the ability to invent powerful explanatory mechanisms, often referring to unobservables.

My experience of researchers in education, psychology, social science and similar fields is that the vast majority of the ones I have encountered have had no experience of building, testing, and debugging, deep explanatory models of any working system. So their education does not equip them for a scientific study of education, a process that depends crucially on the operations of the most sophisticated information processing engines on the planet, many important features of which are still unknown. [Read more…]

History of Mathematics undergraduate essay prize

Winners of first British Society for the History of Mathematics undergraduate essay prize announced.

The BSHM is pleased to announce the inaugural winners of its undergraduate essay prize as Stephanie Crampin for an essay entitled The contribution of Évariste Galois to the founding of group theory and Nicole Johannesen for The application of mathematical understanding in the ancient Olympic Games. Stephanie is a third year student studying mathematics and philosophy at St Hugh’s College, Oxford and Nicole a third year student studying mathematics at St Andrews University.

Mathematics Competitions for University Students

By chance, I learned about existence of Israeli students competition in mathematics. Its website is in Hebrew, but problem sheets are in English and are easily identifiable on the website. They look interesting;here are two samples, 2011 Stage 1 and Stage 2.  Surprisingly, I have never heard about anything similar being organised in UK. There are International Mathematics Competition for University Students (organised by Professor John Jayne of UCL), but they are perhaps too remote from British students.