Yawning Bread. 27 October 2009

Tough tests serve a positive purpose




I noticed the first letter or two in the Straits Times about how difficult the mathematics paper was at this year's Primary School Leaving Examination (PSLE), the nation-wide examination for Primary 6, who are mostly 12-year-olds. Then I believe there were more letters by more parents about how difficult some other subject papers were, but I didn't really take much notice by then.


Apparently, this is the not the first year that some parents have voiced their unhappiness. One claim is that the Ministry of Education, which sets the exam questions, has been steadily making the questions harder in response to rising math abilities. The ministry denied this, saying that like all exams, the questions are a mix of easy, medium and difficult ones (see news story at right).

My first reaction was that it shouldn't matter much if a certain year's exam is slightly harder or easier than the previous years'. The scores may be used to determine entry into secondary schools, particularly the "good" schools which tend to see huge demand, but since there are a set number of places, ultimately it is the pupil's relative score compared to his peers that matters more than the absolute score. No doubt the ministry should be explaining this clearly to the public but for some reason, they are not; or they did, but I missed it.

Then a reader wrote in asking me for my thoughts on the matter. Not being an educationist myself, I didn't think I would have much to say. So once more, I put it out of my mind. However, over the last two days, something bugged me. I sense that there is a conceptual problem that may need to be explained; and conceptual problems tend to give me a little thrill.

I suspect the problem is more one of expectations among parents than it is of how easy or hard the math (or any other subject) paper is. Since relative scores count more than absolute scores, the system can withstand some variability in terms of difficulty. The expectation, however, is that, for an above-average pupil, just about all questions should be do-able within the time given. So when a child comes home and says that he couldn't manage 2 or 3 questions, distress results.

It is the expectation that is incorrect. As hinted at in the ministry's reply, a good testing regime must have questions that range from truly simple to really hard. Children's abilities distribute in some form of bell curve. How does one tell the exceptionally bright ones apart from the bright ones? To do that, there must be a few questions that perhaps no more than 1 or 2 percent of the cohort can manage. This graph below should illustrate the point.

In the above example, a typically bright pupil will be seriously challenged by Question 8 and Question 9, and may find himself at a loss with Question 10.

One suspects that children and their parents are too used to textbook exercises and midterm tests that employ too many "average-difficulty" questions, with the result that above-average children come to expect scores of 9/10 or 10/10. This happens when test questions are all located in the middle part of the ability range:

Such a test regime, however, is useless. One cannot distinguish with it which kids belong to the far ends of the spectrum and who may either need special help or are capable of going onto a faster track.

I see in a few letters that parents feel children should not be demoralised by tough exam questions. Is this a good enough justification for tinkering with what a good testing regime should be? Or is this yet another facet of the expectations problem? When a child and his parents start expecting to be able to do all the questions in a test with relative ease, then of course it will come as a rude shock when he is faced with a paper that contains some truly difficult questions. But this worries me less than the prospect of a society where large numbers of kids grow up thinking highly of themselves, on no better foundation than that they've had a lifetime of flawless exam results.

* * * * *

Last week, I met with four first-year students from the Singapore Management University. They had asked for an interview and I said yes. They needed the interview in order to write a paper about 'The Employment of Gays' or something like that.

I usually make it a point to give tertiary students who interview me a slightly harder time than they expect the session to be, otherwise they won't really learn anything from the encounter. But there are days when I sigh to myself: It's too easy for me; it doesn't take much to give them a hard-ish time.

This interview was one of those.

One of the starting questions from the list they prepared was something to this effect: Is there a difference between foreign multinationals and local companies when it comes to hiring gays?

After a digressive discussion about whether it is meaningful to make a distinction between foreign multinationals as a class and local companies as another class, I thought I'd prompt them to consider whether recruiters even see sexual orientation in the candidate they are interviewing.

So I asked the four: Have you ever seen an employment application form? What questions are asked on it?

None of them had seen one.

My follow-up question, by now rhetorical: Is there even a question about sexual orientation on the typical form?

And in case one or two of them had still not grasped the fact that it was a rhetorical question, I had one more: If sexual orientation is not apparent to the interviewer, how is it going to be a factor in his hiring decision?

I don't think the four left the interview demoralised. Yes, they were challenged. They realised, I think, that they hadn't done enough homework before showing up. The result was a messy interview where instead of getting neat answers to their prepared questions, they got a series of interjections from me querying the assumptions behind the questions. But that is the nature of the learning process, and from what I could see, they took it all quite positively. I wouldn't have done them any favours if I made it too easy for them.

Yawning Bread 



23 October 2009
Today newspaper

Singapore's exam board says PSLE papers not too tough

By Zul Othman

After two outbursts in the last three years from parents about the Primary School Leaving Examination, in particular, the Mathematics paper, is there a nascent trend of questions becoming too difficult for 12-year-olds?

Not so, says the Singapore Examination and Assessment Board (SEAB).

In response to questions from MediaCorp, a spokesperson said that the panel responsible for setting questions ensures "there is a good coverage of syllabus and a judicious balance of easy, average and difficult questions".

This examination panel is made up of "experienced schoolteachers, curriculum specialists from the Ministry of Education (MOE) as well as SEAB assessment specialists" and they take "careful consideration" to ensure "the questions are within the respective syllabus and within the pupil's abilities and experiences".

There are also "processes" in place to "calibrate the difficulty level of each question and to control the overall standard of the paper", said the spokesperson though no specific examples were given.

Asked if there has been grade inflation and hence, tougher papers being set in response to better scores the SEAB spokesperson noted that the level of difficulty of this year's PSLE Mathematics test is "pitched at the same level as that in the previous years".