Yawning Bread. June 2006

Interpreting the IPS voter survey


    

 

 

In IPS post-election forum, part 1, I reported on the paper presented by Gillian Koh from the Institute of Policy Studies (IPS). It was a survey of 985 Singapore citizens aged 21 and above, carried out in the 2 weeks after polling day.

Here in this essay, I shall give my own view about what the results mean, but before that, I need to qualify any observations I am about to make because, in my mind, there is considerable uncertainty as to how representative the results were.

 
Age and household income distribution

The paper said that the survey was based on "random systematic sampling with quota control." This suggests that they started off with the phone book of residential numbers, which, for now, is still a reasonable starting point (but see footnote [1] below).

The sample was quota-controlled for age group and household income, the aim being to match the distribution as detailed in Census 2000. But did it?

As you can see, the 21 - 34 age group was over-represented in the sample while the 35 – 54 age group was under-represented. Koh herself mentioned that they had problems within the limited time getting a good match to the census data. However, what is still not clear to me is whether they re-weighted the demographic segments before computing the results. It does not look like they did.

 
   

Rejection rate

From virtually the first question after Koh presented her paper, members of the audience raised the possibility of the "fear factor" skewing the results. A significant number of Singaporeans are quite paranoid about disclosing their true political feelings; they believe that the government is out to penalise them for any anti-government views they may have. To what degree might this have affected the responses?

I stood up to ask this question: "Of the persons whom interviewers managed to reach over the phone, what percentage simply refused to participate in the survey?"

The answer from Joshua Consulting, the company that did the actual survey, was "about 70%". There was an audible gasp from around the hall.

My friends then prompted me to ask a follow-up question. "In a typical market research survey for corporate purposes, what would be the rejection rate?"

Joshua Consulting said it would be in the region of 50 - 80%. Sensing the direction of my enquiries, the head of Joshua Consulting elaborated further. He said that getting a 30% response rate in the IPS survey is considered good, even going by voter surveys done in America. Moreover, it was not as if people were reluctant to participate; if anything, many respondents gave the interviewers far more information than was needed. They often went into a long spiel about politics while on the phone.

But this still does not answer the question of whether there was any bias in the sampling. With a 70% rejection rate, the sample was, in a sense, self-selecting. What motivated some to agree to being interviewed? What motivated others to decline?

I can imagine that those with little interest in politics -- the "politically apathetic" -- wouldn't want to be bothered. Those who were fearful of being penalised would either decline the interview or else give "safe" answers. On the other hand, a minority with strongly held views would gladly make time for the survey -– which may account for the long spiels.

Controlling the sample for age and household income is a poor surrogate for representativeness. Thus, at the end of the day, it is not possible to say for sure that there was bias or no bias in the sample, nor to what extent. 

See also the box on the right, about margin of error.

So, with the big caveat about representativeness, let's look at some of the data.

 
  

 
Sample size and margin of error

In his guest essay, BL said the sample size was too small to be representative. This, I don't quite agree. A sample population of 1000 is not uncommon for surveys of large populations, running into the tens of millions. The mathematical relationship is really between margin of error and sample size, not population size, and that relationship isn't a linear one, but based on a square root.

So long as the sample size is a reasonable one to obtain a tolerable margin of error, they key issue is how the respondents were selected, not how many.

Also, what interview method was used can be a major factor. Telephone interviews aren't very good for surveys that touch on sensitive questions (and one could say that political affiliation in the Singapore context is considered by many to be sensitive) because people don't feel assured that they are truly anonymous. "After all, you tracked me down by my telephone number!"

Assuming the 985 respondents were sampled without bias – and it's a big assumption, as discussed above – the margin of error for a survey of this size would be 3% (at 95% confidence level). This means that at no time should we read any percentage figure without thinking in terms of a band of plus/minus 3 percentage points around it.

For a layman's guide, visit these sites:

www.robertniles.com/stats/sample.shtml 
and
www.robertniles.com/stats/margin.shtml

 

The top 2 concerns don't mean much

Under "issues", IPS reported that 2 statements garnered the strongest feelings of importance. These were:

  • "Need for efficient government" -– 56% said its was very important, 37% important, 4% neutral, 3% not so important, 1% not important at all. 
  • "Fairness of government policy" -– 39% very important, 43% important, 10% neutral, 6% not so important, 1% not important at all.


Straits Times front page
2 June 2006

 

The Straits Times went to town with the above. It said, "the top concerns were the need for an efficient government and fairness of government policy."

But these two statements are rather no-brainers. How many people in their right mind, after all, would say that efficiency and fairness aren't desirable?

It would have been better if the "efficiency" question had been counterpoised with another to fathom the meaningfulness of its answers. For example, if the survey had posed an additional statement, "Need for right balance between efficiency and compassion" it would be interesting to see what results might have been obtained.

You might say that "Need for right balance between efficiency and compassion" could be a leading statement, but then, so was "Need for efficient government".

Furthermore, as you will see below from the pie charts, I don't even think it was clear that "fairness" was the second most important concern.

Thus, I would dispute the top-level significance that the Straits Times placed on these two results.

 
  

Checks and balances

Coming just behind the so-called top concern of voters (the motherhood value of an efficient government), three statements gave very similar results, after allowing for margins of error: fairness, need for different views and the need for checks and balances. 

Yet, as noted above, the Straits Times grouped "fairness" with "efficiency" in its headline as "the top concerns". Since efficiency is strongly associated with the present ruling party, doing so effectively slips the virtue of fairness to the ruling party as well [2].

But here's another interpretation: that voters see fairness as closely related to having different views represented and checks and balances in Parliament, and that it is this association which results in such similar pie charts for all three statements.

For example, in his letter to the Straits Times Forum, 6 June 2006, Huang Shoou Chyuan wrote:

The Institute of Policy Studies has confirmed, post-election, what I have long suspected: It is that I am indeed part of an overwhelming majority who feel that "fairness of government policy" and the desire for "checks and balances" are important.

See the linkage?

It's also interesting how the same pattern came up with another question much further down the survey.

35% strongly agreed with the statement that it was "important to have elected opposition members in Parliament". 48% more agreed, while 8% were neutral.

 
 

 

 

Why no percentages next to pie slices?

You will have noticed that I do not provide the hard percentages next to the pie slices. This is deliberate. Humans have a tendency to look at a number, and invest it with a specificity and importance that is not statistically warranted, due to the margin of error. Hence, I prefer to let viewers look at the slice sizes, which are fuzzier to the eye. I prefer you see pattern, rather than numbers.

But for the record, IPS reported that:

"Fairness of government policy" -- 39% said very important, 43% important, 10% neutral, 6% not so important, 1% not important at all.

"Need for different views in Parliament" -- 38%, 45%, 9%, 7%, 1%.

"Need checks and balances in Parliament" -- 41%, 41%, 9%, 8%, 1%

 

So why did only 38% vote for the Workers' Party?

Yet despite 75% - 82% avowing the need for checks and balances and opposition MPs, the Workers' Party garnered only 38% of the votes on average, across the constituencies that they contested. Other opposition parties got even less.

What should we make of this?

For one, it may lead back to the suspicion that the survey sample was not representative. Those who were fearful, and out of fear voted for the PAP, might have avoided the survey. Those who were switched off politically and voted for the PAP because they didn't have any idea about alternatives might also have avoided the survey, lest they find themselves stumped by unfamiliar political questions. If so, this means that the desire to have checks and balances or opposition members in Parliament isn't as widespread as the pie charts indicate.

Another possibility may be that the desire is there to the extent that the survey suggests, but that the opposition parties aren't "good enough" for the voters. In this respect, one of the survey questions on the credibility of the various parties may provide useful results:

For the Workers' Party and the Singapore Democratic Alliance, the voters most likely to say they are credible are the upper middle income, the graduates and the professionals. 

In addition, it may appear that those under 40 tend to support the SDA too, and those under 30 tend to support the SDP, but this may be contra-indicated by the next set of findings.

 
 

Political orientation

The IPS researchers classified respondents into 3 categories: Conservative, Swing and Pluralist. Exactly how they did that, based on the answers provided, was not clear from the presentation.

The political orientation of the 985 respondents were then projected onto an age-split of "under 40" and "over 40".

Given the margin of error, one cannot be certain if the differences are all that significant. The IPS researchers however noted that the younger voters tended to be less fixed in their political ideology, i.e. more "swing", based on the observation that more of them gave inconsistent replies to the various questions.

If so, this could be that they're less ideological, but more tactical or issue-selective. Alternatively, it could be that being younger and having had fewer chances to vote, their political views were not fully formed, with the inconsistency coming from not having thought the issues through. 

Of greater significance would be these pie charts representing political orientation by housing type, which is a good indicator of socio-economic class.

(The pies are bigger where the housing category contained more respondents)

It is quite evident that the opposition parties would get their most ideological supporters from the middle and upper-middle classes. This then leads me to a discussion about their election program and style.

 

Opposition parties' programs and styles

The middle and upper-middle classes would be the ones doing relatively well in life. With better education, they would probably be disproportionately English-speaking.

This suggests that opposition parties must ditch their usual habit of pitching their messages to the less educated -- speaking in Chinese dialects and making simplistic promises -- or banging the drums of the economically dispossessed.

As well, they need to ensure that their programs are not out of sync with the concerns of their best supporters. They have to suppress their love for populist rhetoric such as subsidies, since the better-educated and middle class can see through naive economic ideas. Perhaps some of them are concerned with the cost squeeze, but they mostly know that the solution isn't subsidies but cost control. They are also aware enough that globalisation is impossible to avoid. Hence, parties should avoid protectionist platforms, but speak more about cutting red tape, helping entrepreneurs and other small businesses.

On the other hand, the middle class -- in fact, almost everyone -- feel very stressed out by non-economic demands, such as schooling for their kids, medical care and looking after the aged. 

And it may be that being better educated, they are particularly incensed by the way the ruling party talks down to them as if they were children.

The Workers' Party has clearly made significant changes in the recent general election, and no doubt they were rewarded for it. What the IPS survey shows is that this is the right direction to go. 

But another consequence of having the middle and upper-middle classes as your best supporters is that rallies may not be an effective way to reach them. They don't need the entertainment that rallies represent, let alone the traffic chaos. To reach them, the parties must try to get their message out through the mainstream media and the internet.

The survey found that people considered newspapers and television much more important than rallies for information about political parties. Gillian Koh also reported that while the internet was rated less important than these 3 communication channels, the better educated tended to rate it highly.

Unfortunately, during the recent general election, the Workers' Party kept the mainstream media at arm's length. One can understand this from the history of biased reporting, but going forward, they need a better strategy, especially as, with sympathetic young reporters, opportunity may be knocking.

Over the next 5 years, the opposition parties should also learn how to engage with the blogosphere. One idea may be, if a party is concerned about being misreported by the mainstream media, then always invite bloggers to any press conference it holds. That way, if the mainstream media misreports or under-reports, the bloggers will point it out. Always video the press conferences and upload to the 'net. Would that video be a political film? How can it be? If it's a press conference, isn't that news?

The IPS survey shows that it's the more sophisticated voter who is most likely to support the opposition, although it's hard to quantify them. Hopefully, the opposition parties act on this new information and take advantage of it.

© Yawning Bread 


 

Footnotes

  1. One problem which may soon face researchers is the increasing tendency not to have a residential fixed phone line. I myself rely exclusively on mobile telephony, and I know a few others like me. The telephone directory will soon be an inadequate tool.
    Return to where you left off

  2. I don't mean to accuse the Straits Times of deliberate misinformation. More likely it's a case of a reporter needing to rush off a story and just taking the survey report's top two items in the list of issues as THE top two. Compounding it may also be illiteracy in statistical analysis among reporters, a very common ailment.
    Return to where you left off

Addenda

None