Same day, two different polls. Last Tuesday, The
Australian looked at the previous weekend's Newspoll and
declared: "No budget rebound for Crean." Fairfax waved the ACNielsen
survey around and pronounced the opposite: "Poll results put a
spring in Crean's step" (The Age), and "Slap for PM's plans
gives Crean a lift" (Sydney Morning Herald).
So which was correct?
They differed in three ways. First, in questions asked. ACNielsen
found 77 per cent of respondents willing to forgo tax cuts for
better services, 38 per cent favouring Simon Crean's health policies
over John Howard's (29 per cent liked Howard's more), and 72 per
cent opposed to the government's higher-education reforms.
Of Newspoll respondents, 43 per cent judged the budget good for
the economy, and 21 per cent did not. Just 27 per cent thought Labor
would have delivered a better budget, 51 per cent did not.
So one set flattered the government and the other didn't, but
neither contradicted the other.
Second, some almost identical questions returned different
results. ACNielsen and Newspoll put Crean's satisfaction ratings at
30 per cent and 24 per cent respectively (both gave Howard 61 per
cent) and their preferred PM numbers favoured Howard 61 to 23 and 64
to 17. Most importantly, on voting intentions, ACNielsen found 51 to
49 two-party preferred in the governments favour, a two point shift
to Labor, while Newspoll had it stuck on 54 to 46.
There is a small if to this. ACNielsen measures both primary and
two-party preferred, while Newspoll gets only primary support and
notionally distributes, as the minor party and independent
preferences flowed at the last election. But it does so in
aggregate, without differentiating between, say, a Green and One
Nation vote. This overstates the Coalition's lead, by (and this is
complicated by rounding) two points in about every second poll.
Still, even 53 to 47 would be a landslide and 51 to 49 a modest
win, so which is correct?
Well, statistically they're both OK. Put simply, surveys of that
size have about a 2.5 per cent error margin with a 95 per cent
confidence level. That means 19 times out of 20 their result is
within 2.5 per cent of the correct figure for the Australian voting
population. So when ACNielsen finds 51 per cent Coalition support,
it means it is between 48.5 per cent and 53.5 per cent. Probably.
There is one chance in twenty that it isn't.
The pollsters know their craft's parameters and were unfussed
this week. To them the polls said the same thing. But the papers
interpret the figures literally. And news organisations are human
concoctions, with cultures and biases. Statistics need colour and
minimum equivocation. Writers favour the rhetorical brush.
We might guess that when the data arrived, journalists went
straight for voting intentions, skimmed over the rest and set the
story in stone. Everything, including the numbers, then became
subservient. Cue the screaming headlines.
They did us a favour on Tuesday, by illustrating the limitations
of political opinion polling.
Peter Brent is editor of mumble.com.au, a website that
looks at electoral behaviour.