Conflicting polls on S59 law?

The Green Party engaged in a shoot the messenger exercise yesterday and attacked the integrity of the poll done by Curia for Family First on the law that amended Section 59 in 2007.

I’m not planning to respond to the sillyness of the Greens “revealing” I worked in Parliament for National for eight years, as this revelation is on Kiwiblog and known to almost every political commentator in NZ.And anyway I try and keep the silly politics over on Kiwiblog. Curiablog is about polling.

In the release from the Greens, they contrasted the Curia finding of 80% opposition to the 2007 law, with a UMR poll that found only 28% opposition. Now how is this possible many people will wonder?

I thought it would be useful to use this as a case study, to look at how important the order and wording of questions is. Often comparing one poll to another is comparing apples and oranges.

I should make clear that I regard both Curia (obviously as I own and manage it) and UMR as very good professional polling firms, and that any discussion of differences is to help public understanding.

The UMR poll details are here. They did their poll of 750 responses in July 2008 and it was released by their client (Office of the Commissioner of Children) a week after the election.

The first question in the UMR poll was “Should children be entitled to the same protection from assault as adults”.

Unsurprisingly that proposition gets 89% support. It is probably useful to note at this stage that respondents will often give answers to a survey which may appear to be contradictory. For example in NZES surveys, many people say they want to go back to FPP but they like having lots of parties in Parliament.

The second UMR question was

“Are you aware that the law about physical punishment of children was changed last year”

Then UMR asked respondents to rate theri support or opposition for the law change on a 0 to 10 scale.  Only those who responded 0 to 3 were included in the 28% marked as oppossed. Those in the 4 to 6 range were marked as neutral.

Note that as far as I can tell, the law change was not described to respondents. They were simply asked firstly if they were aware of the law change around “physical punishment” and if so, then what they thought of it.

Now you may have people who supported the law change as an improvement on the status quo, but also wanted an examption for light smacking to be maintained. There is a difference between generally asking support for an undescribed law and a specific provision of the law.

Interesting UMR also went on to ask if people agreed:

There are certain circumstances when it is alright for parents to use physical punishment with children

And UMR found 58% of respondents agreed with this statement and only 20% disagreed.

When you look at that finding, then it is worth looking at the Curia poll. The first question was:

In 2007, Parliament passed a law that removes a defence of reasonable force for parents who smack a child to correct their behaviour, but states the Police have discretion not to prosecute if they consider the offence was inconsequential. What is your view of this law?

And 25% said they strongly or somewhat agreed with the law and 65% said they strongly or somewhat disagreed with it. Now when you look at the UMR poll which says 58% said it is okay to physically punish your children sometimes and 20% said it was never okay – well the two polls can actually be seen to be quite close to each other.

There is rarely a definitely right or definitely wrong when it comes to framing questions. It depends on what you are trying to find out. The Curia poll was inquiring very specifically about whether light correctional smacking should be legal. The UMR poll for the Children’s Commissioner was inquiring more generally into attitudes around child disclipline.

Incidentially there will be a referendum in July on the issue of whether light correctional smacking should be legal. It will be interesting to observe the outcome.