Pollsters speaking to greed, not the science of surveys

Opinion polls are very important tools for assessing the opinion of ordinary people about important topics. Polls are conducted for many reasons, but in Kenya, we pay most attention to the popularity of politicians; the horse race.

The response to any poll is predictable: Praise from those they favour, criticism from those they don’t. The response by Musalia Mudavadi to the most recent poll by Ipsos showing that about half of Kenyans would vote for President Uhuru Kenyatta is typical.

Mudavadi, who is positioning himself as the most credible alternative to Jubilee, termed the polls as fake, used by government to mislead the public. Unsurprising. We know that if he’d been leading in the poll, the response would be different.

But these polls have also been challenged by scholars and analysts for failing to meet the methodological rigor required.

Unlike the accusations by politicians, this is sound criticism. A quick look at the available poll reports shows that sample sizes fluctuate often and their representativeness is unclear.

A bad situation is made worse by the poor quality of reporting on polls. It is surprising how often we see the results of a poll without getting the necessary information to assess its credibility.

Recently, a local daily published the results of a poll they had conducted which found that Kenyans have mixed feelings about Raila Odinga. He was reported to be the most controversial and most trusted personality. They also found that President Kenyatta had the most positive impact on Kenyans.

The report of this poll tells us nothing about who was interviewed, when and where, how they were interviewed or even the questions that were asked. Pray, what question would one need to ask to get the response that someone is the most trusted and also the most controversial personality? We are also not told whether they conducted the study themselves or commissioned a researcher. These are some of the issues that give pollsters and reporters a bad name.

An opinion survey is only as useful as its quality and accuracy. The design of the survey and their reporting must therefore inspire the confidence of the reader.

Transparency about the process of gathering data and analysing it will serve them well.

Kenyans will trust pollsters better if they knew that they actually made effort to be as precise as possible in their research. This means that their sample should be representative of the population, they ask proper questions and also report their findings accurately in an informative way.

It is often very difficult to tell what kind of people are interviewed by a pollster. For instance, we know that most surveys are conducted via mobile phone these days but we often don’t know where they get the telephone numbers from.

Presumably, they have to recruit people themselves since there is no directory of mobile numbers and they can’t get the lists from mobile service providers. It is not clear how many numbers they have in their databases. It is one thing to interview two thousand people from a pool of ten thousand, it quite another to do so from a pool of three thousand.

Beyond the numbers, the characteristics of the people they interview are also very important. True representative samples must go beyond geography to address other socially important factors such as gender and ethnicity.

The fact that you interviewed people in Lamu alone is not enough to demonstrate representativeness if the only those you spoke to live in a place dominated by Kikuyus.

The salience of ethnicity in our politics, that we often want to shy away from, is what made Mutahi Ngunyi’s tyranny of numbers trump over the pollsters in 2013. Failing to account for ethnicity in the sampling design will reduce the precision of the survey.

The idea is that we should be certain that the survey reports represent the true feelings of the population.

A truly representative sample of the population should be asked questions they understand and their feedback analysed to generate meaningful insights that is reported in a clear manner.