Why The U.P. Exit Poll Predictions Are Meaningless

NEW DELHI: Opinion polls and exit polls make for front page news and hours of live TV coverage, with analysts and viewers alike placing a huge amount of faith in these numbers that aim to predict election results. Two days ahead of the Uttar Pradesh assembly election results, all eyes are on the exit polls that have been delayed to March 9 on orders of the Election Commission. Exit polls for Goa, Manipur, Punjab, Uttarakhand and Uttar Pradesh are all expected March 9 following the EC order. The question, however, is -- should opinion and exit polls even count as news?

In a famous case gone wrong, Outlook magazine in 2004 ran a bold cover story headline based on an opinion poll, declaring that “NDA leads with 276.” The NDA, as we all know, went to lose that election, with the Congress coming to power. Outlook vowed never to use opinion polls to predict elections ever again, and although the publication has stuck to that decision, the business of opinion polls has continued to surge in popularity. TV channels, especially, insist on relying on opinion and exit polls.

Three recent examples serve as an indication of why opinion polls have zero credibility: Delhi, Bihar and Tamil Nadu. All pollsters got the Delhi assembly elections in 2015 wrong. Half-a-dozen exit polls gave AAP about 31 to 54 seats, putting the BJP at about 17-35 seats. The Week-IMBR, IBN 7-Data Mineria and Zee-Taleem Research Foundation (TRF) gave BJP 36 out of 70 seats. Today's Chanakya gave 48 seats to AAP and 22 to the BJP. The ABP-Nielsen survey gave AAP 39 and the BJP 28 seats. The Times Now-C-Voter survey gave AAP 31-39 seats and the BJP 27-35. The India Today-Cicero gave AAP 35-43 seats and the BJP 23-29. NDTV said AAP could win 38 seats and the BJP 29. In reality: AAP won 67 seats and the BJP got only 3 seats.

Bihar was a similar story, but here exit pollsters got even the leading party wrong. Today’s Chanakya on News 24 channel gave the BJP-led alliance a strong majority with 155 seats, and the rival JDU led alliance 80 seats. India Today-Cicero poll predicted 113 to 127 seats for the NDA and 111 to 123 for the JDU-led alliance. NDTV gave the NDA 120-130, and the JDU alliance 105 to 115. Today’s Chanakya said BJP would sweep the state with 155 seats, and the JDU alliance was predicted to get 83 seats. A handful of pollsters did put the JDU alliance ahead, but here too, the numbers were all wrong. C-Voter and News Nation gave a simple majority (122 seats) to the JDU alliance, while AC Nielsen and CNX gave them 130 and 132 respectively. In reality, the JDU alliance won 178 seats, with the NDA far behind at 58.

The figures predicted by exit polls for Tamil Nadu in 2016 were even more off. Four out of five exit polls predicted a victory for the DMK. News NationTV gave the DMK 116 seats and AIADMK 97. Axis-My Nation gave the DMK 132 seats and AIADMK 95. ABP gave the DMK 132 seats and AIADMK 95. And Today’s Chanakya gave the DMK 140 seats and the AIADMK 90. Only Times Now - CVoter predicted an AIADMK win (139 seats), giving the DMK 78. Four of the five pollsters were entirely off, with reality giving AIADMK 136 seats and the DMK 98.

While Delhi, Bihar and Tamil Nadu are recent examples, opinion and exit polls are largely inaccurate and have been so forever. In fact, “of the 82 electoral surveys that surveyed more than 1.25 million voters for 13 Lok Sabha and State Assembly elections since 1996 that we analysed, zero (yes, zero) have been fully accurate and 15 have been partially accurate,” as pointed out by Praveen Chakravarthy. “We define “fully accurate” as being within a +/-5 per cent range in terms of seats predicted vis-à-vis actual results for both the winner and the runner-up. The 5 per cent test is a fairly standard statistical norm to determine if a certain outcome is purely by chance or by design. We classify partial accuracy as those that predicted either the winner or the runner-up but not both, within the range of 5 per cent. If we relax the +/-5 per cent accuracy levels to 10 per cent, then 9 out of 82 surveys have been fully accurate and 27, partially accurate.”

Further, “accuracy levels of these surveys show no signs of an improving trend since 1996 though sample sizes and number of surveys have increased substantially. Similarly, there is no evidence of these surveys being able to predict a State Assembly election better than a Lok Sabha election or vice-versa. Neither is there proof of any specific survey organisation(s) getting their predictions consistently right vis-à-vis others. Accuracy levels of these poll surveys are understandably as random as electoral outcomes. Thus, at best, one in three poll surveys over the last two decades have been partially accurate and at worst, zero has been fully accurate.”

(Source: The Hindu)

Of course, being inaccurate hasn’t impacted the business of opinion and exit polls, which is booming. While exit polls employ some sort of vague methodology (most pollsters haven’t revealed their exact research methodology), analysts in the business of predicting election results have even less credibility. News pages have been dominated by self-processed political pundits giving their predictions of the upcoming elections, down to the last seat. Astrologers too are often referred to when it comes to election results.

This is not to say that exit and opinion polls, political analysts and astrologers will invariably be wrong. Instead, let’s just say Paul The Octopus has a far, far more accurate record in predicting football games than any of the above pollsters or analysts have in predicting election results. Yet, the cephalopod’s predictions didn’t make for hours and hours of live TV coverage and analysis.