Reporting Surveys: How to report the tight race (Aquino vs. Villar in the latest SWS Survey)

Photo by Hellier Gavin right-clicked from allposters.com under terms of use of said site used here for non-commercial purposes

SWS President Mahar Mangahas sounded peeved (or impatient or irritated) when he said that the latest SWS survey showing the margin between Aquino and Villar at 2% should not be reported as a statistical tie. This was in a phonepatched interview by ANC (ABS-CBN News Channel).

Since this is their survey, maybe they can decide how it should be reported. Note however that it is the media partner of SWS, Businessworld, that first reported it as a statistical tie, and it is linked in the SWS website, as follows:

From the SWS website:

A very tight race

The national status of the three races for President, Vice-President, and the Senate from the BusinessWorld-SWS February 24-28, 2010 Pre-Election Survey is in the March 9, 2010 issue of BusinessWorld, and at the BusinessWorld website.

BusinessWorld (BW) is the media partner of SWS in polling, for exclusive publication by BW, the public’s voting preferences for candidates for national positions (President, Vice-President and Senators) in the 2010 election. The original SWS report will be posted in the SWS website two days following the BW publication.

And from the linked news report, from its media partner Businessworld:

Top ‘presidentiables’ Aquino, Villar in statistical tie

THE RACE to Malacañang has become very tight with the top two “presidentiables” locked in a statistical tie, a Social Weather Stations (SWS) survey commissioned by BusinessWorld showed.

The poll, the third conducted by the survey research institution for this newspaper, found the front-runners — senators Benigno Simeon “Noynoy” C. Aquino III and Manuel “Manny” B. Villar, Jr. — both losing ground among voters with little over two months to go before the May 10 national elections.

The cost, however, was more substantial for the Liberal Party’s Mr. Aquino, who saw his overall score cut by a substantial six points to 36%, compared to just a one percentage-point drop to 34% for the Nacionalista Party’s Mr. Villar.

Mangahas said that any news report should just give the figure, and state: Aquino leads Villar by 2%; he said that this was the more neutral statement and that any report stressing the statistical tie was leaning in favor of one candidate.

Note again that in the past, here was how the SWS reported the one-per cent margin between candidates, (at that time referred to as “recommended successors” in the survey) ( posted it in my blog before ):

7 November 2008. Third Quarter 2008 Social Weather Survey:

Noli De Castro, Manny Villar, and Loren Legarda are people’s top recommendees for 2010

Social Weather Stations

Vice-President Noli De Castro, Senate President Manny Villar, and Senator Loren Legarda are the top recommended successors to President Gloria Macapagal Arroyo in 2010, according to the Third Quarter 2008 Social Weather Survey, fielded over September 24-27, 2008.

xxx

(And in the second to the last paragraph: )

The Third Quarter of 2008 Social Weather Survey was conducted over September 24-27, 2008 using face-to-face interviews of 1,500 adults in Metro Manila, the Balance of Luzon, Visayas, and Mindanao (sampling error margins of ±2.5% for national percentages, ±6% for Metro Manila, Visayas and Mindanao, and ±4% for Balance Luzon). The area estimates were weighted by National Statistics Office medium-population projections for 2008 to obtain the national estimates.

In this report, SWS did not use the term “statistical tie”, but neither did it report the survey as “De Castro leads Villar by one per cent while Villar leads Legarda by one per cent”.

Rather, it enumerated the names of the three, then stated that they were the top recommended successors in the survey. It didn’t say, “tight race” or “narrow margin” or “leading by one per cent”, it said that the three were the top choices – that kind of reporting leans towards saying that they were practically in the same rank – a statistical tie.

Okay. Since the SWS President did not want the term “statistical tie” used in the news reports, maybe we can use as pattern its previous media release, like: enumerate the names then state they “ are the top candidates in the SWS Survey…etc”. That was how the SWS reported the three-way statistical tie in that survey.

Survey results should always be reported with context. Journalism requires context.

You can report “leading by 2 per cent” then give the margin of error, etc., time frame, etc.

Some news reporters and editors however want to spell out the meaning, so they report it as a statistical tie then give the figures and margin of error.

Statistical tie simply means there is a narrow lead that is within the margin of error (usually between 2.2% and 2.5%). It doesn’t mean they are in an absolute tie, it just means that, because of the margin of error, there is a probability that they could in reality be tied but it is just a probability.

So, the news reporters and the editors just made a short-cut of it and reported that it was a statistical tie.

Well, you can say one leads the other and the lead is within the margin of error.

Happy?

ANC later reported it as “Aquino and Villar in neck-and-neck race.” And later reported it as “Aquino keeps top spot” (guess towards whom this headline is leaning), with the lead paragraph stating that they were in a neck-and-neck race; this kind of report avoids mentioning Villar’s name in the head.

Just say it’s a tight race. You can use an active verb if you want, Aquino and Villar running in neck-and-neck race, you can either mention both in the head or not mention either, and all because the news source doesn’t want you to use the term “statistical tie”.

(Updated) When the reporting of surveys becomes propaganda (Dear Tangé; Tangé is my term of endearment for friends)

             A tv news program  reported  an SWS survey by saying that Noli de Castro was still the top choice for President and gave the figures as follows: Noli : 29%; Manny Villar: 28%; Loren: 26% .

Photo by David.R.Stoecklein.Tail-End.  Right-clicked from www.allposters.com, used here for non- commercial  purposes, under the terms of , free service by blog-use of image provided by said site.
Photo by David.R.Stoecklein.Tail-End. Right-clicked from http://www.allposters.com, used here for non- commercial purposes, under the terms of , free service by blog-use of image provided by said site.

   

 

 

     Not that i like any of the presidentiables, i don’t; i’d vote for Chief Justice Puno even if he isn’t/ wouldn’t be a candidate: but one of the most  important feature to mention in reporting surveys is the margin of error. I checked the SWS site and the margin of error for this particular survey is  plus or minus 2 .5% for national percentages (bigger for regions: plus or minus 6% for Metro Manila, Mindanao, Visayas, and plus or minus 4% for “Balance Luzon”).

        That means that figures within plus or minus 2.5%, or a range of 5%, are statistical ties.

       That means the three are statistical ties (not that i like any of them). For that matter, figures from 24% to 29% are statistical ties (24% plus  a 2.5% margin of error or  a maximum of 26.5%;  and 29% minus 2.5% margin of error or a minimum of 26.5% are statistical ties).

     (the tv news program changed the headline to Noli, Manny, Loren top picks; as i was writing this. Anyway.) Update: the evening news (ANC) reverted to the news headline and “story”: “Noli de Castro leads the survey, etc.” without mentioning any margin of error at all, while the free tv version (TV Patrol Channel 2) also ran the same story: “Nangunguna si Kabayan Noli sa survey, etc.”  also without mentioning any margin of error at all.  All these are forms of propaganda and bandwagonning in favor of Noli de Castro in the guise of news stories. The required reading for corruption in the media is “News for Sale” volumes 1 and 2 by Chay Florentino-Hofileña for PCIJ, which investigated and documented how media organizations sold news airtime and space to politicians to promote them.

      In the 2004 elections, the government-owned or government- controlled tabloids then  always bannered that GMA was leading the pack according to surveys when, on those rare occasions, she was ahead by only 1%; they also never reported  when FPJ led the surveys. Those banner headlines and news  stories are grossly misleading (as stated, figures within a 5% range are usually statistical ties) and intentionally misleading. They’re classic examples of surveys as propaganda, or the reporting of surveys as propaganda (i know,  i know, i promised last year to summarize the studies on the reporting of surveys, they’re in the back issues of the Philippine Journalism Review, which are cited in my syllabus, etc. etc., i never got around to it, if you’re a reporter or if you’re reporting on surveys you have a duty to  look them up yourself).

      The other important features that should be mentioned in reporting surveys, of course, are: who sponsored/ financed the survey, the method, the time frame, sample size, etc., etc., they’re all in those studies i mentioned in reporting surveys.  

       I have to run along now i will edit this and put a picture later; as it is, it already  took 45 minutes from me!

       Here’s the SWS media release on this particular survey, from www.sws.org.ph

     Quote “7 November 2008. Third Quarter 2008 Social Weather Survey:

      Quote “Noli De Castro, Manny Villar, and Loren Legarda

are people’s top recommendees for 2010

     Quote “Social Weather Stations

      Quote “Vice-President Noli De Castro, Senate President Manny Villar, and Senator Loren Legarda are the top recommended successors to President Gloria Macapagal Arroyo in 2010, according to the Third Quarter 2008 Social Weather Survey, fielded over September 24-27, 2008.

        Quote “To the survey question, “Sa ilalim ng kasalukuyang Konstitusyon, ang termino ni Pang. Arroyo ay hanggang sa taong 2010 lamang at magkakaroon ng halalan para sa pagka-pangulo sa Mayo 10, 2010. Sinu-sino sa palagay ninyo ang mga magagaling na lider na dapat pumalit kay Pang. Arroyo bilang Presidente? Maaari po kayong magbanggit ng hanggang tatlong sagot.” [Under the present Constitution, the term of Pres. Arroyo is up to 2010 only, and there will be an election for a new President in May 2010. Who do you think are good leaders who should succeed Pres. Arroyo as President? You may give up to three names], 29% named Vice-President De Castro, 28% named Senate President Villar, and 26% named Sen. Legarda.

      Quote “No list of names was provided to prompt the respondents.

      Quote “They were followed by Sen. Panfilo Lacson (17%), Sen. Francis Escudero (16%), former President Joseph Estrada (13%), and Sen. Mar Roxas (13%).

      Quote “The survey found 1% mentioning Pres. Gloria Macapagal-Arroyo, MMDA Chairman Bayani Fernando, Sen. Antonio Trillanes IV, Sen. Juan Ponce Enrile, Sen. Francis Pangilinan, Sen. Mirriam Defesor-Santiago, Sen. Richard Gordon, Sen. Jinggoy Estrada, Cong. Jose De Venecia, Sen. Jamby Madrigal, and Sen. Pia Cayetano.

       Quote “Nine percent could not give an answer, and 9% had no one to recommend.

       Quote “Compared to the previous quarter, those with an increase in mentions were: Senate President Villar, by 3 points, Sen. Escudero, by 2 points, former President Estrada, by 2 points, and Sen. Lacson, by 1 point. The question wording has been exactly the same in the past five quarterly surveys shown in Table 1.

       Quote “The mentions of Vice-President De Castro’s name went down by 2 points, while the mentions of Sen. Legarda and Sen. Roxas were steady.

      Quote “Survey Background

      Quote “The Third Quarter of 2008 Social Weather Survey was conducted over September 24-27, 2008 using face-to-face interviews of 1,500 adults in Metro Manila, the Balance of Luzon, Visayas, and Mindanao (sampling error margins of ±2.5% for national percentages, ±6% for Metro Manila, Visayas and Mindanao, and ±4% for Balance Luzon). The area estimates were weighted by National Statistics Office medium-population projections for 2008 to obtain the national estimates.

       Quote “The survey item on the recommended leaders to succeed Pres. Arroyo is not commissioned, but is done on SWS’s own initiative and released as a public service, with first printing rights assigned to BusinessWorld. #” Closed-quote.