Tools in Evaluating News Reports on Surveys using the Sept.23-27 SWS Survey
Check the data upon which the news report is based: Look at the methodology and double-check the figures (add up the figures) and check whether the report included these:
The SWS Survey of Sept. 23-27 was conducted using face-to-face interviews of 1,500 adults (18 years old and above) nationwide as follows:
300 Metro Manila; 600 in Balance Luzon (or Luzon outside of Metro Manila); 300 in the Visayas; and 300 in Mindanao
(for a total of 1,500 respondents.)
Look at the question asked:
The question asked was “nasisiyahan ba kayo sa pagganap ng tungkulin … ni Rodrigo Duterte?” (read the original) or are you satisfied with the performance of functions of …Rodrigo Duterte?”
Check if the margin of error was included in the news report and factored in when the lead and headlines were written (for example: In an election year or in the runup to election campaigns, editors should careful in reporting that “there is a lead” or “this is leading” or “this is the frontrunner”: The margin of error might show that there is no statistical difference).
In the SWS Sept. 23-27 survey of performance of functions, the differences are statistically significant (differences are larger and even more than double the 2.5% margin of error) and can be used as data for qualitative analysis.
Margin of error of the SWS Sept. 23-27 survey: Sampling: The proportions of the sampling (by area, gender, economic sector, age group above 18 years) is based on existing data and on figures of the Philippine Statistics Authority.
The margin of error is plus or minus 2.5% for the national percentages. “Margin of error of 2.5%” means take or add 2.5% to the final figure to approach certainty.
The margins of error for each island/group of islands is as follows: ±4% for Balance Luzon, and ±6% each for Metro Manila, Visayas, and Mindanao.
To understand the data, summarize the figures for your notes (make your own tabulation so you can double-check the reported figures):
For the SEPTEMBER 23-27 survey:
67% are satisfied with Duterte’s performance of functions 19% are dissatisfied 14% are undecided;
for a net satisfaction rating of 48%, that is:
67% satisfied, subtract 19% dissatisfied equals a 48% net satisfaction rating (at 2.5% plus or minus margin of error)
In the JUNE 2017 survey, the results were as follows: 78% were satisfied with Duterte’s performance of functions 12% were dissatisfied 10% were undecided;
for a net satisfaction rating of 66% ; or 78% satisfied, subtract 12% dissatisfied
equals: 66% net satisfaction rating.
His progress, or decline or fall, is computed by comparing the previous with the present; or the June with the September, like so:
66% June net satisfaction rating, subtract 48% Sept. 23-27 net satisfaction rating equals a difference of 18%;
or a fall of 18% in the number of people satisfied with Duterte’s performance of functions.
The June net satisfaction rating is 66% while the Sept.23-27 net satisfaction rating is 48%. Therefore, he fell by 18% in net satisfaction rating.
But you would also be interested in where the fall came from, first: in the big national numbers.
So, compare it by category of those who were satisfied before and those who are satisfied now; and those who were dissatisfied before and those who are dissatisfied now — like so:
(June) 78% were satisfied with Duterte’s performance of functions, subtract (Sept) 67% were satisfied with Duterte’s performance of functions equals 11% ;
Or a fall of 11% from among those who were “satisfied with Duterte’s performance of functions”, or from among those who were previously supportive of Duterte’s performance of functions.
Where did those who were formerly satisfied go?
They went here:
They went to increases in the dissatisfied and the undecided — (but where exactly: read on)
(June) 12% dissatisfied, subtract (Sept) 19% are dissatisfied equals -7% (difference) or there are more dissatisfied by Sept at 7%.
and the undecided as follows: (June) 10% were undecided, subtract (Sept) 14% are undecided equals -4% (difference) Or there are more undecided in Sept. at 4%.
For “straight news reporting”, this is more properly led off as (or headlined) as a fall of 18% in net satisfaction rating; or… a fall of 11% in gross satisfaction rating.
But since surveys are reported out on their net satisfaction rating rather than the gross, the first is the more appropriate lead or headline : 18% fall in net satisfaction rating.
But surveys on performance of functions of government officials should not just be straight news reports since they reflect social phenomena: A news report on the empirical data should have some data-based qualitative analysis to make sense to viewers and readers.
What are the big national figures telling us?
The net fall or fall in net satisfaction rating at 18%, came from the “falling out” of 11% from the decidedly supportive (or those who were previously satisfied with performance of functions); and not from the undecided, i.e., they came from those who were supporting Duterte’s performance of functions.
Did the falling out go to the undecided, i.e., did those who fell out continue to give him the benefit of the doubt?
No; the figures point in the other direction.
Of those who “fell out” :
63% (of the “falling out” or of those who fell out) went decidedly to the “dissatisfied”
(dissatisfied rose to 7%; or 7% of the 11% who fell out; that’s 63%)
In other words, there was a swing in the opposite direction –- and not to the middle ground.
The actions of Duterte in July to September and the events of July to September had a polarizing effect on the nation.
While some analysts dismiss the Duterte fall in ratings as normal for any sitting president, a historical comparison, i.e, comparison with presidents who were elected and had regular terms of six years, would show that Duterte’s fall in ratings, compared to Ramos and PNoy (the other presidents were not compared with because they were not installed by elections or they did not have regular six-year terms): Duterte’s fall in ratings is the steepest when graphed, i.e., percentage of fall over number of days shows the steepest decline, i.e., it is the most accelerated, or… to laymanize that further: it is the fastest fall.
A more detailed qualitative analysis can be made based on the economic categories of the respondents, and the areas surveyed as follows
(reporters can interview non-aligned social scientists for a data-based qualitative analysis):
Qualitative Analysis by Prof. Segundo Eclar Romero Jr., PhD, published by Vera Files : (the author is formerly U.P. Diliman Political Science professor and now Professorial Lecturer at the Development Studies Program of the Ateneo de Manila University) Excerpts
From Prof. Segundo Romero Jr: “The main insights are: “The poor are most exasperated. The poorest class, Class E, is least satisfied with Duterte’s performance (net satisfaction of 35%). They also show the most precipitous drop in satisfaction rating between the 2Q and 3Q surveys (32%). This is the largest socio-economic class, and should be watched closely. This is where the Kadamay, and the Marcos wealth redistribution hopefuls come from.
(blog admin’s note: Just based on news reports without yet personally gathering data, i’m guessing that this socio-economic class — Class E, or Classes D and E combined — is where most of those who were the subject of killings came from. To be technically empirical, one can get data on the socio-economic classes of those who were killed — i’m guessing most came from Class Eand D… There could be a correlation — i’m not saying there is a cause-and-effect because the two sets of data– the SWS survey and data on those who were killed — were separately and differently gathered. A specific study can be made if you want to be technically empirical.) “The Filipinos from Luzon outside of Metro Manila feel most neglected. Filipinos in the rest of Luzon outside of Metro Manila are the second least satisfied (net satisfaction rating of 36%) with the performance of President Duterte. This was a drop of 22% from the 2Q survey net satisfaction rating of 58%. The 1Q survey rating of 51% was the lowest for that quarter. Apart from being sidelined in the discourse on major projects for the south, this may also be partly because Balance Luzon contains the support base of the former President (PNoy) and the current Vice-President (VPLeni).
(blog admin’s note: There is existing data on the areas where the killings occurred, i.e, big data from the ABS-CBN research group. Without yet looking at it now, but based on news reports, i’m guessing most of those who were killed in the so-called “drug war” were killed in Metro Manila and Balance of Luzon. There could be a correlation — i’m not saying there is a cause-and-effect because the two sets of data were gathered differently. If you want to be technically empirical, a specific study can be made.) “Mindanaoans continue to be most upbeat. Across the 3 quarters, Mindanaoans have consistently had the highest rating for President Duterte, starting with a high 87% net satisfaction. This dropped by 12% between the 1Q and 2Q surveys, the highest drop across all socio-demographic groups for this period. Between the 2Q and 3Q surveys, there has been no significant change. Military action in Marawi seemed a justifiable man-made disaster. The promptness and adequacy of the rehabilitation effort will likely be an eye-opener for Mindanao. “The rich feel most confident and secure. Across the 3 quarters, Class AB respondents gave the second highest net satisfaction rating, although the rating stands almost 20% lower than the Mindanaoan rating. The changes in ratings are very small: 3% between the 1Q and 2Qsurveys, and only a reduction of 2% between the 2Q and 3Q surveys. The stability in the ratings suggest the Class ABC feel impervious to natural and man-made adversities and disasters, and see opportunities in any regime. They believe their political and economic safety nets are regime change tested to be reliable. “Metro Manilans best reflect the social climate. NCR respondents had mean initial ratings (64%). This increased insignificantly to 66% in the 2Q survey, and dropped 18% in the 3Q survey. This also best exemplifies the national trend. The NCR may well be the bellwether of Filipino public opinion as far as Duterte is concerned. “Filipinos remain hopeful, but show clear signs of worry. Overall, Filipinos gave a 63% net satisfaction rating to President Duterte. This remained unchanged in the 2Q survey, but dropped 18% in the 3Q survey. Three quarters of the same rate of drop will bring President Duterte’s satisfaction rating into negative territory, which means more people are dissatisfied than satisfied with his performance. Dramatic positive changes will need to be visible over the next six months if President Duterte will reverse this trend.” (Romero, 2017)