The Newsweek Poll - Is Someone Cooking The Numbers?
Sunday, June 22, 2008 at 05:00PM Newsweek released a poll yesterday (June 21) purporting to show Barack Obama with a 15-point lead over John McCain. There is something amiss - - either there is bad methodology, or the poll numbers are out of whack for some reason not apparent.
Looking at the polls tracked on RealClearPolitics, I note the following:
DATE POLL OBAMA MCCAIN OBAMA LEAD
June 18 Reuters/Zogby 47 42 +5
June 19 IPSOS 50 43 +7
June 19 FOX News 45 41 +4
June 21 USA Today/Gallup 50 44 +6
June 21 Newsweek 51 36 +15
June 22 Rasmussen Tracking 49 42 +7
June 22 Gallup Tracking 46 44 +2
The Gallup Tracking results have been fairly stable for days, and the Gallup organization so reported on its Web site. Rasmussen's chart similarly shows very little variation recently. So, with six other polling organizations reporting an Obama lead in the range of +2 to +7, averaging +5, Newsweek comes out with a fifteen-point Obama lead. This report has been dutifully "analyzed" by the pro-Obama press, including MSNBC, of course.
On the one hand, I am not uncomfortable with John McCain's position in the race at this time. He's the underdog and that is the position from which he runs his strongest race. (Ask Mitt Romney!) On the other hand, I majored in political science, studied polling techniques, and took part in taking opinion-poll surveys, so I know something about this; and I am puzzled, to say the least, that the Newsweek numbers are so far out of the range of random error that something seems to be wrong.
If anyone has a better explanation, I'd be glad to hear it; but it looks to me like this is bad methodology or home cooking.
View Printer Friendly Version
Email Article to Friend
Reader Comments (13)
Something isn't right with the polls is putting it mildly. If one poll is so out of whack with the others, then either their polling techniques were faulty or they are fudging the numbers. I want to believe the first, but fear it might be the latter.
I have to believe is that "good news" spreads good news. Everyone wants to go with a winner. And so the media make the "good news" that they feel is fit to print. Look at what happened in NC. The polls showed Obama up by 20%. Then they showed Clinton coming close. They reported a 5% lead by Obama. Come the day after the primary, Obama wins by 14%. We are being manipulated by the new type of media...the one's that don't just report.
Among pollsters there is a term, "bandwagon effect," for the phenomenon of poll numbers going up because poll numbers have gone up. The "go with a winner" attitude outweighs, at least for a time, the "help the underdog" feeling. If Newsweek had reported Obama up by +9, that would be credible - - I'm simply not buying +15. Not that even +15 would be bad at this stage; Bush 41 was down by -17 in the summer of 1988. For a lot of reasons, the early polls can be off. I just don't get the feeling that the press, on the whole, is being fair - - Jay
If they are to deceit all nations with selecting this man, what are polls to them? The media has the power- remember what our old doctrine teaches that Satan is the Prince of Air-waves.
As a political scientist (with a phd) who studies American politics, I would tell you that there are two possibilities that come immediately to mind: (1) the sample was not a good one - either they oversampled Democrats or left-leaning Independents or young people or African-Americans (all who are likely to be supporting Obama in larger numbers than other groups) and/or (2) when the statistics are calculated for public opinion surveys, they are calculated to what is called the "95% confidence interval" - this roughly translates to meaning that 95% of the time, the sample means (those numbers which show where McCain and Obama are in terms of percent of support) will fall within +/- the margin of error. This, however, means that 5% of the time, the sample means can be outside this margin (e.g., the 15% spread!) and they may have no relation to what the "true" means are. In short, it could just be a "bad" survey that through nothing other than "random chance" simply fell outside of the margin.
My suggestion is to simply ignore it and watch the polls that come in over the next few weeks. If Obama has really widened his lead among the public - then more independent polls will confirm it. Personally, I don't believe he has. I think that he is somewhere around 4-5 points ahead of McCain - and that is only because he's enjoying something of a bounce from "winning."
Hope that helped. BTW - I like your Backcountry Notes!
Ok - it is the sample - here is the data - too many Dems. and Independents - seems to me that there are too many Dems and Independents - I would be suspicious of any poll that has only 23% of Republicans in its sample.
NEWSWEEK POLL:
DEMOGRAPHIC PROFILE
Total Total RVs
(n=1010) (n=896)
% %
Gender
Men 49 48
Women 51 52
Age
18-39 35 32
40-59 38 38
60+ 24 27
Undesignated 3 3
Party ID
Republican 22 23
Democrat 36 38
Independent 37 35
Other/Don’t know 5 4
Education
College graduate 28 31
Some college 24 24
No college 47 44
Undesignated 1 1
Race/Ethnicity
White, non-Hispanic 72 74
Total Non-White 26 24
Black 11 11
Hispanic 9 9
Asian 3 2
Other/Mixed race 2 2
Undesignated 2 2
This is nothing new. Just read the dirty politics that started Obama's political career
http://www.audacityofhypocrisy.com/2008/06/09/the-dirty-politics-that-started-obamas-political-career/
As a former pollster, I can tell you that coming in that far out of line, even when hitting the rare 5% confidence interval, in a survey of this size is an extraordinarily rare feat.
Either the design and/or sample and/or collection method of the survey was poor.
LBrown - - thanks for the insight, and feel free to visit Backcountry Notes again (and make suggestions for things you might like to see there).
Paul - - good information; Newsweek rated its margin for error at 3.8 per cent, rounded up to 4 per cent, with no separate statement of confidence level.
Another possibility - - polltakers have a "model" which is used to crunch the raw numbers; results are only as good as the model. Rasmussen has a wicked number-crunching model and a poll-taking method which is reasonably free from bias - - done by phone robots. I find that a bit cold, being somewhat of a Luddite at times, but Rasmussen seems to be reasonably accurate over time and place. Quinnippiac seems to do well also. Zogby seems to be having some modelling problems. Newsweek - - actually, Princeton Survey Research Associates International, which did the survey for Newsweek - - described a very rudimentary modelling technique: "Results are weighted so that the sample demographics match
Census Current Population Survey parameters for gender, age, education, race, region, and population density." Voter registration status was self-reported. The survey is taken by persons, not machines, and of course we don't know what bias the questioners may have introduced. So, yes, the survey technique itself is probably subject to substantial error; although a variance of that magnitude still seems out of line.
What I find encouraging about RealClearPolitics is the averaging-out approach; the assumption being that sampling and methodology errors will tend to cancel out by doing this.
What I find disturbing is that a probably-flawed poll like this Newsweek item is simply accepted at face value by the talking heads on the news programs. I watched a bit of the MSNBC treatment (I can only take so much); they might as well have had Brittney Spears doing the "analysis." Of course, it may be that the powers that be at NBC, CNN, MSNBC, and so on know full well that the Newsweek poll is flawed, but don't care as long as it helps their favorite candidate.
In statistic Newsweek's numbers are called an outlier (thank God for Business school.) If you have a bunch of numbers that are consistent with each other (the other polls) and then there is one (newsweek's pool) that is way out of left field, statistician throw it out. It signal something is wrong with it. A lot of things are wrong with newsweek's numbers.
Mccain is not ahead but he is not 15 points behind. These numbers are what they call BS numbers where I come from. The reason for newsweek putting them out could be many. My personal opinion is people who follow polls are the same people who follow trends. They want to know which side to take on an issue. They want to be on the bandwagon when it leaves. 15 point lead is very big. Newsweek is saying that Obama has won and there is no point to even come out and vote for McCain. Since I know BS when I see, I immediately laughed at the numbers when I first saw them.
Without knowing too much about polling & stats (and admitedly a McCain supporter), the Neewsweek numbers show a spike I believe there isn't out there. I think Obama is ahead of McCain in the range of the older polls (+4/5%) but not in the +15% range. I don't know whether the Neewsweek survey time-frame covered 0bama's turn on a dime on electoral financing and his Possum morphing - in case so, those facts would have hurt him pretty bad (perceptions: not on the MSN island but definitely on the continental USA). So, something's amiss here...
First of all Newsweek is owned by The Washington Post and they are completely in the tank and might as well be on the Obama/Axelrod payroll. They refuse to vet him.
Second, Newsweek Sr. Editor, Jonathan Alter also works for NBC news and everyone knows what's going on there.
Polls can be easily skewed by oversampling. This poll had almost 20 % unregistered voters. Voters self proclaimed their party and show less than 25% Republicans. They also don't show which states were sampled.
Don't believe for a minute that Newsweek polls are reliable or reflect reality.
Truth: There is also a connection between Newsweek and MSNBC, which made a big deal of the Newsweek poll. Of course, MSNBC doesn't have that big an audience . . . and maybe that's what is bothering me under the surface: Newsweek and MSNBC are both suffering in sales/ratings and therefore ad revenue - - Newsweek is being bested by Time, and MSNBC is getting hammered by FOX News. The suspicion creeps into my mind that they may have cooked the numbers to boost readership/viewership. As the other posters point out, the strong likelihood is that this poll was an "outlier," a statistical anomaly - - but it sure wasn't presented that way.
On the other hand, it's to some extent like the bad news - good news about the insipid "Baby Alex" ad. "Bad news, Senator McCain . . . that Baby Alex is still running on tv." "So, what's the good news?" "It's running on MSNBC, so nobody's watching it."