Washington Post columnist Dana Milbank is as lousy a statistician as he is a political commentator.
Milbank claims journalists have been harder on President Joe Biden in the first year of his presidency than they were on former President Donald Trump in his last.
To back this claim, he points to a report prepared by the group FiscalNote, whose “artificial intelligence” reviewed some 200,000 news articles to determine “sentiment” based on variables including adjective placement.
The group’s findings, the Washington Post columnist writes, confirm his worst fears: “My colleagues in the media are serving as accessories to the murder of democracy.”
His thesis is obviously bunk — as if a computer could ever hope to gauge “sentiment” accurately depending on adjective placement. However, after a little additional digging, most notably by Don't Walk, Run! Productions, it turns out Milbank’s theory is even faultier than initially suspected.
For starters, FiscalNote’s data include a combined 514 Trump-related articles published last year by CBS News and MSNBC. However, the data include exactly zero Biden-related articles published this year by the networks. Either CBS and MSNBC simply forgot to cover the Biden White House, or FiscalNote omitted both networks’ coverage from its research.
Second, when it comes to gauging “sentiment,” the group explains that a truly neutral article is ranked at 0.000. The higher the number, the more positive the coverage; the lower the number, the more critical the coverage.
However, from a brief reading of the data, it’s clear there’s no consistency whatsoever to the research.
On Jan. 21, 2021, for example, CNN published an article titled “READ: Joe Biden's inaugural address.” There’s nothing to the article. No opinion whatsoever. It’s simply a raw transcript of the president’s inaugural speech.
Yet, the “AI” ranks the article a +0.260 on the "sentiment" scale.
Newsmax likewise published a wholly neutral article in January titled “Text of President Joe Biden's Inaugural Address.” There’s nothing to the story but the text of Biden’s inauguration speech.
The “AI” ranks the Newsmax article a +0.355.
It gets worse.
On April 29, 2021, CNN published another transcript under the neutral headline “Read President Joe Biden's first address to Congress.” Again, there’s no opinion or commentary included. It’s just the text of the president’s speech.
The “AI” ranks the article a +0.2688.
The Washington Examiner, meanwhile, did the exact same thing as CNN, publishing only the raw text of Biden’s address to Congress. The “AI” ranks the Washington Examiner article a -0.1010.
This is to say nothing of the fact that the “AI” inexplicably hands out positive rankings to critical coverage and critical rankings to positive coverage.
A January ABC News puff piece heralding Biden and Vice President Kamala Harris’s first tweets from the White House registers at -0.0701 on the “sentiment” scale. Meanwhile, an Associated Press report bearing the none-too-flattering headline “Immigrant detentions soar despite Biden’s campaign promises” clocks in at +0.0333.
An Associated Press fact-check titled “Biden distorts Virginia’s electoral history,” which concludes that the president did indeed distort Virginia’s electoral history, ranks at +0.1790. At the same time, an ABC puff piece hailing Biden’s signing of an anti-Asian hate crime bill, which the newsgroup describes as a “significant break” in “partisanship,” ranks at -0.1632.
It gets worse.
The data also contain multiple duplicates. This is in and of itself a problem, as duplicate stories will naturally throw off the weighted averages. But it’s actually worse than that. As it turns out, the “AI” gives different “sentiment” grades to the exact same stories.
An ABC report titled “Biden lands overseas without deal after House delays infrastructure vote” is listed twice in the data spreadsheet. The first time it appears, it's ranked at -0.1400. The second time it appears, it's ranked at -0.0824.
The Associated Press article “Pro-Biden groups to spend $100 million on August ad blitz” appears twice. The first time it appears, it's ranked at +0.1581. The second time it appears, it's ranked at +0.0540.
A New York Times opinion article titled “Biden Should Finish the Wall” appears on three separate occasions.
Its ratings are -0.3743, -0.1726, and -0.3136.
Also, a number of articles included in the “Articles Analyzed-Biden” data subset have exactly nothing to do with Joe Biden, including an Associated Press report titled “Lee Hart, wife of 1984 presidential hopeful Gary Hart, dies.”
The name “Joe Biden” doesn’t appear even once in the story. Yet, it ranks at -0.3617 on the “sentiment” scale as a negative against the current president.
One final thing: The “Articles Analyzed-Trump” data subset is supposed to review reports published between January and November 2020. However, the publication dates as reported by FiscalNote do not match the actual publication dates for dozens of stories.
FiscalNote claims the ABC story “GM announces jobs, electric vehicle after Trump criticism” was published on Jan. 1, 2020. However, it was actually published on March 22, 2019. The data also claim a Chicago Tribune editorial titled “Trump trifecta: He backs Hong Kong, supports democracy, annoys China” was published on Jan. 1, 2020. In actuality, it was published on Nov. 28, 2019.
And so on.
Put simply, the data are trash. Milbank either is incredibly lazy, illiterate, or a mixture of both.
It’s no surprise he’d get it so wrong. This is to be expected.
But what are we to make of FiscalNote and the supposedly straight news hosts who uncritically parroted the group’s findings? Did no one actually read the data?