The USAGM Audience Increase: Less Startling than Meets the Eye

A February 2019 email newsletter from John Lansing, CEO of the U.S. Agency for Global Media (new name for the Broadcasting Board of Governors) proclaimed that “2018 was a banner year” for USAGM. He noted that “USAGM programming was consumed by 345 million adults weekly worldwide—including radio, television and internet—an unprecedented year-on-year increase of 67 million from 2017.” This refers to the USAGM’s Audience Impact Overview for 2018 issued November 2018. 

That would be a 24 percent increase in one year. As an international broadcasting audience research analyst for 42 years (10 in academia and 32 for the Voice of America and its parent agencies), a 24 percent “year-on-year” increase gets my notice. Why there was such a dramatic increase? Did a major war break out, causing people to seek information from abroad? No, nothing beyond the ongoing simmering regional conflicts. Did one of the USAGM entities score a new television affiliate, with a prime-time slot in a populous country? If that had caused the increase, I am sure it would have been highlighted in the report.

According to the press release (apparently no longer at the USAGM website but available here), “The measured weekly audience grew to 345 million people in FY 2018, from 278 million people in FY 2017, an unprecedented increase of 67 million.” The reader could easily interpret this as a sudden one-year audience increase of 67 million, but it is actually a change in the “measured” audience. 

There really is no “year-on-year” increase, because that would require surveys every year in every country reached by the USAGM entities. There certainly is not enough budget for such an undertaking. Increases, or decreases, occur when new data replace old data. Some of the old data can be several years old. So what really happened is that new data have replaced older data, resulting in a new global estimate.

The USAGM report points to China as a key contributor to the audience increase.  The previous USAGM (then BBG) survey was around 2014. With a typical sample size of 3,000, a positive response from one person interviewed can represent about 400,000 members of the audience.

The best solution would be for international media audience research to be conducted by a third party, or by a consortium of the players.

But beyond that, it is not so much the gap in years involved, but also a change of methodology that would explain the increase, or what appears to be an increase, in the USAGM audience. This paragraph from the USAGM report requires some concentration but it does explain some important factors:

These figures are considerably higher than previous estimates for both audiences, and much of the increase is attributable to new questions about the use of individual programs and blogs that also were identified by the RFA [Radio Free Asia] or VOA [Voice of America] brand. When the audience for the USAGM broadcasters is measured by the VOA or RFA brand only, without mention of any individual programs, the weekly reach of USAGM brands combined was 1.7 percent of Chinese adults. This figure is most methodologically comparable to previous estimates of USAGM audience, which was measured at 0.3 percent in 2014.

So the new methodology asks about all the program and blog names, and not just the top-level brand names of VOA and Radio Free Asia. That would explain an apparent increase of audience in China from 0.3% in 2014 to 6.2% in 2018—an increase that strains credibility. But even by the “most methodologically comparable” process—using VOA and RFA brand names only—there was an increase from 0.3% to 1.7%—a whopping 566% increase. What caused such a huge uptick in the audience size? A caption in the report offers this explanation: “VOA & RFA have expanded on digital and social media platforms in China and achieved dramatic growth in a country where they are censored on mainstream platforms”

That is possible. But as the skeptical research analyst, I have seen marked increases or decreases from one survey to the next that were attributable to something in the methodology, such as the wording of the questionnaire, the conduct of the interviewers, or the selection of the sample.

As for asking about program names and blog names, this becomes a trap for false positives, because there is very likely a program or blog with the same or similar name inside the target country. Yes, the report says the programs and blogs “also were identified by the RFA or VOA brand.” But when the respondent hears the question “Have you read the blog Trans-Pacific Business (fictional example) on Měiguó zhī yīn (VOA),” he/she might think, yes I have read that blog, but I didn’t know it is from something called Měiguó zhī yīn. The respondent may actually have read a blog with a similar name from a domestic source.

A possible example of the program-name false-positive trap might be USAGM’s claim of a “market-leading audience share within Iran.” If true, it would mean that VOA Persian and Radio Farda have a larger audience in Iran than BBC Persian or even the popular Manato TV or the entertainment-oriented Pars TV. In Iran, by anecdotal discussion and reaction from the Tehran regime, BBC is mentioned much more often than the U.S. international broadcasting outlets.

Before I retired from VOA’s audience research office in 2017, a survey in Iran (actually using telephone calls into Iran from a nearby country) showed a large increase in audience for VOA Persian and even indicated that VOA’s audience surpassed that of BBC in the country. Fascinated, I dug in to the data set. The bulk of the increase was attributable to two VOA Persian TV programs: Early Report and Late Report.

Well, of course there are programs on Iranian domestic television with very similar names to that. I went to the Islamic Republic of Iran Broadcasting website, looked at the schedule grid, then enlisted a Persian-speaking friend to verify this. Also, the purported viewers of VOA’s Early Report and Late Reports did not match the characteristics to those of the international broadcasting audience in Iran. For example, many of them don’t have satellite dishes, nor do they use any other foreign news sources.

False positives from program names that resemble program names from domestic broadcasting can result in larger audience estimates, so this phenomenon might be considered a feature rather than a bug. Could audience increase by dint of false positives occur elsewhere? A prominent addition to USAGM’s output to Russia is Current Time, an online video news service that is a joint project of RFE/RL and VOA. “Current Time” in Russian is Nastojaŝee Vremya. Anyone familiar with Soviet and Russian television knows about flagship domestic news program Vremya.   

International media generally do not challenge domestic media in the target country for audience size. International media audience research therefore is most useful when it compares the audience sizes of international outlets trying to reach audiences in the target country. Beyond comparing audience size, it can explore why some international media outlets are more successful than others. There are different approaches to international media, e.g., level of commitment to independent journalism, consolidation versus plurality of resources, etc., resulting in a great field experiment. 

This requires identical methodology for all the players. The best solution would be for international media audience research to be conducted by a third party, or by a consortium of the players. And, for the sake of transparency, instead of cherry-picking results for press releases, the datasets and questionnaires should be made available to scholars and researchers. This could be done in a full-day deep dive involving researchers inside the international media business as well as outside observers, with lots of questions and answers.