Challenges in Measuring Public Diplomacy

To be an effective public diplomat, you need a well of sufficient soft power resources on which to draw. There is no canonical definition of public diplomacy (PD), but the official practice of it involves using informational, educational, and cultural tools to engage with international audiences to advance foreign policy goals. For the United States, PD officials often design strategies and employ tools that leverage what makes America popular and attractive. These tools include social media platforms that help build virtual networks with foreign citizens, and/or educational and cultural exchanges that ensure that those citizens have a direct and authentic experience with Americans and/or the United States that can help them build a more nuanced understanding of the country. PD activities also encourage American citizens to have a relationship with the world that is deep and enduring, so that we can better understand people who are unlike ourselves.

Relationships, however, are inherently complex. They can take many different forms over many years. The empathy one country has for the population of another can ebb and flow. In the international system, it can acutely depend on events and the people in power. While we now have a framework to understand how a country builds and sustains soft power, assessing the impact of public diplomacy activities that rely on that soft power over the very long term has been more elusive.

For the U.S. government, there is no shortage of PD outputs to boast of. As of June 2017, there are more than 90 educational and cultural programs; 450 U.S. embassy and consulate websites with millions of followers; 700 American Spaces, or hubs for foreign citizens to gain information about – and interact with some dimension of – the U.S.; and more than 450 expert speakers dispatched abroad to engage foreign audiences on various topics about the United States. Of the more than one million U.S. sponsored exchange program alumni worldwide, 485 of them are former or current heads of state. This costs just $1 billion, less than 2% of the combined diplomatic and development budget for the U.S. – a budget that is a fraction of what the U.S. spends on defense spending.

But what does all this mean? What are the outcomes?

This work is propelled by a sense of faith that for the American economy to grow, for our borders to stay secure, and our culture to become richer and deeper, we must build these global relationships and networks. The outcomes may not be immediately tangible, but they are nevertheless essential for good statecraft and our national security. For some though, including Members of Congress, this conviction is not enough: this $1 billion investment in American national security needs to create considerably high and clearly measurable returns. But whether you are an advocate for or sceptic of public diplomacy, there is incredible value in creating systems of research and evaluation to understand public diplomacy’s impact.

One can measure the impact public diplomacy has on the U.S. economically. For instance, one program alone, the International Visitor Leadership Program (IVLP), contributes $52 million to the American economy. In addition, nearly one million foreign students attending American higher education institutions, a pillar of American soft power, are often supported through roughly 400 EducationUSA advising centers worldwide. These international students contribute $30.5 billion to the American economy. These programs are undeniably essential not just for the tuition money it injects into the coffers of higher education institutions and the wider economy, but also for the inherent value foreign students’ talent, perspectives, and capacity for innovation bring to the United States. There are many other benefits foreign students bring to U.S. colleges and universities – deeper understanding of religions, civilizations, and cultures that enrich American students’ education, making them sharper, expanding their own thinking, and ultimately ensuring they are better prepared to work in international markets.

But how does one measure the networks and influence that public diplomacy action can create?

In recent years, the U.S. Department of State, the official home of official public diplomacy activities, has been building its capacity to find better, more systemic and rigorous methods to understand foreign audiences and the impact American PD work has on them. Yet the bureaucratic cultural challenge in building this capacity is tremendous. Bureaucracies are infamously risk-averse and are reluctant to embrace systems that would evaluate and scrutinize their work for fear that any admission of setbacks would not make programs stronger in the long-run, but any lead to the eradication of programs altogether. Leadership needs to communicate the importance of not just thorough audience research, aligning PD activities with foreign policy goals, and setting clear objectives from which to measure the outcome – they also need to encourage risk-taking and an open, non-defensive way of analyzing its results.

Leadership in any public diplomacy operation needs to communicate that impact evaluations are essential, while also encouraging risk-taking and an open, constructive way of analyzing its results.

Yet research and evaluation is a specialty, which requires real quantitative and qualitative research skills not common among official and unofficial practitioners of public diplomacy. Social and computer scientists need to be enthusiastically welcomed into PD communities to help determine frameworks to assess which strategies and tactics work in complex environments to build trust and understanding, and which ones do not. One positive pathway for such complicated research is from the Broadcasting Board of Governors (BBG), the United States’ international media agency.

The BBG’s Impact Model uses a dozen fundamental – and two dozen optional – indicators to assess their broadcast programming’s impact. The data that is collected for this includes metrics from their digital platforms, audience size, public polling and representative surveys, focus groups, and anecdotes. They look for indicators unique to the context in which BBG’s networks are operating, and can assess how programs are performing. Yet creating and maintaining such a model requires rigor and a commitment to studying the impact of programs over decades. This work is feasible, but to be consistent, public diplomacy professionals need time, resources, and training.

However, there are many steps practitioners can take to create new norms of research and evaluation. The best way to build a public diplomacy practice with measurable results is to think of it as a campaign. Before taking any action, you must first complete initial research on the target audience, and then determine the objectives for the outcomes of their desired behavior, the content of the messages you want to convey, the PD tactics in which that message will be delivered, and the methods in which you will evaluate the outputs and outcomes. This is known as the ABCDE marketing model for audience, behavior, content, delivery, and evaluation. The exact methods of evaluation can vary depending on the campaign and its context. But one should find inspiration in the BBG Impact Model in how it uses a mix of quantitative and qualitative indicators. This includes polling, focus groups, in-depth interviews, but also statements from influencers and anecdotes from participants.

There should not be a one-size-fits-all method to understanding the impact of public diplomacy, as it will require a mix of them. And there must be a long-term commitment to collecting the various data points that reflect the complexity of public diplomacy work and the relationships they try to create and maintain. Simultaneously, leadership in any public diplomacy operation needs to communicate that impact evaluations are essential, while also encouraging risk-taking and an open, constructive way of analyzing its results.

The success of any country’s public diplomacy efforts ultimately rests on its reserve of soft power. Soft power is reflective of the mosaic of people and institutions that make up a country and its collective creativity, ideas, innovation, and progress. To develop the kind of empathy in international affairs we need to mitigate conflict, a country’s official public diplomacy efforts to build these relationships must feel authentic to the receiver. But by creating better systems to strategize and measure public diplomacy’s impact, we can better understand foreign audiences, how they relate to us, and calibrate and re-calibrate how to connect with them for the very long term. 

Note from the CPD Blog Manager: This piece originally appeared in the 2017 Soft Power 30 report.

Photo by arielrobin I Pixabay License