Think Tank Credibility

After listening to a podcast with Tristan Harris (co-founder of the think tank, Centre for Humane Technology), I’ve started to take a closer look into think tanks over the past few months. Given the recent Congress Hearings with Tech leaders and upcoming elections in US (and Guernsey where I have resided since 2016), I am slightly fascinated in the role think tanks play in democracy.

As part of the research, I recently came across a brilliant article by Andrea Baertl on the topic of ‘think tank credibility’. Obviously, credibility is crucial for a think tank. To be able to effectively inform policy and practice they need to be and be seen as credible sources of information and advice.

In the current environment- where fake news, fake think tanks, bad and fake research abound- think tanks need to be trustworthy sources of information and advice to their stakeholders.

In Andrea’s article, she provides an annotated reading list of resources that address the concept of credibility and think tanks. Below, I have provided a selection of the key resources to check out for further insight into the changing role of think tanks and the challenges of the new world:

A problematic context: post-truth, bad research and clandestine lobbying

Lewandowsky, S., Ecker, U. K. H.,& Cook, J. (2017, in press). Beyond Misinformation: Understanding and coping with the post-truth era. Journal of Applied Research in Memory and Cognition.

This article discusses the terms post- truth, fake news and misinformation. It outlines the societal trends that gave way to the current misinformation environment: a decline in social capital, growing economic inequality, increased polarisation, declining trust in science, and a fractionated media landscape. It also shows how misinformation influences people and the pervasive effects this can have. It finally discusses how people respond to corrections, showing how difficult changing people’s minds can be, and finishes with recommendations on how to combat misinformation

Leach, M. (2017Research and evidence in the ’post-truth’ era. Institute of Development Studies. Opinion.

This opinion article discusses the role of research and evidence in the current environment in which experts and facts are rejected by some groups. It argues that there is still need for research and evidence, but done differently. Research, evidence and knowledge needs to evidence how and why we need to create a fairer and more sustainable world, and how research can contribute towards that goal.

Gutbrod, H. (2017) Fake news, fake tanks, and the general election: Britain’s democracy under threat?. Transparency International Blog.

The author reflects on the impact that fake news and fake tanks can have on UK elections. He describes how fake tanks have effectively generated false news that are picked up by the main media outlets arguing that, yes, there is cause for concern. Editors and journalists often cannot tell the difference between real think tanks and fake ones (who are usually fronts for lobbyists or other powers) and gives examples of them further propagating fake news. Transparency, he argues, is a useful tool to identify if a think tank is credible or not, and that could and should be used to combat fake news and fake tanks. He finally argues that governments should not fall into the trap of more regulations, as that would stifle existing think tanks, but instead the focus should be on improving the media and ask them to fact check and refuse providing outlets to fake tanks and dark money groups.

The concept of credibility

Rieh, S. Y. & Danielson, D. R. (2007). Credibility: A multidisciplinary framework. In B. Cronin (Ed.), Annual Review of Information Science and Technology (Vol. 41, pp. 307-364). Medford, NJ: Information Today.

Rieh and Danielson discuss the concept of credibility and its relationship with trust, quality, authority and persuasion. They focus on identifying critical concepts and dimensions of credibility and the factors that influence its assessment. The focus is geared towards general communications, web design and information science, but the review of the concept they do as well as the framework proposed are very useful and a good introduction to understanding credibility.

Hilligoss, B., Rieh, S.Y., 2007. Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context. Information Processing and Management 44 (2008) 1467–1484

Based on interviews the authors propose that there are three levels of credibility judgements: 1) Construct, which is the way a person defines or operationalized credibility, 2) Heuristics, which are the rules of thumb that people use to assess credibility in particular situations, and 3) Interaction level, which is how these two interact with the cues elicited by the source. Additionally, they propose that context frames these assessments. This is a very interesting framework and useful to understand how individuals assess credibility. The authors do a very good job of explaining to readers how assessments at every level are made and how different aspects influence credibility judgements.

Policy research, think tanks and credibility

Judis, J.B. (2017) The credible think tank is dead. New Republic.

Judis discusses the ousting of a Google critic at the New America Foundation and argues that donors have corrupted Washington’s policy and research institutes. The author traces the story of think tanks in the United States and shows how they have transformed, and concludes- in an un-optimistic tone- that a reduction of the role of money throughout American politics is needed to revive the older vision of the think tank: carrying out disinterested research.

Mendizabal, E. (2018). Is it all about credibility?. On Think Tanks Article.

This is a reflection article on different aspects that On Think Tanks has focused over the years: governance, business models, transparency, research quality, communications, etc.- and how these different issues all lead to a think tank strengthening and showcasing its credibility.

Doberstein, C. (2017). Whom Do Bureaucrats Believe? A Randomized Controlled Experiment Testing Perceptions of Credibility of Policy ResearchPolicy Studies Journal, 45: 384-405.

Highly recommended research that shows the power of heuristics when assessing the credibility of a source. Doberstein ran an experiment in which participants (government bureaucrats) were asked to read research summaries and assess their credibility, for half or respondents the affiliation/authorship of the content was randomly reassigned. The findings showed that credibility was basically assessed via heuristics and regardless of the actual piece of research: academic research is perceived to be more credible than think tank or advocacy organisation research. The author did this follow up study with similar findings: Doberstein, C. (2017) The Credibility Chasm in Policy Research from Academics, Think Tanks, and Advocacy Organizations. Canadian Public Policy, 43, 4. Both articles can be found in academia.edu and the author also published an abridged version.

Rich, A. (2004) Think tanks, public policy and the politics of expertise. Cambridge University Press, New York.

This book is an excellent introduction to understanding and studying think tanks. Regarding, credibility chapter three “Political Credibility” is highly recommended. Rich analyses the perceptions of think tanks among US congressional staff and journalists (as key actors in policymaking), their views on the influence and credibility of think tanks, and how their visibility and marketing efforts affect their influence and perceptions of credibility.

Stone, D. (2004) Private authority, scholarly legitimacy and political credibility. Think Tanks and informal diplomacy. In. Higgot, R., Underhill, G.R.D., Bieler, A. (2004) Non-State Actors and Authority in the Global System

The work of Stone on think tanks in general is highly recommended. This chapter is very interesting to understand the credibility of think tanks from a political viewpoint. The author describes how think tanks as non-state actors act as policy entrepreneurs on both domestic and international policy domains and contribute to policymaking. Despite not being fully academic actors, they operate within that world as well, which in turn lends them credibility.

Baertl, A. (2018) De-constructing credibility: factors that affect a think tank’s credibility. On Think Tanks Working Paper 4. On Think Tanks

The paper explores the concept of credibility, explaining that credibility is constructed through the interaction of characteristics and actions of an organisation, and the assessment of others in the context within which communication takes place. Stakeholders give (or take away) credibility based on their assessments of the information they have and the influence of the current context. The paper argues that the credibility of a think tank goes beyond the quality of its research, and that there are a common set of factors from which individuals draw from to assess the credibility of a think tank. These are: networks, past impact, intellectual independence, transparency, credentials and expertise, communications and visibility, research quality, ideology and values, and current context.

Ensuring credibility

The following are a selection of articles that focus on specific factors that relate to the credibility of think tanks, and give ideas on how think tanks can ensure and showcase it.

Research quality

Méndez, 2012. What’s in good? Evaluating IDRC Results: Research Excellence. IDRC

Although a little dated now- therefore missing the latest research quality frameworks (REF and RQ+)- this is an excellent overview of the literature on research quality and excellences, as well as some of its gaps. The article discusses the elusive concept of research excellence or quality and demonstrates that there are no common definitions, but several commonalities in it. This document is included in this credibility reading list because research quality is at the core of a think tank’s credibility and a needs to be reflected on before moving any further on to assess its credibility.

McLean, R. (2018)Credibility and research quality- time for a paradigm shift? On Think Tanks Article.

The author discusses the RQ+ framework of the IDRC as a way forward to measure and ensure the quality of research and lead to its credibility. The article starts by questioning impact indicators an argues that they are essentially a proxy indicator of how popular the publication is, and that they say very little about the importance of the topic, the quality of the research or their impact on policy or practice. The framework developed by the IDRC is a way to ensure all of this, which would in turn lead to credible research.

Transparency

Gutrod, H. (2018) Credibility- the role of transparency. On Think Tanks Article.

This short article reflects on the relationship of transparency and credibility, arguing that transparency does not guarantee credibility for a think tank, but it is a necessary step towards achieving it. Gutbrod says transparency can also contribute to the debate on credibility- after all, every organisation has particular interests, motivations and affiliations. The problem for the credibility of the organisation arises when these are hidden.

Bruckner, T. (2017) Think tanks, evidence and policy: democratic players or clandestine lobbyists?. LSE Impact blog

Think tanks are thought by some to conduct sound policy research aimed at enriching policy discussions, and by others as covert lobbyists financed by corporations to suit their needs. Bruckner discusses the role that transparency can (and is) playing in establishing which think tanks are legitimate and credible organisations and which are not.

Communications and credibility

Fogg, B.J. (2002). Prominence-Interpretation Theory: Explaining How People Assess Credibility. A Research Report from the Stanford Persuasive Technology Lab, Stanford University

Prominence-Interpretation theory proposes that in order to assess the credibility of something (in this case, websites) people first need to notice something (prominence) and then make a judgement made about it (interpretation). People only base their credibility judgements on aspects that they notice. This highlights the importance of good communications as part of a think tank’s credibility strategy.

Williams K (2018). Three strategies for attaining legitimacy in policy knowledge: Coherence in identity, process and outcome. Public Admin. 2018;1–17.

The author outlines three types of coherence that enhance the legitimacy of organisations based on an analysis and interviews to individuals from 12 development research organisations. Williams argues that the credibility of knowledge production organisations is enhaced by demonstrating a coherent identity; showing adequate processes for maintaining independence, integrity and transparency; and creating the ‘right’ products that impact on their audiences.

Schwartz, J, 2018 Credibility and think tank communications. On Think Tanks article.

Schwwartz argues that credibility is at the heart of all effective communications (as without credibility the message will not be adequately received by the source). The argument, is that to build its credibility, a think tanks needs to: be evidence based, and showcase this in its communications; be brand-conscious and build consistent arguments over time, and; be useful, working with and for their audiences, and making its ideas easy to find and use.

Westerman, D., Spence, P. R. and Van Der Heide, B. (2014), Social Media as Information Source: Recency of Updates and Credibility of Information. J Comput-Mediat Comm, 19: 171–183. doi:10.1111/jcc4.12041

This very interesting article analyses how information available on social media impacts the perceptions of credibility. Although not directly focused on think tanks, it does offer very interesting lessons for them. The findings showed that recency of tweets positively impacts the credibility of the source, although this process is not automatic and is mediated by cognitive elaboration.

Newman, E. J., & Schwarz, N. (in press). Good sound, Good Research: How the audio quality of talks and interviews influences perceptions of the researcher and the research. Science Communication

Although the focus of this research is not think tanks per se, the implications of the findings are important for think tank communications. The authors ran an experiment in which they presented identical conference talks in high and low audio quality and asked people to evaluate the piece. People evaluated the research and researcher less favourably when they were presented with the poor audio quality audio. This has important implications for think tank communications, as efforts in curating the quality of their pieces will have larger implications in how their audiences perceive them.

Flanagin & Metzger 2017. Digital media and perceptions of source credibility in political Communication. In Kathleen, K.K & Jamieson, H. (2017) The Oxford Handbook of Political Communication. Oxford University Press

The authors compare the credibility of digital versus traditional channels, and the dynamics and nature of political information online. They also reflect on the following aspects: the link between credibility and selective exposure, the potential for group polarisation, and the role of social media in seeking and delivering credible political information. They analyse these issues and offer challenges and opportunities that can be used by think tanks to better engage with the public.

Corporate Governance And Innovation: 10 Questions for Boards

To be successful, companies must be led by leaders – the CEO, top executives and board of directors – who are deeply and irrevocably committed to innovation as their path to success. Just making innovation one of many priorities or passive support for innovation are the best ways to ensure that their company will never become a great innovator – Bill George, former CEO and Chairman of Medtronic and Professor at Harvard Business School

A few weeks back I gave a talk focused strategic response, adaptability and innovation in a COVID world to an audience of NEDs mainly focused on off-shore financial services (FS) sector firms.

Given how highly regulated and risk-adverse many off-shore FS firms are, unsurprisingly questions were focused on the challenges of balancing risk vs innovation, how to make change happen at board level, and how to navigate director duties.

It got me thinking….

What are the ways for boards to show their real, concrete commitment to innovation and technology, and its governance?

As I discussed in my talk, all global business and technology trends point in the same direction: there is a need for more proactive and far-sighted management of innovation. Innovation for business reinforcement and growth – and for transformation in particular – are, of course, the prime responsibility of top management. Innovation governance – a holistic approach to steering, promoting and sustaining innovation activities within a firm – is thus becoming a critical management imperative.

Boards of directors also need to be more than just observers of this renewed management interest in innovation, because so much is at stake in an increasingly pervasive digital and COVID world. In a growing number of industries and companies, innovation will determine future success or failure.

Of course, boards do not need to interfere with company leaders in the day-to-day management of innovation, but they should include a strong innovation element in their traditional corporate governance missions. For example:

  • Strategy review;
  • Auditing;
  • Performance review;
  • Risk prevention and, last but not least;
  • CEO nomination.  

It is therefore a healthy practice for boards to regularly reflect on the following questions:

  • To what extent is innovation, broadly defined, an agenda item in our board meetings?
  • What role, if any, should our board play vis-à-vis management regarding innovation?

To facilitate their self-assessment, boards should answer a number of practical questions that represent good practice in the governance of innovation. According to various innovation governance experts, including Professor Jean-Phillipe Deschamps at IMD Business School and author of Innovation Governance: How Top Management Organises and Mobilises for Innovation (2014), below are ten good-practice questions and perspectives to incorporate into any board evaluation:

1) Have we set an innovation agenda in many, if not most, of our meetings?

Board meetings are always crowded with all kinds of statutory corporate governance questions, without talking about the need to handle unexpected events and crises. So, unless innovation issues are inserted into the board agenda, they won’t be covered. It is a good practice to include innovation as a regular and open agenda item in at least a couple of board meetings per year. It should also be a key item in the annual strategy retreat that many boards set up with the top management team. Many of the following questions will provide a focus for this open innovation agenda item.

2) Do we regularly review “make-or-break” innovation projects?

In some industries, like pharmaceuticals, automotive, energy and aerospace, company boards regularly review the big, often risky innovation projects that are expected to provide future growth. They also do so because of funding issues – some of these projects may require extraordinary and long-term investments that need board approval. But in other industries, boards may be only superficially aware of the new products or services under preparation. Arguably, there may be several projects that are still small in terms of investments but could become “game-changers,” and it would be wise for the board to review them regularly in the presence of R&D leaders and innovators.

3) Do we regularly review and discuss the company’s innovation strategy?

Boards are generally aware of – and discuss – the company’s business strategy, particularly when it involves important investments, mergers and acquisitions and critical geopolitical moves. But what about the company’s innovation strategy (if it exists and is explicit, which is not always the case)? There are indeed important decisions that might concern the board in a company’s innovation choices because of their risk level and impact. Think of the adoption of innovative new business models, the creation of totally new product categories, or the conclusion of important strategic alliances and partnerships for the development, introduction and distribution of new products. Management’s adoption of a clear ‘typology’ of innovation in its board communication would definitely facilitate such reviews and discussions.

4) Do we regularly review and discuss the company’s innovation risk?

Boards usually devote a significant amount of time to risk assessment and reduction. But their focus tends to be on financial, environmental, regulatory and geopolitical risk. Innovation risk may be underestimated, except in the case of large projects involving huge investments and new technologies. But internal innovation risk is not limited to new project and technology uncertainties. It can be linked to the loss of critical staff, for example. Innovation risk can also be purely external. Will competitors introduce a new disruptive technology that will make our products and processes obsolete? Will new entrants invade our market space through different, more effective business models? Will our customers expect new solutions that we have not thought about? Assessing innovation risk is critical to avoid what Ravi Arora calls “pre-science errors” – underestimating the speed and extent of market or technology changes – and, even worse, “obstinacy errors” – sticking to one’s solution too long after markets or technologies have changed. It is the duty of the board to prevent such errors.

5) Do we set specific innovation goals for management?

Boards often exert strong pressure on management by setting performance goals. But most of these goals tend to focus on financial performance: top and bottom line growth, earnings per share, capital utilization ratios, etc. Some companies add other goals to focus management’s attention on worthwhile new objectives, such as globalization or sustainability. But what about innovation if it increasingly becomes a growth driver? A number of highly innovative companies have indeed included innovation goals in the CEO’s balanced scorecard. One of the most commonly found is the percentage of sales achieved through new products, typically products introduced in the past few years. But there are many other innovation goals to incite conservative management teams to take more risk – for example, the percentage of R&D spent on high risk/high impact projects. Innovation goals are interesting because they actually determine much of the company’s long-term financial performance. It is therefore good practice to discuss these goals with the management team and retain the most meaningful ones.

6) Do we review innovation management issues with the CEO?

Most sustained innovation programs raise many issues. Some of them are managerial – how to keep innovators motivated and reward them? Others are organizational – how to decentralize R&D to tap the brains of our international staff? Many deal with intellectual property – how do we practice open innovation while maintaining our IP position? Others deal with strategic alliances and partnerships – how do we share the efforts and risks of new ventures with our partners? And there are many more issues. The question boards should ask is: Are we aware of the most acute issues that management faces as it steers the company’s innovation program? The board’s mission is of course not to interfere and become too deeply involved in these innovation issues. However, its mission is to keep informed and help the CEO and top management team reflect on their options. This is why it is essential to keep a short open agenda item – “innovation issues” – in board meetings with a specific innovation agenda. 

7) Do we expect management to conduct innovation audits?

Many companies embarking on a major innovation boosting program rightfully start with an internal audit and, sometimes, a benchmarking exercise against best-in-class competitors. Where are we deficient in terms of strategy, process, resources and tools? Do we have the right type of people in R&D and marketing, and do we tap their creativity effectively? Do we cover all types of innovation, i.e. not just new technologies, products and processes? Are our projects well resourced and adequately managed? Are they under control? How good is our innovation climate? These audits are extremely effective for highlighting priority improvement areas, and it is therefore good practice for the board to suggest that management undertake such audits and keep them updated. These audits will provide the board with a rich perspective on the company’s innovation performance issues.

8) Do we expect management to report on innovation performance?

This question is directly related to the questions on innovation goals (5) and innovation audits (7). Once innovation goals have been set and an audit conducted, it will be natural for the board to follow up and assess innovation performance. To avoid having to delve into too many details, innovation performance reviews should be carried out once or twice a year on the basis of a reasonably limited number of innovation performance indicators. Good practice calls for these indicators to cover several categories. A couple of them should be lagging indicators, i.e. measuring the current result of past efforts – the percentage of sales achieved through new products being one of them. A couple of others should be leading indicators, measuring the level of efforts done today to ensure future innovation performance – for example, the percentage of the R&D budget devoted to high risk/high impact projects mentioned above. One or two others should be in the category of in-process indicators – the most usual measure being the percentage of projects managed on schedule and on budget. Finally, it is always interesting to include a learning indicator to measure the reactivity of management and its ability to progress on key issues.

9) Do we know and occasionally meet our main corporate innovators?

Nothing conveys a company’s strong innovation orientation better than a visit by the entire board to the labs and offices where innovation takes place, both locally and abroad. Such visits, which are often carried out by innovative companies, have a dual advantage. They enable board directors to be aware of the real-world issues that the company’s innovators face, and they provide them with a good understanding of the risks and rewards of innovation. They also motivate the frontline innovators, who often lack exposure to top management.

10) Do we take innovation into account when appointing new leaders?

This last question is probably the most important. The nomination of a new CEO is undoubtedly one of the board’s most visible and powerful contributions to the company. It can herald a new and positive era for the company if the capabilities of the CEO match the company’s strategic imperatives. But it can sometimes lead to damaging regressive moves if the values of the new CEO are innovation-unfriendly. Management author Robert Tomasko notes that CEOs often fall into one of two broad categories: fixers and growers. The former are particularly appreciated by boards when the company needs to be restructured and better controlled. But fixers often place other values and priorities ahead of innovation. Growers are more interested in innovation because of its transformational and growth characteristics. This does not mean that boards should always prefer growers over fixers. There are times when companies require drastic performance improvement programs and an iron-handed CEO is needed. The board should, however, reflect on the impact the new CEO will have on the company’s innovation culture and performance. This is why it is so important to look at the composition of the entire management team. How many growers does it include and in what position? Will these senior leaders be able to counteract excessive innovation-unfriendly moves by the new fixer CEO?    

If you are interested in this topic, I suggest starting with Professor Jean-Phillipe Deschamps book Innovation Governance: How Top Management Organises and Mobilises for Innovation (2014)