Guide
By Kat Hart, Analyst at Overton
This guide by staff at Overton brings together pro tips for using the Overton Index to create research with impact.
There are lots of ways throughout the research cycle you can use Overton to influence policy, from the earliest stages of planning your project, all the way through to disseminating your published outcomes. In this guide we show how you might find evidence gaps identified by policymakers and understand how your research might fit within current policy debates.
We hope there will be something for everyone here – researchers with no previous knowledge of policy impact as well as experienced policy engagement specialists looking for ‘super-user’ inspiration!
To make effective policy, it must be based on good quality evidence. One of the best ways policymakers can collect this evidence is to utilise the knowledge and experience of scholars and other experts in the field. Indeed, as our world faces increasingly complex global challenges, there’s a growing call for researchers to engage ever more directly with policymakers to assist in finding solutions.
On top of this, in many knowledge economies there are increasing expectations by governments and other research funders for researchers to showcase the influence of their work outside academia. Formal citations of scholarly outputs or mentions of researchers by name in policy documents can provide concrete examples of your ‘reach’ and the real world salience of the scholarship – an increasingly important research assessment indicator. Examples of policy impact are helpful (and sometimes essential) when writing grant proposals, reporting on grants, applying for promotion and developing impact case studies.
Finally, engaging with policy helps further the core mission of a scholar’s role – to advance research.
We recommend that researchers and research managers plan for impact, rather than just evaluating it after the fact. This means making sure that the research is relevant, interesting and readily consumable by the end user (in this case decision-makers). We also find it’s helpful to understand that your contributions to policy debates don’t have to be specifically tied to findings from a given research project – to policymakers, your ‘general expertise’ can be just as useful. This kind of engagement can be found by looking for informal ‘mentions’ in Overton.
There are lots of ways of approaching this, and the approach you take will be highly dependent on the discipline, your goals and the public policy context. So we’ve outlined a few ideas that you might want to try out and adapt to your own needs.
There are several ways in which policymakers might ask for evidence, or outline an evidence requirement which – if relevant – could be used as the basis for your next project. This could include evaluatory policy documents, areas of research interest, or rapid evidence reviews. As a starter for ten, running these terms through the Overton database, combining this with your area of interest or filtering to topics relevant to you may be helpful here.
For example, if I was looking to influence policy within the US, I might search for documents published recently by US federal and state government bodies, containing phrases like “evidence gap”, “gap in the evidence”, “further research is needed” or “further research is required” – by default, Overton will look for the search terms within the full text of policy documents. If I wanted to be more specific about this, I could filter it by topics that relate to my field of research e.g. housing or limit the search to more recent publications only.
Search results for “Housing”
Although they may not have made their evidence requirements clear in the body of their publications, speaking directly to policymakers can be a really useful way of finding this out. You can use Overton to build a list of policy organisations that are already engaged in your subject area . If there’s an author name and email address in the policy document, even better!
To do this, you can use Overton as a grey literature searching tool to build a cohort of policy documents that relate to your area of interest.
Search for more specific terms to your specialist area(s) e.g. “electric vehicle”
Use the tags feature to create a subset of relevant documents
Explore the policy documents to understand how the issues are perceived from the policy perspective
Shape your outreach and strategy, and reach out to policymakers to discuss their current priorities and evidence needs
When policymakers don’t outline their evidence requirements explicitly, we need to dig a layer deeper to understand the existing scholarly evidence base that is currently used by policymakers and find the evidence gaps this way.
Using the cohort of policy documents that you defined in the previous section, you can search for scholarly work cited by these publications.
Click “see the scholarly research these documents cite”
You will then see a list of scholarly articles cited by these policy documents, in the original context.
You can then assess this evidence. Has there been significant development in this field in recent years? Did these papers contain any recommendations as to future research directions? Did the policymaker pick up on any of this in the context of where the research has been mentioned or cited in the document? Are these ‘evidence gaps’ something you’re interested in working on? Do you already know any of the researchers whose work was cited, and who are their co-authors?
It helps to understand how issues and topics are talked about in policy circles, which is often quite different to how they are referred to in academia.
Understanding policymaker’s agendas and the terminology they use helps you to present your research in a way that makes it obvious that they need to take note.
For example, your specialism might be “health inequality in relation to respiratory issues”, but the policymakers might be talking about “improving the quality of low income housing” or “lowering air pollution in city planning”. In this example, we might run a search of policy documents related to “air pollution” using Overton’s Topics filter, and published by governments in South America.
We can then generate a report to see which countries are publishing the most on this topic and the trend in the number of policy documents published over time. We can also read through the underlying policy documents to understand the themes and issues in more detail.
Using a different example, we can see how combining filters can narrow your search down further. Perhaps I’m a researcher looking at how climate change affects health. Let’s search for UK Government policy documents related to the topic of “health” and also contain the phrase “climate change”. 73 documents were published in 2010, which increased to 146 in 2020. This might indicate that the relationship between climate change and health has been getting higher on the policy agenda of the UK Government for over a decade.
This may suggest that it is a growing area of interest and is unlikely to go away anytime soon. Combined with the fact that more than 75% of this sub-set of documents contain a citation to something else, we could infer that this a rapidly evolving area with a fairly healthy appetite for either iterative development (cites other policy docs or research) or evidence-based research (cites lots of scholarly articles).
Remember, policy engagement is about relationships. You can use the knowledge gained from the previous tips to study the dynamics in between the citing policy documents and the cited research, and start to start planning your outreach strategy. It might be worth seeing:
Past activity don’t necessarily indicate whether a topic will be important in the future – it might be that policymakers don’t know about it yet.
Policy trends tend to reflect real world events in real time, whereas scholarship can be far more forward looking. There’s also resistance to the idea of ‘jumping on a bandwagon’ or twisting your research to fit the current zeitgeist to ‘chase’ impact.
We know that the use of science in policy is increasing in general, and we also know that innovation is key to solving major social challenges. In fact, you should consider the possibility that you are ahead of the curve and publishing on topics that will soon be recognised as pressing!
We believe policymaking should be evidence-based and transparent – we think this is the best way of ensuring policy decisions have a positive impact on the people they affect. We aspire to bridge the gap between the academic and policy worlds.
Through the Overton platform we make it as easy as possible for researchers to understand their influence on the policy process. Our flagship product is the world’s largest database of policy and grey literature. We collect millions of documents from policy sources all over the world (including governments, IGOs, think tanks and NGOs) and pull out the evidence that they used in the publication – we identify references to the research, people, organisations and other policy documents that are cited or mentioned.
The platform can be used as a research tool, for finding policy and grey literature, and as a tracking tool to find and evidence mentions of individual or institutional ‘impact’ in policy. Importantly, this data allows you to analyse how influence works in the policy sphere, to identify trends and extract learnings about the use of evidence. This can help you in your pursuit of policy impact.
Fill out the form below to get a free trial of the platform, and explore the analytical approaches outlined by Kat.
by Kat Hart
Analyst at Overton
Kat helps to find stories in our data. Her background is in Higher Education management, becoming interested in the policy-research interface as a knowledge broker on the Capabilities in Academic-Policy Engagement (CAPE) project. In her spare time, you’ll find her working towards her PhD project at the University of Nottingham, themed around river management policy.
We’ve launched a new database of Areas of Research Interest to improve government’s access to external expertise, and help the research community better understand policy priorities.
Overton is the world’s largest searchable index of policy documents, guidelines, think tank publications and working papers.
Copyright Overton.io 2023 All rights reserved
