What Do We Mean When We Talk about Impact?

Impact is something we hear a lot about these days. Most researchers, especially in a global health institution like ours, want their research to have an impact on people’s lives but may not know how to identify, measure or demonstrate it. At the same time, funding and evaluation agencies are increasingly expecting research impact. The problem is that people often do not mean the same thing when they talk about impact.

So, what exactly is impact?

I had the opportunity to attend a course organised by CERCA (Research Centres of Catalonia) to help its research centres get started on the impact journey and, more specifically, to develop impact case studies similar to what is done in the UK. I was a bit sceptical at first, but the course turned out to be fun (thanks, Julie[1]!) and an eye-opener. I am still far from being an expert, but here are some basic concepts that I hope will be useful to all those who have come to me with questions.

Impact means change. And that change needs to happen outside our academic bubble (i.e. in the “real-world”). Easy to say, but not so easy to identify or prove.

Identifying impact

To identify the impact of our research (either retrospectively or prospectively), it helps to ask ourselves three questions:

1. Is society better off because of our research? Hopefully, yes.
2. What changes / has changed? This is the key question. It can be a conceptual change (how we think about something), an instrumental one (changing something), or a change in capacities (the ability to do something). In addition to the nature of the change, it also helps to know its direction: have we increased something that was lacking or low? (e.g. improved health, skills, access, awareness); have we decreased something that was in excess? (e.g. mortality, costs, risk, etc); or have we stopped something from happening (for example, from deteriorating).
3. Who benefits? This helps to determine the domain in which the impact occurs (health, societal, environmental, political, economic, etc), as well as its reach (local to global).

Before going any further, just a few words about what impact is not. If impact equals change, then impact is NOT dissemination, academic reputation or citations. It also means that having 200 people in an outreach activity or in a capacity building course is not impact per se, but a means to achieve your impact. The real impact is the change that results from that activity (e.g. did it motivate the students to become more interested in STEM careers or did it improve the trainees’ ability to diagnose a particular disease?).

Needless to say, a single body of research can have different impacts in different domains and at different times. And the effects can rise from the research findings or from the research process itself (e.g. changes resulting from co-production activities). And yes, basic research is further away from being used outside academia, but, as well as setting up the longer term impact journey, it can have direct impacts on, for instance, the way society understands the world. 

Returning to the definition of impact, it can be summarised as “the provable benefits of research in the real world.”

Demonstrating impact

The devil in the definition above is the word provable. Proving our impact claims can be challenging, but it helps ensure that those claims are accurate and verifiable. For example, we might say that our research was used to change disease treatment guidelines or WHO recommendations. But can we prove it? It may take some detective work to gather all the evidence that links our research to changes in the real world, which may occur several years after the research project has ended. Evidence may include data and statistics, citations in policy documents or in meeting briefs, licensing agreements, government reports, surveys, testimonies, interviews, etc. This is definitely not something a single person can do, and it becomes more difficult when looking back at past research projects (something I learnt the hard way when writing some case study exercises). However, it is less of a challenge if the evidence is collected on a regular basis in current or future research. The CERCA course provided several resources and links on how to document evidence. 

I would like to think that developing an impact culture is not just about meeting the requirements of funding or evaluation agencies. It can – and should - be about achieving change that is meaningful to society

Planning impact

Ideally, research-led change involves identifying a problem or need, defining what should change (impact goals), and working backwards to see how to get there: what research needs to be done, who will we involve in the process (partners, stakeholders), how will we mobilise the knowledge, and what indicators of success will we use.

Understanding how to plan for impact is also important for Horizon Europe projects. I suspect that many of us have felt a bit lost in section 2 of Horizon calls (especially the “Pathway to Impact part”), which is usually left until the last minute. Section 2 is about describing how the project results (outputs) will contribute to the outcomes specified in the topic of the call and to the wider impacts specified in the work programme. You must also describe how you will get people to know about and use those outputs in order to achieve those impacts. It is the equivalent of designing a “mini” impact strategy except that, in the case of a Horizon project, the European Comission has already identified specific needs and specified in the call what impacts it wants you to achieve.

Raising our impact literacy

I would like to think that developing an impact culture is not just about meeting the requirements of funding or evaluation agencies. It can – and should - be about achieving change that is meaningful to society. I have no doubt that at ISGlobal we all agree on this. Making sure that we all mean the same when we talk about impact will make the task much easier. 

We will soon organise a series of impact sessions for the different departments at ISGlobal.

Meanwhile, ISGlobal staff can access the CERCA training sessions and resources here.

For more reading: Creating meaningful impact, by Julie Bayley.

[1] Dr Julie Bayley was our marvellous course facilitator. She is Director of Research Impact Development and the Lincoln Impact Literacy Institute, at Lincoln University, UK.