How can universities better create and assess research impact? Dr David Phipps, Assistant Vice President, Research Strategy and Impact at York University, Canada and Dr Julie Bayley, Director of Research Impact Development, at the University of Lincoln, UK share insights on assessing the wider impact of university research on society.
How might a university determine research themes?
This varies by nation and institution. For example, all Canadian universities must develop research themes as part of a required strategic research plan (SRP). Every university will use their own method, but they generally involve speaking to stakeholders to identify strengths and areas of opportunity, responding to local, provincial and national priorities. Sometimes the government informs their priorities, commonly via a Science and Technology Strategy, Budget Speech or a Speech from the Throne.
For example, York University in Toronto, Canada has a Strategic Research Plan (SRP) 2018-2023 which guides all investments in research, and it specifies six areas of strength and five areas of opportunity. This strategy is developed through approximately 18 months of consultations before launching the five-year SRP, with a review late in the third year to inform the next round of consultations leading to the next SRP. The Vice-President of Research and Innovation leads the development of the SRP and has the responsibility for implementation.
Within the UK, there is no formal SRP requirement, and themes tend to reflect current research strengths, policy priorities and location. For example, the University of Lincoln is located within a historic city, in a very rural county, and its research themes reflect needs of the region (such as the challenges for health, community, digital infrastructure and economic growth in rural areas), alongside activities in support of heritage and conservation.
The societal impact of research from most universities is often not well captured in the promotion of researchers, how can we influence our universities on this?
While there is no singular approach, David offers a few strategies to support this below:
- Meaningfully include engagement and impact in career progression or tenure policies
- Set academic policy through collective bargaining and/or committee/Senate structures rather than through ‘top down’ diktat
- Ensure academic colleagues are represented on Committees, university councils, the design of progression frameworks, and other arenas where policies and related ‘markers’ are decided upon
In terms of getting ‘buy in’, it’s important to consider where the institution needs to engage with society through research and how it relates to institutional strategy, such as:
- Requirements by funders
- Assessment agendas
- Alignment with the Sustainable Development Goals
- University mission
- Partnerships with the wider sector
- Shifts in national and international agendas, and looking for examples from elsewhere to demonstrate its value.
- ‘Ground up’ moves from engaged scholars
Ultimately, a change in policy by management is not going to be effective; it needs to be collectively built through participation, reflective of sector needs, pressures on academia, and feeling like a change people want to see.
Sometimes impact occurs despite institutional support. What happens when the agendas of the researchers and the university are aligned?
Many universities don’t offer support for impact outside of commercialisation and industry liaison, so a lot of impact does occur despite institutional support. Where support is minimal, impact tends to arise from researchers feeling it is the right thing to do for themselves, their research partners and sometimes the research funder - for example, all health charities want to fund research that will have an impact on patient care. This can create a significant toll on those individuals driving for change without support, which is one of the reasons we developed our work on institutional health.
The alignment of agendas and expectations is absolutely key – researchers needs to understand how their efforts fit into their professional landscape, and unless the institution understands the support needed for this, effects will be lost. We have a table in the updated ‘Impact Literacy’ paper which covers misalignment and the importance of impact literacy.
When institutions and individuals build their impact literacy together it helps to ensure that a researcher’s impact literacy is aligned with that of the institution.
What quality assurance parameters can be applied to research impact?
This is a difficult question because we don’t have metrics to measure the quality of real-life impact. Societal impact is experienced outside of academia, and manifests in the provable effects of research in the real world.
Issues such as quality, appropriateness and sufficiency are in the eye of the beholder, such as the businessperson, the health commissioner, the schoolteacher. Quality is best understood as what matters to them. However, there undeniably remains a drive for quality to attest to the robustness, transparency, and trustworthiness of any ‘data’.
For impact, this arguably relates to evidence. Impact evidence is the substantiation of any claim of effect, and as such must stand up to scrutiny. In this context, ‘quality’ relates to the source of the evidence, the extent to which is substantiates the claim, and issues around proportionality (is the claim proportional to the evidence?), attribution (does the evidence show an effect arising from the research rather than something that would occur anyway), and credibility (eg. is the source actually able to attest to the claim?).
How can research impact be mainstreamed into the university research and development agenda to support the Institutional Advancement strategy?
Advancement is primarily about raising money whilst impact is about making a difference. There are overlaps, however the key difference is in the direction of the effect. The defining characteristic of impact is that the benefit is felt outside of academia. Income generation, in contrast, is about benefits for the academy. Being practical, pressures on institutions to generate income mean that whilst impact-as-altruism is commendable, it needs to fit within broader growth plans. In this respect, mainstreaming impact helps to build reputation by demonstrating benefits to society, often via case studies, retain faculty and student talent by valuing their engaged activities, attracting research funding by addressing policy issues, and enhancing the regional engagement of the institution such to attract investment. Money might be a metric, but it should not be a goal for impact work.
Could partnership agreements and joint research agreements include a section on impacts to encourage other partners to consider impacts more thoroughly?
In short, yes. Universities should stay in touch with the research sponsor in sponsored research agreements after the end of the funded period to request any ‘evidence’ of how the research is used. Research impact typically doesn’t occur during the life of a funded research project. You certainly can change attitudes and awareness of participants through the funded research period, but funding timelines are often too short to get a new product to market or a new social service rolled out. Establishing the parameters of a partnership beyond the final date of the project can go a long way in tracking the impact of the project.
What can universities do to make impact
There are many ways to make impact,. The two most fundamental answers we can give are:
- Invest in individual literacy, supporting academic and research management staff (of all levels) to build critical understanding of how impact works, so they can identify stakeholders, plan pathways, design mobilisation activities and gather evidence
- Invest in institutional health, building literacy at an organisational level and ensuring processes, systems and strategies support healthy approaches to impact (rather than simply exerting pressures of expectation. We cover this in our Institutional Healthcheck self-assessment tool to identify strengths, weaknesses and areas of focus for strategic development.
Research impact is seen as citations in many universities. What is the distinction between research impact and citation and how do we shift this perception in our universities?
This was addressed explicitly at the ACU Ambassadors Summit on Creating and assessing research impact, so we encourage you to watch the recording.
Impact is not citation. While acknowledging the continued importance of bibliometrics across academia, and their dominance in jurisdictions which are not yet ‘doing impact’, it’s important that thinking about citations revisits both the definition of impact, and the way in which it arises.
Firstly, impact is the provable effects of research in the real world that benefits people outside of academia. Citations are expressions of attention within academia, and therefore cannot show change in the real world.
Secondly, impact is not dependent on a publication. Research can have an impact even if it is never published – for example, in building non-academic capacity through involvement in research itself, not through use of the findings. It’s important that we remember that publications are the currency of academia – ‘products’ by which we share learning and are simultaneously judged by our peers – and not something necessarily valued by others. Put simply, the research may be important, but the publication may or may not be. Thus, citations are not synonymous with impact as they disregard routes which bypass a publication.
More information
To hear from more members about supporting research, join the ACU Supporting Research Community.
The Supporting Research Community is a forum for all staff at ACU member universities who are involved in supporting the research process, including research management and administration, research impact and uptake, and library services. Researchers at all levels are also welcome to join.