The Australian Government's prestigious Australia Awards programme offers scholarships to emerging leaders from the Indo-Pacific region to undertake full-time undergraduate or postgraduate study, or short-term fellowships and short courses. In order to monitor and evaluate the impact of these scholarships and fellowships, the Department of Foreign Affairs and Trade (DFAT) has designed the Australian Awards Global Tracer Facility.
What are the Australia Awards?
Beginning as the Colombo Plan in the 1950s, recognised for a number of years as the Australian Development Scholarships, and known today as the Australia Awards, these scholarships and fellowships form a key part of the Australian Government's aid programme. Awardees are sponsored to undertake a period of study in Australia, usually within an Australian university. The scholarships are designed to provide educational opportunities to improve living standards and stimulate economic growth in developing countries. The scholarships are also intended to help in generating a network of global leaders with ties to Australia. Since the 1950s scholarships have been offered to more than 80,000 high achievers from the Australasian region and beyond.
What is the Australia Awards Global Tracer Facility?
Most of us who work in the area of international scholarships management or evaluation can appreciate the benefits gained by an individual who receives an award to study overseas. These awards and scholarships aim to offer an opportunity for students to embrace new ways of thinking and learning, and foster new knowledge and skills. Awardees often have profound stories of transformation through their overseas learning experiences; of making sense of a new culture and way of life, making new friends, and forging life-long connections with a place and people. But what actually occurs when recipients return home? Do they bring with them new skills and knowledge, and do they connect with new networks? If they do, how are these skills, knowledge and networks put to use – and to what effect? What is the impact on scholarship alumni's communities, workplaces, industries and countries? And are there cumulative benefits of investing in scholarships over time?
The answers to these questions and more are sought by the research team engaged on the Australia Awards Global Tracer Facility (the Facility); a project funded by the Australian Government to provide a consistent and centralised approach to monitoring and evaluating the impact of scholarships and fellowships. The Facility is a four-year investment, which commenced in May 2016, to assess the development contributions and public diplomacy outcomes of Australia's investment in the Australia Awards. The Australian Council for Educational Research (ACER) is managing and implementing the Facility on behalf of DFAT.
The Facility aims to generate high quality information about former awardees, focusing on less recent alumni, in order to provide a strong evidence base for the Australian government and its diplomatic posts to evaluate the impact of Australia Awards on recipients, their home institutions and countries.
In order to fulfil these objectives the Facility will use a mixed methods design, whereby qualitative approaches will be designed and implemented based on what is learned from initial quantitative research (see diagram below for overview).
The key data collection methods are an Annual Tracing Survey, administered by phone to at least 1,000 alumni per year, and case studies of alumni in various country contexts.
The Annual Tracing Survey instrument is based on a survey developed by the Australia Awards programme over the past few years and involves a short, targeted interview with alumni using Computer-assisted Telephone Interviewing (CATI). This approach is being trialled in the first year of the Facility, it is hoped that telephone surveying will improve response rates and has been shown to be an effective means of engaging alumni based on work previously undertaken by DFAT in South Asia.
Case studies will be thematic, explanatory, and linked to the survey results. Themes will be identified annually based on the Tracing Survey, but are expected to cover a range of development priorities, as well as cross-cutting issues such as inclusiveness, relating to gender and disability. Four case studies are planned for the first year of the Facility.
Why is the Facility needed?
This substantive investment in a centralised Facility for monitoring and evaluation of the Australia Awards has been many years in the making; as noted earlier, the Australian government has supported high achievers from the Indo-Pacific region and beyond to study in Australia for over 60 years. These scholarships and fellowships have been awarded to generations of global leaders, providing high quality educational experiences and fostering links and networks between countries. Many of these alumni have advanced in their careers and now occupy high profile leadership roles in their respective countries.
A large number of alumni were awarded scholarships some decades ago and have since lost touch with their cohort and networks. A unique feature of the Facility involves locating and engaging with alumni dating back to the 1950s, by working in partnership with universities and alumni associations. These efforts align with the aim and objectives of the Australia Global Alumni Network.
Prior to establishing the Facility, monitoring and evaluation was conducted at the country or programme level, leading to country or programme-specific approaches to data collection and reporting. Data and findings could not be compared across countries or programmes, or aggregated to evaluate the impact of the Australia Awards as a whole. A need for high quality, consistent and comparable data was determined and as a result the Facility was designed.
The investment in Australia Awards monitoring and evaluation is representative of DFAT's commitment to enhanced performance monitoring and aim to strengthen and grow international education. The latter is articulated in Australia's National Strategy for International Education 2025, which was released in April 2016. Another initiative that aligns closely with the Facility is DFAT's Australia Global Alumni Strategy, which includes a virtual global network created to connect and build an international community of scholars who have studied in Australia and Australians who have studied overseas.
What's next?
The Facility research team, led by Dr Daniel Edwards, has spent the past few months planning, refining its data collection instruments and piloting. The inaugural survey commences administration in early November 2016. The first case study was conducted in late October 2016 in Fiji. Reports will be submitted to the Australian Government from November 2016 through to February 2020. Other dissemination activities will be undertaken by the Facility to further engage alumni and share findings with the wider research community. The Facility will also be developing a Research Repository for collating a range of scholarships-based research over the coming years, and will be liaising with interested researchers in developing this aspect of its work.
For more information about the facility, take a look at the DFAT website
The Measuring Success blog series draws from the ACU's experience in scholarship design, administration, and analysis, and our connections in the sector, to explore the outcomes of international scholarship schemes for higher education.