“We have a lot of great innovators at UNSW,” says the University of New South Wales (UNSW) Director of Educational Innovation Simon McIntyre.
“One of the things we’re working hard to do is bring them closer together so we can maximise the impact of the work that they’re doing.”
He believes it’s testament to a culture that has seen UNSW embrace a range of modern technologies, including artificial intelligence (AI), to improve its teaching and learning outcomes. In fact, AI is a key pillar of the UNSW Educational Technology Roadmap 2024-2028.
“These technologies are going to be critical for the future of education because they enable a level of personalisation that we’ve never seen before,” McIntyre says.
Personalising student journeys with AI
One way UNSW is using AI is through its Data Insights for Student Learning and Support project, led by the university’s Learning Analytics Intelligence team, with significant support and contributions from the IT and UNSW Planning and Performance (UPP) teams. The project aims to use machine learning to detect early on when students are at risk of academic failure and connect them to the right support and services when they need it most.
“The hope is that we can create a much better and more supportive environment for our students by quickly spotting who needs assistance and helping them make the right decisions to seek out the support they need,” says McIntyre.
The project uses a modular approach built around an Academic Success Monitor (ASM). The ASM employs a predictive machine learning model trained on historical data from learning and administration systems. This model identifies potential academic risks based on student engagement in the digital learning environment, allowing academics and students to take proactive measures.
McIntyre told EducationDaily that, to effectively combine real-world support services with the AI-driven support, “a large part of this project has been about working with our students and support services on the design of the solution”.
“Support services often report seeing students at a late point in the term when their issues have snowballed and become more difficult to work through,” he says. “Students have also expressed a desire to have more agency to help themselves privately when issues first arise.”
He says the ASM is “designed to meet the requirements of support services and give the students agency over their own success”.
“If the ASM prediction shows that a student might be at risk, it will first send a private message to the student with more details of the prediction. It will also share suggestions of information or services that might be relevant given the different data points that contributed to the warning. These suggested resources are provided to ASM by the support services based on experiential knowledge of students’ needs built over many years,” McIntyre told EducationDaily.
“However, if the risk profile does not subsequently change, the system is designed to reach out to a support team to coordinate proactive human interaction with the student because this is where a one-to-one conversation may be of greater benefit.”
Balancing human touch with AI tech
By endeavouring to understand where the ideal balance of effective AI vs human intervention meets, McIntyre says evaluating the model over time will enable the university to see how the system matures through the observation of “hard data about changes in student engagement, retention, and success”.
“One critical guiding principle we are working under is that the system is not about replacing our invaluable human expertise. It is about ensuring we can help get this expertise to those students who really need it as early as possible. ASM aims to enable a student to find the right person to talk to at any point should they wish to do so,” McIntyre says.
“Our aim is to try to ensure that we match the right person, at the right time, for the specific situation the student finds themselves in. Students have told us that often issues can get worse over time if they do not know where to turn or what support is available, and ASM can help provide this information.”
If the AI can help give students more self-agency to help themselves as early as possible, he says the hope is that more students can adjust their own course.
“This prevents small problems from becoming big ones over time. It ultimately gives our support services more time to provide meaningful human interaction with students whose problems need a more nuanced level of human understanding,” McIntyre told EducationDaily.
“The project is also now moving into a development phase that is using the same technology to make suggestions to students who are not at risk of failing but seeking to further enhance their study or university experience. Tips on successful study patterns from students who have previously taken the same class, relevant societies or events, and other opportunity-based guidance is all possible. We are excited to use this technology to help provide more time for people to be more informed and unburdened by some of the complexity that a university environment can bring about so they can spend more time focusing on what is important to them.”
“We’re not replacing support services and doing their job for them – we’re just systemising that approach so that we give students awareness of relevant support options and the autonomy to take action to help themselves. We’re also providing our support teams with a ‘heads up’ as early as week 2 in the term so they can reach those students who may need more specialised help.,” says McIntyre.
Testing the benefits
The ASM’s initial small-scale testing in 2023 involved 33 academics and 25 courses across all UNSW Sydney–based faculties. The results were promising, with the model confidently identifying 79 per cent of at-risk students in the first few weeks of a course.
Testing then expanded into a pilot in early 2024 for 80 courses, which included around 17,000 students and 83 academics. The ASM identified 284 students who were at risk of failing and in need of support, provided academics with updates and insights about student engagement in their class. In addition, 75 per cent of academics stated the ASM identified potential risks much earlier than previously possible, and 49 per cent of students who received proactive nudges from the system showed statistically significant increases in class engagement.
Associate Professor Lynn Gribble from UNSW’s School of Management & Governance has experienced the project’s benefits first-hand.
“Being on this project has really enabled me as a lecturer in charge to understand some things about my students in a quantifiable way,” she says.
“We know that students who perhaps leave something to the last minute, go missing from Moodle [UNSW’s learning management system], or aren’t engaging with the course materials will not do as well as students who do. I can [also] personalise ASM messages to these students, directing them to support services and helping them get back on top.”
Exploring a unique holistic approach
McIntyre believes this holistic approach makes the project unique.
“We believe we’re among the first university to look at this whole thing as a connective ecosystem,” he says.
“We’re not just putting all the onus on an individual academic to interpret the data and then act. We’re offering relevant suggestions based on the data, and also giving our students information about their engagement and personalised advice to help them succeed based on their own circumstances.
“The project is giving our support teams more exposure, reach, and insight into larger student cohorts then previously possible – working towards making their services more accessible and targeted to those who might benefit most.”
Enabling responsible AI use at scale
Implementing such a comprehensive project has not been without challenges. According to McIntyre, UNSW teams first needed to collaborate on the development of more AI-capable data infrastructure. UNSW are working with Microsoft Industry Solutions Delivery to further explore and prioritise AI use cases for expansion through a Three Horizon plan, supporting architecture Frameworks and building organisational AI Aptitude for prolonged sustainment.
“We had data scattered across the university and it isn’t necessarily unified. So, my team worked extensively with our UPP and IT teams to set up a data lake that could use AI and ML at scale.”
McIntyre notes that collaborating with UNSW’s Chief Data & Insights Officer (CIDO), Kate Carruthers, and Microsoft partners Accenture and Altis significantly bolstered the project’s success.
”The support of Microsoft and their partner Accenture really helped us kickstart everything through the co-development of a prototype in the Power Apps Innovation Centre Program. Our CIDO and Altis then helped wire custom configurations [of our Microsoft technology stack] together, which we wouldn’t have been able to do as quickly on our own.” he says.
Ensuring responsible and ethical use of AI has also been a top priority. A steering committee oversees the project, including UNSW students, teaching staff, and members of its Educational Innovation team and legal department. A privacy impact assessment was also conducted to ensure compliance with legislation, and the university’s student privacy agreement was updated to include AI use.
“We’ve worked extensively with our student groups, talking to them directly about what they do and don’t feel comfortable with [about the use of AI], and co-designing solutions with them,” says McIntyre.
Further expansion and integration
Looking ahead, UNSW has ambitious plans for its Data Insights for Student Learning and Support project and related initiatives. The ASM is set to roll out to all first-year students and teaching staff at the start of 2025 and then reach all 80,000+ students and 7,000+ staff by the following year.
In addition, UNSW is also investigating other uses of AI in the learning ecosystem to understand the additional value they would bring. “We’re also working on prototyping an orchestrator-style chatbot architecture with multiple AI bots underneath to act as personal concierges,” says McIntyre.
“We’ve already got a modest pilot project starting in the latter half of the year exploring the use of AI bots for roles such as student-facing administrative support, academic support on interpreting course information and lecture notes, and future student recruitment.”
This pilot of the chatbot technology will be assessed for its suitability to become the primary interface for the Data Insights for Student Learning and Support project and hopefully other university functions.
“All the things we’re exploring in this project are providing great case studies to assist the university’s exploration of how we can really leveraging the power of AI at scale,” says McIntyre.
“It’s been a fantastic project to bring people together to discover its potential and help the university move forward.”
