ChatGPT will not be replacing human teachers anytime soon, with a study into the technology’s capabilities finding it can’t help students critically understand academic assessment solutions.
The study – ChatGPT and academic on accounting assessments – from Charles Darwin University (CDU) explored whether ChatGPT could help students understand the answers in two financial accounting course unit assessments.
Test results show teachers lead best
Study author and CDU Professor of Accounting Indra Abeysekera said ChatGPT and later version ChatGPT-4 were fed a series of multiple-choice questions from assessments from an introductory and advanced financial accounting course unit respectively.
ChatGPT correctly answered eight out of 10 questions from the introductory test and scored five out of 10 in the advanced test. ChatGPT-4 correctly answered nine out of 10 questions and scored seven out of 10 in the advanced test.
New learners need more support
Professor Abeysekera says while the technology had the ability to correctly answer questions, it couldn’t support beginner learners to develop the knowledge and skills to find solutions.
“The solutions provided by ChatGPT showed that it is a solution provider rather than a teacher or instructor,” Professor Abeysekera says.
“ChatGPT can be constructive to a competent learner who has reached the competency level to further develop critical understanding. As research has indicated, high achievers can have a fear of numbers, and they can benefit by using ChatGPT solutions as validation checks for their learning.”
Professor Abeysekera says that the testing revealed ChatGPT does not provide scaffolding for novice learners to take over their learning and develop individual competencies to be less, or not reliant on it.
“It can be destructive to an incompetent learner and can serve as a platform to simply find the solution or as a channel to ease their fear of numbers,” he says.
“Furthermore, the findings showed that ChatGPT is not a foolproof solution provider, especially when questions have discipline-specific underlying assumptions and increased technical and task complexity.”
Understanding risks to academia
Professor Abeysekera says the study was inspired by the opportunities and risks ChatGPT poses to academia. His finding showed that, while ChatGPT had many capabilities, it could not surpass the critical thinking benefits of academia.
“ChatGPT can process unstructured data, convert it into information, and share it as knowledge,” Professor Abeysekera says.
“It can meet some criteria that meet originality, such as synthesising information differently and providing a new interpretation using known information.”
However, he says, “there are originality tasks that ChatGPT is incapable of meeting, at least for now”, including testing someone else’s idea in a different context, or developing a research tool.
“Hence, academic focus on learning and assessment should shift toward original knowledge.”
