If you’re a high school student and future accounting major hoping to receive your undergraduate education from a friendly chatbot, we’re sorry to say that won’t be happening any time soon. A study authored by Charles Darwin University Professor of Accounting Indra Abeysekera and published in Journal of Open Innovation: Technology, Market, and Complexity examined how well ChatGPT can explain the solutions to problems and found it’s about as good as a terrible professor.
Professor Abeysekera chose accounting to study in particular because the accounting syllabus comprises both narrative-based and mathematical-based learning.
The abstract:
ChatGPT is considered a risk and an opportunity for academia. An area of threat in contemporary settings is whether it can become a student agent for assessments in academia. This study determines how ChatGPT can become a human agent for students on two financial accounting course units, multiple choice question assessments. The study provided five numerical-based and five narrative-based multiple choice questions. There were ten questions for the Introductory Financial Accounting and 10 for the Advanced Financial Accounting course units. ChatGPT received one question at a time requesting a solution. In the Introductory Financial Accounting section, ChatGPT produced incorrect answers because it incorrectly assumed the underlying assumptions contained in those questions. In Advanced Financial Accounting, ChatGPT presented incorrect answers because of the complexity of the task contained in those questions. ChatGPT demonstrated similar competencies in providing solutions to numerical-based and narrative-based questions. ChatGPT obtained the correct answers to sit in the 80th percentile in the Introductory Financial Accounting course unit assessment and the 50th percentile in the Advanced Financial course unit assessment. ChatGPT4 showed improved performance, with the 90th percentile for Introductory Financial Accounting and the 70th percentile for Advanced Financial Accounting. The findings indicate that the knowledge construct requires reflective thinking with ChatGPT in the ecosystem, and what is assumed and assessable knowledge must be revisited.
Said Professor Abeysekera (as relayed by Phys.org): “The solutions provided by ChatGPT showed that it is a solution provider rather than a teacher or instructor.
“ChatGPT can be constructive to a competent learner who has reached the competency level to further develop critical understanding. As research has indicated, high achievers can have a fear of numbers, and they can benefit by using ChatGPT solutions as validation checks for their learning.
“ChatGPT does not provide scaffolding for novice learners to take over their learning and develop individual competencies to be less or not reliant on it. It can be destructive to an incompetent learner and can serve as a platform to simply find the solution or as a channel to ease their fear of numbers.
“Furthermore, the findings showed that ChatGPT is not a foolproof solution provider, especially when questions have discipline-specific underlying assumptions and increased technical and task complexity.”
The language in this paper is spectacular considering it was authored by a professor of accounting and not literature. A few highlights:
With nights and days passing serenely, a big surprise as morning dawns on November 30, 2022, has made a lasting change in how we have taken what we know, to know, and beyond for granted.
ChatGPT serves as a lighthouse with the metaphor of a sea route, where the light it surveys is perceived as a route for academic construction or destruction. ChatGPT provides the shining light, and the pathways are chosen by academia, defined by the five Ws: what it is being used, who uses it, where it has been used, when it is used, and why it is being used (Robertson, 1946)
The ChatGPT provides access to structured knowledge to anyone and uses a utilitarian position because it determines right or wrong based on the greatest number of people who have access to knowledge (Kay, 2018). Academia takes a deontological position to knowledge promoting the highest good as developing reasoning in people as a duty – rights and wrongs are determined by dutifulness (Misselbrook, 2013).
Beautiful.
As the professor points out in the paper, ChatGPT offers a lot of utility to learners of accounting, especially those familiar and comfortable with the concepts, but “can be destructive to an incompetent learner” as critical thinking is a necessity for understanding higher level accounting concepts. Anyone who’s had a lousy Intermediate teacher will tell you a professor can make or break a grade depending on how well they can explain the fundamentals.
With a critical shortage of PhD accounting professors that’s been brewing for well over a decade now, it’s too bad ChatGPT can’t do much to fix it in the short term.
Expert tests if AI can help teach students accounting [Phys.org]
ChatGPT and academia on accounting assessments [Journal of Open Innovation: Technology, Market, and Complexity]
Earlier: