Mathematical learners of high school encountered a decline in performance while employing an AI tutor of advanced model GPT-4 for their math studies.
In a groundbreaking study conducted at the University of Pennsylvania, researchers have highlighted the potential pitfalls of using AI tutors in education. The study, titled "Generative AI Can Harm Learning", was published by the Wharton School of the university.
The research involved nearly a thousand high school students in Turkey and focused on the use of two different AI tutors, powered by GPT-4: GPT Base and GPT Tutor. On the actual test, students who used GPT-based tutors performed 17% worse, with the GPT Tutor group performing the same as the control group.
Interestingly, GPT Base tutor, which provided answers to math problems while helping students, resulted in students performing 48% better on practice exams, while those with GPT Tutor, designed to help students find answers on their own, performed 127% better. However, GPT Tutor seemed to mitigate the negative impact of an AI tutor on students, though it didn't help them either.
The study's one-sentence summary is, "We find that generative AI could hurt learning because students potentially use it as an answer machine, as opposed to a tool that is conducive for learning."
Professor Sungu, one of the researchers, states that the findings underscore the need for careful design considerations for AI tutors in education to prevent harm and promote effective learning. He believes that for an AI tutor to become more effective, it will have to focus on learning rather than productivity.
Sungu emphasizes that there are certain skills, such as critical thinking and problem-solving, that are vital for students to develop without relying on machines. He notes that AI is already good at providing answers, and the design of assessment and educational delivery needs to be reconsidered.
Design considerations for AI tutors in education include human-centered design, personalized learning, accessibility & equity, engagement, data privacy & security, transparency, support for educators, safeguards against misuse, and immediate & actionable feedback. These principles aim to enhance learning outcomes while minimizing potential harm such as overreliance, inequity, loss of human interaction, privacy breaches, and academic misconduct.
Sungu believes that even though AI might get more advanced, it won't change the inherent challenges surrounding AI and learning. He mentions that there are technologies that make some skills obsolete, such as the calculator, but argues that testing student achievement without the use of AI might still matter, as there are skills that are vital for students to develop independently.
The researcher also states that there is already a lot of literature showing that AI can help professionals accomplish tasks more efficiently and effectively. However, he stresses that the design of AI tutors in education needs to be carefully considered to ensure they complement, rather than replace, educators and preserve the essential human connection in education.
- The study on generative AI in education suggests that students might use AI tutors as an answer machine instead of a tool for learning, potentially leading to decreased performance.
- To prevent harm and promote effective learning, AI tutors in education should focus on learning rather than productivity, and incorporate design principles such as human-centered design, personalized learning, and immediate feedback.
- Critical thinking, problem-solving, and other essential skills can't be developed solely through AI assistance, indicating a need to reconsider the design of assessment and educational delivery.
- As AI technology evolves, it's crucial to ensure that AI tutors complement rather than replace educators, preserving the human connection in education for education-and-self-development.
