What Really Makes AI Effective in the Classroom?

Many of the predicted harms of Generative AI in Education, such as reduced long-term retention, are not inherent to Gen AI usage. Rather, they result from unstructured and poorly guided use of Gen AI in educational settings. With skills-focused AI-course policies and training in metacognition and usage strategies, educators can use Gen AI to enhance personalized support and accelerate learning—while minimizing its potential harms.

Introduction

In early 2023, the world flipped on its head as ChatGPT skyrocketed to become one of the most used apps of all time (Hines, 2023). Fast forward two years, and every field of study and commerce has had to adapt to Generative AI or risk getting left behind. Among the fields most affected, and perhaps the most crucial to our future, is education.

Ever since educators were introduced to Gen AI, whether through Professional Development seminars or by excited students, they have been locked into constant debate on the merits of this technology in education. Some educators argue that rampant AI usage will decrease student ideation, long-term retention, and social interaction (Crawford et al., 2024; Mollick, 2025). Meanwhile, others argue that AI will allow for more personalized learning experiences that increase student engagement, long-term learning, and accessibility (Gibson, 2024; Mollick, 2025).

This divide has led to wide disparities in how educators handle AI: some ban it outright, others embrace it as a tool, and still others allow it only in select contexts and with proper citation (C. Eaton, personal communication, July 23, 2025). This inconsistency extends to students, whose use ranges from seeking occasional feedback to outsourcing entire assignments. Without guidance, many students risk the long-term harms of poor AI use. Thus, understanding how students interact with AI — and how instructional design shapes those interactions — is key to maximizing its benefits and minimizing its risks.

1. Impacts on Individual Learning

Understanding AI’s impact on long-term learning requires first understanding metacognition, i.e. the student’s ability to reflect on their thought processes and knowledge base (Mannion, 2018). Together with behavioral self-regulation, this forms the basis of Self-Regulated Learning (SRL), where students with stronger SRL skills can adapt new learning strategies more quickly and effectively, leading to better retention and engagement. This raises a central question: how does AI use influence and get influenced by a student’s self-regulated learning?

1.1. Effects on Cognitive Development and Retention

To begin with, when AI is misused, it can negatively impact a student’s cognitive development and retention. For instance, in their paper on the impacts of Gen AI on novice programmers, Prather et. al (2024) suggest that AI usage can uniquely lead to two key metacognitive difficulties: 

• Mislead, where the AI steers students away from the correct solution due to hallucinations, missing context, or prompt misinterpretations

• Progression, where the AI overcompensates for a student’s weaknesses and makes them feel overly-confident in their abilities

In both cases, the student is unable to accurately self-reflect on their learning, potentially leading to reduced understanding, practice, and retention. Similar patterns appear beyond programming. In essay writing, Kosmyna et al. (2025) found early AI use reduced metacognitive engagement and led to more generic work. Similarly, Wadinambiarachchi et al. (2024) found that in visual design activities, participants who were exposed to AI-generated images experienced more design-fixation, as building ideas off of the AI generated images was often more convenient than coming up with completely new ideas and essentially redoing the AI’s work. As a result, this design-fixation led to a smaller variety of unique ideas being produced, thus reducing the originality of participants’ outputs (see Figure 1 below for more details).

Figure 1. A Grounded Theory Model of how the inspiration stimulus (problem statement, ideation cues, AI generated images, etc.) affect the overall ideation process and its output (Wadinambiarachchi et al., 2024). The choice of inspiration stimulus influences time spent sketching versus seeking inspiration, which affects fluency (number of sketches). Higher fluency tends to increase variety, which in turn raises the likelihood of generating more original ideas.

However, that’s not to say there are no upsides. One such benefit is that AI can support cognitive development. For example, personalized explanations and analogies generated by LLMs can help students connect concepts in meaningful ways (Bernstein et al., 2024), while immediate, tailored feedback can accelerate correction of misunderstandings and allow students to iterate on their work (Seo et al., 2021). Additionally, AI can perform the rote tasks (e.g. type annotating code, grammar-checking, etc.) of a project to free more time for the students to spend on higher-order thinking and learning.

1.2. Promising Strategies for Productive AI Learning Integration

Thus, it is clear that AI can have a variety of both positive and negative impacts on individual learning. This leads to an important question: how can individual students maximize the benefits of using AI?

First, students must learn to prompt properly. By writing more effective prompts, students can avoid the miscommunication issues that lead to the Mislead difficulty and decrease the design-fixation associated with AI co-ideation (Prather et al., 2024; Wadinambiarachchi et al., 2024). There are two general guidelines for writing an effective prompt: providing lots of context and being specific (MIT Sloan Teaching & Learning Technologies, 2024b). These help the AI understand what the student is looking for and to come up with more useful outputs. In an academic setting, this might mean sharing example work as part of the prompt or asking the AI to role-play as a domain expert.

Moreover, these guidelines can be useful for more complex prompts involving restrictions. Researchers have found that by restricting an LLM and having the student engage with its output (e.g. the AI can only provide guiding questions and the student must answer them on their own to progress), the students experience deeper engagement and improved learning gains (Kazemitabaar et al., 2024). Thus, learning to prompt can allow students to add friction to their AI to maximize their learning gains.

Furthermore, the timing of AI use also matters. In their study on AI-assisted writing, Kosmyna et al. (2025) found that participants who delayed their AI usage had better retention and writing performance than participants who used AI from the start. Similarly, Wadinambiarachchi et al. (2024), found that by spending more time brainstorming before prompting the image generator (rather than directly copying the problem statement), participants were able to increase the originality of their drawings.

In short, AI’s impact depends less on the model and more on how and when it’s used. Prompting effectively, engaging critically with outputs, and ideating before seeking AI assistance can help students maximize learning benefits while avoiding harm.

2. Impacts on Assessment

In addition to individual learning, the instructional design surrounding a student plays an equally important role in shaping the quality of their education. Thus, it is important to consider how an educator’s course policies and assessment design affect the ways in which students interact with AI.

2.1. Prevalence and Contexts of AI Use in Coursework

The first step to solving AI-related course policy issues is to understand how students behave in the current assessment environment. According to Freeman’s (2025) student survey, up to 88% of students are using AI for assessments, with the most common uses being to “explain concepts”, “summarise articles”, “suggest research ideas”, and “structure my thoughts” (see Figure 2 below). The middle two of these 4 uses unfortunately conflict with the “good” AI usage guidelines described in section 1, as they can lead to less engagement with the content and less original ideation from students.

Figure 2. A graph showing the distribution of students using generative AI across different academic contexts (Freeman, 2025). 

2.2. Challenges to Traditional Assessments and Assignments 

Lack of engagement is one of the most common issues AI creates for asynchronous assessments. Over 30% of students believe AI-generated work could earn a good grade in their subject (Freeman, 2025), so many are tempted to offload large portions of their assignments.

In early 2023, students often submitted work written entirely by AI. This was easy to detect, as AI-produced content followed predictable syntactical patterns and lacked original synthesis (M. Blaauw-Hara, personal communication, July 24, 2025). However, as students learned to blend AI into their workflow, it became harder to tell where human effort ended and AI assistance began.

Today, many instructors can no longer reliably identify AI-written sections, and AI checkers remain inaccurate (MIT Sloan Teaching & Learning Technologies, 2024a). Moreover, strict detection-and-punishment policies risk false accusations, eroding trust between students and educators. This climate drives students to waste time hiding their AI use and teachers to spend time policing instead of teaching (D. Zingaro, personal communication, July 21, 2025). As a result, both sides lose: students engage less deeply with assignments, and instructors give less meaningful feedback.

Thus, rather than trying (and failing) to police the students’ AI usage, some educators are rethinking their assessments to guide students toward productive AI use.

2.3. Innovative Course Policies and Assessment Designs

In the literature, there is a general consensus on two guidelines for designing courses that encourage effective AI use:

1. Be transparent and encourage open dialogue. Through open discussions, instructors can guide students to use AI to supplement their learning, as opposed to replacing their critical thinking, and help students solve the issues they face when using AI (MIT Sloan Teaching & Learning Technologies, 2024a). Since harshly penalizing AI use disencourages communication, this means that AI policies must be both somewhat lenient and completely transparent.

2. Encourage reflection on AI use. Metacognitive skills, such as those discussed in section 1.2, can be developed by asking students to experiment with AI use strategies and evaluate how those strategies align with the learning outcomes of the assignment or course. This self reflection also teaches students to take responsibility for their learning and to avoid passive engagement.

These principles can be implemented in a variety of concrete ways. For example, instructors can directly teach AI metacognitive skills through Design Activities. As suggested by Dr. Eaton (personal communication, July 23, 2025), these are low-stakes classroom exercises where students brainstorm or problem-solve either alone or in a group before integrating AI later. This approach trains students to think critically and ideate on their own, then to compare their own ideas to the AI’s and reflect on the process. Other, less direct, ways to implement these guidelines include Active Learning strategies such as Peer Instruction (Centre for Teaching Excellence, 2024), Flipped Classrooms (Harvard University, 2023), or structured debates. These activities build critical engagement in the classroom so that when students go on to work on their asynchronous assignments, they already have their own ideas and knowledge base to build off of.

Ultimately, every educator must decide which skills to preserve and which can be outsourced to AI. What matters most is intentionality: design courses that make clear which learning outcomes matter, and guide students to use AI as a tool to strengthen those skills rather than shortcut them.

3. Impacts on Student Life

Outside of the classroom, there are a wide variety of experiences that shape a student’s quality of education such as their social environment, ability to access key resources, and socioeconomic condition. Thus, it is important to consider how AI affects and is affected by all these different experiences.

3.1. Shifts in Student Social Life

The impacts of AI on students’ mental health and social interaction is a very new field, with a lot of nuanced and often conflicting theories. However, there is a general consensus that there is a positive relationship between increased AI usage and loneliness among (post-secondary) students (Klimova & Pikhart, 2025). Fortunately, this link is not an inevitable fact of AI use, but rather an exaggerated symptom of pre-existing student isolation issues. Crawford et al. (2024) found that the presence of social support such as friends, family, or extracurricular communities “mediated the relationship between loneliness and AI usage”. This suggests that through intentional campus support design and investment into extracurricular communities, educational institutions can help mitigate the impacts of AI on student loneliness. Moreover, through collaborative in-class exercises such as the Active Learning activities from section 2.3., instructors can foster a more connected student community and further reduce the risk of student isolation.

3.2. Improvements in Accessibility

One of AI’s most cited educational benefits is improving communication for English as a Second Language (ESL) students—or any learner who struggles with the institution’s primary language. (Gibson, 2024; D. Zingaro, personal communication, July 21, 2025). This democratization of written communication helps make asynchronous assignments more fair for ESL students in courses where “English communication skills” is not one of the primary learning outcomes. By helping these students more clearly communicate their ideas, AI can also help these students foster a deeper sense of community and belonging when they otherwise might feel isolated by their language skills.

Furthermore, AI can improve accessibility and inclusion for many students who have disabilities such as (Gibson, 2024):

• LLMs and integrated AI products like Astica.ai can provide image descriptions for the visually impaired

• AI can be used to more quickly create audio descriptions or live captions for videos and other viewable content

• AI-powered apps such as Ava can be used to transcribe live conversations and make them more inclusive for hard of hearing students

• Voice control can allow a wide variety of students to interact with technology in ways that they could not access previously

• AI tools can help instructors create more inclusive materials easily, such as GPT Accessibility CoPilot to ensure that code structure matches accessibility criteria, or Ask Microsoft Accessibility which is an AI tools that offers suggestions to users on how to make their content more accessible

However, it is important to note that AI will not solve accessibility on its own. Most AI products are still developed without inclusivity in mind, and students with disabilities are often the ones who also struggle the most to gain access to these assistance tools (Pillai, 2023). Thus, more important than which AI inclusivity tools an educational institution uses is how they use these tools: how are instructors encouraged to implement them, how does the institution raise awareness of these tools, and how do their existing accessibility supports carry the slack where AI may be unable to help.

3.3. Impacts on Student Inequality

Socioeconomic inequality is one of the biggest factors that impact the quality of learning that a student receives (American Psychological Association, 2017). As such, it is important to consider the relationship between AI student usage and their socioeconomic status.

To begin with, researchers have found that although AI can improve learning outcomes, it often helps students with better academic skills and AI use experience more than it does weaker students (Kazemitabaar et al., 2023; Prather et al., 2024). Since students with a higher socioeconomic status often have access to more tools & resources from an earlier age, this means that AI could potentially widen the gap between students in different socioeconomic levels by way of their educational experience.

In addition, students with a higher socioeconomic status usually have more experience with AI usage (Freeman, 2025). As a result, they often have better metacognitive AI skills and are able to integrate its outputs more cohesively into their work. For instance, some professors have noted that first-generation and working-class students tend to rely on AI uncritically, such as directly copying its outputs without checking for hallucinations, perhaps due to less experience with the technology (M. Blaauw-Hara, personal communication, July 24, 2025). This experience gap can potentially snowball, with socioeconomically advantaged students getting constant opportunities to iterate on their AI skills and boost their productivity while less advantaged students are left behind and do not receive guidance on how to catch up.

Fortunately, these inequity risks can be mitigated through various methods. The learning experience gap could be closed through increasing existing interventions such as extra office hours, tutoring programs, or collaborative exercises such as pair programming in CS or peer reviewing in English classes. Meanwhile, the AI skills and experience gap can be minimized by taking extra class time or creating free extracurricular resources to directly teach AI related skills. By closing these gaps, educators can ensure that AI becomes a tool for equitable learning, not a driver of deeper divides.

Conclusion

In summary, AI integration into education is a double-edged sword. If wielded correctly, it has the potential to make learning experiences more inclusive, personalized, and effective. However, careless use by students or misconceptions among instructors can deepen existing problems like inequity and disengagement with assessments. Thus, students and instructors must work together to ensure that AI does not undermine the learning experience for future generations.

For students, they must make sure to reflect on their AI usage and learning strategies consistently. They must practice applying some of the metacognitive skills discussed in this article to maximize the learning benefits they can get out of AI. In addition, they should also engage with peers and instructors inside and outside the classroom, seeking guidance on responsible AI use and avoiding the isolation that can result from its overreliance.

As for educators, they must acknowledge AI’s presence and, rather than resist it, encourage students to think critically and engage in higher-order thinking that AI cannot yet replicate. They should prioritize the most critical learning outcomes and clearly show students how assignments connect to those goals, as well as where AI fits into that process. Most importantly, rather than immediately penalizing AI use, instructors should address cases where it conflicts with learning outcomes through open discussion, seeking to understand the student’s choices and offering guidance.

Educators once debated whether AI belongs in education, but that debate has long since become obsolete. AI is here to stay, with extraordinary potential for both good and harm. The question is, can students and educators work together to design learning environments where AI is not treated as a distraction or afterthought, but embraced as a tool for deeper learning, stronger communities, and more equitable opportunities?