Implementation of Socratic AI Tutors via RAG and Role Engineering in Physics Education

The rapid evolution of Large (Multimodal) Language Models (LMFM) highlights the limits of traditional university and high school teaching and assessment. Incremental adaptations are insufficient (Kortemeyer, 2025); a shift is needed towards disciplinary-based Research-Based Instructional Strategies (RBIS) that emphasize modeling, evidence-based reasoning, and active learning. We present concrete methodologies for using generative AI to support this pedagogical transformation, shifting its role from an automatic solver to a guide in the learning process. We focus on designing and implementing collaborative, Socratic AI tutors. This leverages new LMFM capabilities (Küchemann et al., 2025) to create personalized learning assistants. While developed for physics, these methods are broadly applicable.

We illustrate two complementary personalization strategies: 1) Retrieval-Augmented Generation (RAG), using platforms like Google NotebookLM to anchor tutor interactions to instructor-verified documents (e.g., pedagogical "Training Manuals"), ensuring traceability and pedagogical control (Tufino, 2025b). 2) Role Engineering, personalizing LMFMs (e.g., Gemini Gems) via detailed "scripts" to create pedagogical partners that guide students with targeted, multimodal feedback (Tufino & Gregorcic, 2025). Both strategies maintain the teacher's central role in defining interaction rules. While requiring AI Literacy (Polverini & Gregorcic, 2023; Tufino, 2025a), our goal is to provide replicable models for integrating AI as a cognitive partner. We discuss preliminary qualitative resultsand challenges, such as managing motivation during Socratic dialogue. Our approach aims to utilize  AI to make teaching more authentic, rigorous, and focused on competencies that AI cannot replace.

References

Kortemeyer, G. (2025). The Boiling-Frog Problem of Physics Education (arXiv:2508.08842). 

Küchemann, S., Avila, K.E., Dinc, Y. et al.On opportunities and challenges of large multimodal foundation models in education. npj Sci. Learn. 10, 11 (2025). https://doi.org/10.1038/s41539-025-00301-w

Tufino, E. (2025a). Exploring large language models (LLMs) through interactive Python activities.Physics Education, 60(5), 055003.https://doi.org/10.1088/1361-6552/adea28

Polverini, G., & Gregorcic, B. (2024). How understanding large language models can inform the use of ChatGPT in physics education. European Journal of Physics, 45(2), 025701. https://doi.org/10.1088/1361-6404/ad1420

Tufino, E. (2025b). NotebookLM as a Socratic physics tutor: Design and preliminary observations of a RAG-based tool.(in print The Physics Educator)

Tufino, E.,Gregorcic, B. (2025). Creating a customisable Socratic AI physics tutor. (arXiv:2507.05795)  https://arxiv.org/abs/2507.05795