Learning in the Panopticon: Examining the Potential Impacts of AI Monitoring on Students

Bingyi Han, George Buchanan, Dana McKay

Session: Dignity and Emerging Systems - Wednesday 2022-11-30 11:00-12:20 AEDT
Paper Track
In-Person Presentation


In a panopticon, people are intrusively monitored across all areas of their lives. AI monitoring has been ever more widely adopted in education, with increasingly intrusive monitoring of students. These changes potentially create ethical harms, but current ethical discussions predominantly focus on legal and governance issues. The concerns of the majority of users—namely students—are neglected. Overlooking students’ concerns further increases their vulnerability. We use a student-centred and speculative approach through the Story Completion Method (SCM) to explore how students would potentially respond to intrusive AI monitoring in a higher education setting. Our study included 71 participants who elaborated on the story stems we provided to them. Through a blending of thematic analysis coding and the techniques of developing grounded theory, we reveal that the common responses of students to extensive AI monitoring included impacts on personal psychology, changed behaviour, and cognition. There are likely major disruptions to personal autonomy, identity and educational relationships. If we are to avoid a future ‘big brother’ classroom, further investigations using HCI methods are critical to understanding how to protect students in AI-dominated learning.