Since light was a factor, Ross had a feeling that the AI proctoring program was racially biased. After all, her light-skinned classmates didn’t seem to have any issues taking their test. Ross is Black.
These kinds of problems are well documented in AI and they stem from the lack of diversity in who creates the software. The outcome is often racist. Recent examples of AI discrimination include incidents from Palantir, IBM, Microsoft, Google and Amazon.
— Janice Wyatt-Ross, EdD (@JaniceWyattRoss) March 26, 2021
While corporations may prioritize business interests over the harmful consequences AI can have on the broader population, schools are different. Children are still developing and discriminatory ed tech can cause lasting damage as young people grapple with their place and purpose in the world. Despite racial bias and ableism claims leveled against popular AI proctoring systems like ExamSoft, ProctorU, and Proctorio, many colleges continue to use proctoring software to administer millions of tests.
This raises questions about what’s acceptable by education decision makers and who is served by AI proctoring services. When remote learning was sudden and new, teachers who were not familiar with online proctoring felt panicked. “Test proctoring companies came along and said, ‘We have a solution to help you address some of these issues.’ And so lots of places found it easier – at least initially – to adopt those kinds of approaches,” said MIT educator Justin Reich, author of the book, “Failure to Disrupt: Why Technology Alone Can’t Transform Education.“
“Teachers have hard jobs and they have to make hard choices, and I’m sympathetic to folks who decide them,” he said. After all, teachers were also navigating uncertainty, anxiety, grief and loss as COVID-19 caused school closures and devastated communities. AI proctoring systems seemed to promise that testing could function normally.
Because these software programs disadvantage Black and brown students – not to mention, those who don’t have adequate internet or share space with family members and siblings – Gilliard said they have no place in schools. “There’s no acceptable level of harm when we’re talking about these kinds of systems. Even if you’re only discriminating against two percent or three percent or 10 percent of your population, that’s completely unacceptable.”
100% Test Scores
Cheating is, however, a big problem, especially during distance learning, when teachers couldn’t walk up and down the rows of desks in the classroom.
Students kept getting 100% on their tests in Julia Anker’s precalculus class when she was teaching online during COVID-19 school closures. But when she gave out a different test that required students to explain how they got their answers, the average grade on the test was significantly lower than usual. That confirmed it for her: “There was rampant cheating,” said Anker. Phones and tech tools gave students the ability to cheat in ways that would not have been possible even ten years ago. “There are these apps where they can scan the problem with their phone camera and it’ll give them the answer,” she said.
Being new to teaching remotely, Anker didn’t feel like there was much she could do at the time. “I just told them, ‘You know what, if you guys are choosing to cheat, this is precalculus – you’re going to have a bad time in calculus next year,’” she said.
Anker wasn’t alone in being unsure about how to address cheating. It has stumped teachers since long before the pandemic. One out of three students admit to cheating online and the same proportion of students admit to cheating in-person. Many educators feel that if cheating is allowed to go unchecked, it puts students on an uneven playing field, cheapens assessments and, in some cases, tarnishes a school’s reputation. To curb cheating, many schools have academic integrity policies in place.
Even with policies, some students will still take their chances with cheating; getting into college has only gotten harder, in addition to the internships and jobs a college degree is supposed to unlock. “We want there to be some kind of academic integrity and there are enormous pressures on students to be academically successful,” said Reich about why students opt to cheat. “They perceive the cost of not succeeding as high.”
Some teachers breathed a sigh of relief when virtual proctoring technologies became available at their schools. Sophie Morton was a live proctor for her fifth grade students in Georgia when they had to take their yearly Measure of Academic Progress (MAP) test online. She monitored her students on Zoom and required them to keep their cameras on during the test. She also used GoGuardian, which allows teachers to see students’ screens. The tool gave her access to data, such as how long students spent on each question.
Ultimately, she was happy to have a way to keep her 5th grade students focused. “I was comfortable using it. I’m seeing your face, looking at your body language. I can see if you get up off the chair,” she said. Morton also emphasized the importance of having a relationship with her students before using surveillance and monitoring technologies. She had taught these same students the year before. “The behavior definitely could have been different or the results could have been different. They know who their teacher is versus if we would have gotten substitute teachers to monitor testing.”
However, Gilliard questions traditional testing and the proctoring services it requires because it applies a misunderstanding of how learning happens.
“Learning is a very social activity,” he said. For instance, a veterinarian who encounters an animal with a rare disease they’ve never seen before might go to a message board or call up a colleague to get more information about how to treat the disease.
“In these old-fashioned or traditional ways of testing, there’s an idea that you’re a solitary person by yourself and the knowledge that you have in your head at that moment somehow represents your capabilities. If you don’t know the answer to a particular question at that time, then you’re somehow seen as lacking or deficient,” said Gilliard.
Teachers claim tests prepare students for their future in the real world, but students don’t see the connection between high-stakes testing and holding down a real job. “You’re going to have Google and all this other stuff at your fingertips,” said Ross. “It’s not like you’re not going to have these resources. So trying to be so rigorous and say you should know this information doesn’t make sense.”
Pushback from students
In the age of COVID, rising prices, climate change and polarized politics, people are rethinking the value of everything. And students are questioning how they learn.
During distance learning, people craved connection, but some students say AI proctoring has frayed the relationship between teachers and learners. While the full picture of the pandemic’s effects on student engagement is incomplete, many schools report that significantly more kids are chronically absent.
Students have been pushing back against these technologies being a part of their learning experience, with petitions springing up in dozens of states across the U.S.
Zoe Harwood, an intern at Oakland-based youth organization YR Media, created Surveillance U to highlight students’ experiences with proctoring software.
“I want to make people more aware of [AI] proctoring and try to protect the little bit of privacy we have. Granted, we live in a day and age where – let’s face it – I don’t even know what privacy means,” said Harwood. “I have grown up my entire life with Google and Apple and Facebook and Instagram and all the major tech companies mining me for every single bit of data I have.”
In Surveillance U, students shared that virtual proctoring feels invasive and adds more anxiety to already stressful circumstances. Additionally, many students spoke about racial bias, telling stories similar to Ross’s about having to stand on tables to get enough light for their faces to be detected.
“There’s this misperception that AI is colorblind when study after study, after study showed that is just not true. And the last thing I think we want to do is automate racism,” said Harwood.
Gilliard urges educators to discuss data and security with students instead of leveraging these technologies to exploit students further.
“They’ve grown up using a lot of these technologies, but they’ve also grown up under the microscope of these technologies,” said Gilliard. Even video monitors, at one time used to make sure babies are sleeping through the night, have become increasingly widespread and used past the baby stage. As a result, kids are becoming used to less privacy and possibly more prone to narcissism. “Some of them don’t know, for instance, that there existed a way of being on the web in the before times when every single action that you did wasn’t tracked,” said Gilliard.
Given the near future of AI proctoring, students have reason for concern.
As school buildings reopened and students are no longer learning from home, these surveillance tools don’t seem to be going anywhere any time soon. The ability to take a test at home remains appealing to those who don’t want to commute to a testing facility; even the SATs will be offered online starting 2024 in the U.S.
Schools may not know it, but they play a meaningful role in teaching students privacy and data practices. Reich does an exercise with his MIT students in which he asks them to list all the data they think their school has on them. “People start with the obvious like, ‘They know my age. They know my grades,’” he said. “And then they’re like, ‘I need to have this smartphone app to use the laundry.’”
School-related apps, campus WiFi and even a keycard used to scan into buildings provide schools with all kinds of information about a student’s movement and online activity.
“We’ve got to think really carefully as educators [about] what kind of world we want to model and invite young people to be in,” said Reich.
Before inviting a new technology into a school, Reich suggests school leaders do a deep dive into what these systems say they offer. “Find out what kind of research there is about them and whether or not the lofty claims that they often make have any bearing in truth and to what extent these companies are engaged in a level of hype that promises things that they can’t deliver.”
Though many proctoring companies say they reduce cheating, there has been no independent research that supports this claim. Reich also encourages schools to ask themselves a simple question: “Will this new technology make students feel like it’s okay to be surveilled?”
How to makes a class “cheat-proof”
Teachers are finding a way to make tests totally cheat-proof, and it turns out, what they’re doing is really just good teaching.
In her role as distance education coordinator, Maritez Apigo was approached by students who said they don’t want to be required to use virtual proctoring services. She worked with a team of instructional designers and accessibility specialists to draft guidelines for online testing. They focused on accessibility and equity and ended up with a guidance memo that shows ways to break away from Scantron tests and virtual proctoring. The majority of educators voted to adopt the guidelines, which lay out the equity concerns with AI proctoring and ways for educators to do “authentic assessment,” which reduces the need for proctoring services.
It takes a lot of time to create tests in general and it’s even more time-consuming to create assessments that are “cheat-proof.” “It actually requires more work to grade, especially if you’re giving feedback to students,” said Apigo. “But you do get to be more creative in your assessment.”
In a biology class, instead of having a 100-question multiple choice test where students have to regurgitate information that they’ve memorized, an authentic assessment method may require students to instead create a brochure that might be found in a doctor’s office on a topic they studied. Alternatively, teachers can provide students with a list of topics and students can choose one to create a final project around. Students are able to demonstrate what they’ve learned while eliminating the ability to cheat because there is not one right answer.
“You need to set up your class keeping cheating already in mind, so it’s already part of your design. [Then] the types of assessments that you give your students are already designed so that students can’t cheat,” said Apigo.
