\Recess\
Advice from the Educator’s Playbook

4 Tips for Teaching Algorithm Auditing to Help Students Detect Bias in AI Outputs

A

s students integrate AI into their learning at an increasingly rapid pace, they need the tools to critically assess the outputs it provides—and how much they can trust them. That’s where algorithm auditing comes in.

Yasmin Kafai Headshot

Yasmin Kafai
Lori and Michael Milken
President’s Distinguished Professor
“The question is, since we’re all interacting with AI systems, how do we know whether the output these systems generate is fair or biased?” said Yasmin Kafai, the Lori and Michael Milken President’s Distinguished Professor at Penn GSE. “A one-time interaction might raise some red flags, but in order to figure out whether there is a bias in the results, we need a much more systematic investigation.”

A leading learning designer and researcher, Kafai worked with Danaé Metaxa, the Raj and Neera Singh Term Assistant Professor in Penn Engineering’s Computer and Information Science department, to develop the AI Auditing for High School toolkit to help teachers introduce students to algorithmic bias and guide them through hands-on audits of real-world AI applications. Until very recently, she explained, most people thought that AI systems were too complex for the average person to understand how they work, and thus potential algorithmic bias was a fact of life.

“But algorithm audits have actually shown that you don’t need access to the algorithms, you don’t need access to the data used to train the system,” she said. “By just systematically investigating or auditing different inputs, you can see what kind of output the system generates, and then with some simple statistics you can make a judgment call: Is this biased or not?”

Kafai provided the following advice for teachers looking to implement algorithm audits in their curriculum, whether in a computer science classroom or any subject dealing with media and information literacy.

1.

It’s OK to keep data sets small

Students don’t typically work with as large of data sets as experts do. But Kafai and her colleagues found that when experts have conducted the same data analysis at scale, they have arrived at the same conclusions students did. So you don’t need huge data sets to observe variations and differences in outputs.
An icon of a magnifying glass containing a red play button with bidirectional arrows.
An icon of a magnifying glass containing a red play button with bidirectional arrows.
1.

It’s OK to keep data sets small

Students don’t typically work with as large of data sets as experts do. But Kafai and her colleagues found that when experts have conducted the same data analysis at scale, they have arrived at the same conclusions students did. So you don’t need huge data sets to observe variations and differences in outputs.
2.

Incorporate students’ perspectives and interests to diversify data

In Kafai’s research, students included variables that experts don’t usually consider, such as including age in a gender-bias analysis of image-generator outputs for various professions. They also included professions that wouldn’t typically be found in an expert’s data set, like tattoo artists and nail technicians. “They brought perspectives that have been missing from that research,” Kafai noted. This diversity in students’ perspectives helped enrich the study—and it’s exactly what’s needed to evaluate AI systems.

“Many youths we worked with as advisors . . . said they felt validated in their expertise for the first time because these were topics and issues they were concerned with,” Kafai added.

3.

Be thoughtful about assignment design

Assignments should be tailored to your students’ level. While high school students can work with their own data sets, it may be better to create a light version for middle school students by implementing some data control. For example, teachers may want to create a constrained data set and add biases into it for students to find, Kafai said. That way teachers will already know what the outcomes of students’ work should be.

Keep in mind that algorithmic systems change frequently, which means outputs of live data, like online materials, will likely be different from year to year. From a teaching perspective, developing materials is a big investment, and teachers may want data sets they can rely on for multiple years.

4.

Keep the curriculum grounded in the scientific method

Many teachers and students are already familiar with the scientific method, so building off that provides students the scaffolding necessary to guide them through each activity in the curriculum.

Kafai also emphasized the importance of the scientific method’s sixth step: reflecting on what was learned. Teachers play a critical role in this by organizing classroom discussions and getting students thinking about the process, outcomes, and possible improvements.

“I think this is a very empowering approach,” Kafai said. “You don’t need any programming skill. You don’t actually need technical skills. You just need to be systematic and rigorous, and that’s what we want students to be anyway.”

Looking for MORE ADVICE from experts?

Quote
Looking for
MORE ADVICE
from experts?

These tips are adapted from the Educator’s Playbook, a Penn GSE newsletter that distills faculty research into useful advice for educators and parents. Visit penng.se/playbook to sign up!
EDUCATORS PLAYBOOK
Access the curriculum resources kit Kafai and colleagues developed: penng.se/algoaudit
Back to Issue