Thank you, Madam Chair.
Thank you for inviting me to speak today to the whole committee. I'm Blair Attard-Frost. I'm a Canada CIFAR AI chair and research fellow at the Alberta Machine Intelligence Institute and an assistant professor in political science at the University of Alberta, where my work addresses issues of equity in the governance of AI.
If carefully governed and thoughtfully adopted, some applications of AI do have the potential to enable greater inclusion and opportunity in the Canadian economy. However, to deliver on the promise of AI, there are various harms we must also prevent.
With funding and supports from CIFAR, Amii and SSHRC, my research team and I have been working since July on reviewing literature and a global sample of over 1,500 real-world AI incidents to identify gender-based harm caused by a variety of AI use cases. Our study is still ongoing, but our initial findings reaffirm a decade of existing research, while also adding new insights into areas of concern and policy options.
We found that AI systems have the potential to enable gender-based harm across many application areas, and evidence of harm is often obscured by limited transparency practices. The most frequent type of harm we found in our review is the use of generative AI to produce hateful content or non-consensual sexual deepfakes. These kinds of incidents account for 35 of the 93 incidents of gender-based harm we've identified so far, and the victims are overwhelmingly women, girls and 2SLGBTQI+ individuals.
In health care, we find biased diagnostic tools and decision support systems. For example, a 2020 study by a team from the University of Toronto, Vector Institute and MIT found that X-ray classification models trained on public medical datasets can exhibit racial and gender biases, resulting in disparate error rates between men, women, white patients and Black patients. In 2024, a suicide prevention algorithm used by the United States Department of Veterans Affairs prioritized care for white men over female veterans and survivors of military sexual trauma.
In HR and recruitment, we find gender bias in hiring tools. In 2017, an experimental résumé screening tool made by Amazon was decommissioned when it was found to score women job applicants lower than men. A 2020 study by a group at Cornell University found unclear or limited debiasing across 18 hiring assessment tools. In 2024, the U.K. Information Commissioner's Office conducted an audit of several AI recruitment platforms and found widespread issues of inaccuracy and bias.
Facial recognition, gender recognition and other forms of biometric classification pose risks of discrimination, particularly for women of colour and for people whose gender differs from their sex assigned at birth. A 2018 study looking at three facial analysis models showed the lowest error rates on lighter-skinned male faces and the highest error rates on darker-skinned female faces. In 2023, a pregnant Black woman in Detroit was wrongfully arrested on the basis of false information provided by a facial recognition tool. Notable biometric incidents affecting transgender people include flagging for additional screening by airport body scanners, loss of access to services requiring facial ID due to changes in facial structure and proposals to use gender recognition to restrict access to gender-segregated spaces.
Canada already has some tools at our disposal to intervene in these impacts, but we will also need new legislation. The Charter of Rights and Freedoms and the Canadian Human Rights Act guarantee protections against discrimination on the basis of attributes such as sex, sexual orientation and gender identity in legal matters, services such as health care and employment-related matters. Human rights law doesn't cease to exist when decisions are made by an automated tool rather than by a clinician or a hiring manager. In the age of AI, we must re-enshrine the protections guaranteed by the charter and the CHRA. We should require developers and operators of AI tools with potential human rights impacts to mitigate risks of discrimination and to be transparent when discrimination does occur.
On addressing harmful generative AI systems, the government has recently made progress by tabling Bill C-34 and amending the Criminal Code through Bill C-16, adding new criminal offences for distributing non-consensual sexual deepfakes. This approach can be further strengthened through stronger independent oversight by criminalizing creation and possession of these deepfakes, prohibiting a broader scope of imagery and requiring operators of generative AI services to take more proactive measures to prevent gender-based harm before it occurs.
Finally, recent security incidents involving AI agents infiltrating corporate and government systems have prompted concern about catastrophic AI risks. As is the case in other types of catastrophes, such as natural disasters and pandemics, vulnerable groups are especially at risk when catastrophes do occur. If we want to protect women, girls, 2SLGBTQI+ people and other vulnerable groups against the most harmful potential impacts of AI, we will need legislation that imposes stronger requirements for public safety, accountability and transparency on developers of large, advanced AI models, while also updating Canadian cybercrime law to address advances in AI technology.
Thank you for your consideration. I welcome any questions.
