The harm that AI can cause
Neerja Singh
In a shocking incident (https://tinyurl.com/aimisuse) at a reputed CBSE school in Bengaluru, a parent of a class 9 student has filed a police complaint regarding the spread of AI-generated inappropriate photos of her daughter on social media. This disturbing event has quite understandably rattled the school community, raising serious concerns about privacy, cyberbullying, and the misuse of technology among students. Another student from the same class also had their images manipulated, though no formal complaint has been filed in that case.
The issue surfaced when the AI-generated images were shared in a private Instagram group chat with around 20 classmates of the victim. Although the affected student was not part of the group, her friends informed her about it. She was deeply disturbed by this violation and suspects that the images were taken from her private Instagram account. This leads her to believe that someone she knows, possibly a friend, could be responsible for this harmful act. This is what one would call a jugular betrayal!
Unfortunately, this is not an isolated incident. It is part of a growing global trend where technology is misused to harm individuals, especially minors. In recent years, there have been numerous cases worldwide involving AI-generated and altered images used to harass and intimidate young people.
In the United States, a significant case occurred in 2020 (https://tinyurl.com/aimischief) when a mother in Pennsylvania was charged with creating deepfake videos of her daughter’s cheerleading rivals. The AI-generated videos showed the girls in compromising positions, causing severe emotional distress to the victims.
In October 2023, some 10th grade girls at Westfield High School, New Jersey alerted administrators that boys in their class had used artificial intelligence software to fabricate sexually explicit images of them and were circulating the faked pictures (https://tinyurl.com/epidemicteenfakes).
Images courtesy: www.clipart.com
It is a wakeup call for those responsible for the health of our youth. Nearly 21% of the Indian population is adolescent (about 243 million). They make a formidable demographic and economic force. And their mental and emotional wellbeing is the nation’s collective responsibility. Given the challenge of meeting their varied needs, life circumstances, and socio-economic conditions, artificially induced mischief is unaffordable.
All these unfortunate incidents emphasize the urgent need for schools to take proactive measures to protect their students from such digital threats. It is impossible to predict what long-term harm could befall these hapless young victims given the persistent nature of online content. What key steps could a school therefore take to check this harmful trend?
The time has come for schools to implement comprehensive digital literacy programs that teach students about responsible technology use, the importance of privacy, and the dangers of sharing personal information online. These programs should also cover the ethical implications of AI and other emerging technologies. Both the perpetrators and the victims deserve to know the nature of this beast known as the unchecked and uncivilized technology.
No such measures will be effective, however, unless and until schools begin to put in place some strong cybersecurity policies. This would include guidelines on the acceptable use of school networks and devices, as well as clear protocols for reporting and addressing cyberbullying and online harassment.
Another potent defence would be to hold regular workshops for students, teachers, and parents on topics like cyber safety, digital footprints, and the risks of social media. This way, schools will build a well-informed community.
Schools will have to prepare themselves to provide counselling and support services for students affected by cyberbullying and digital harassment. These experiences can be devastating, bringing intense shame upon those affected, leaving them to face a lifetime of struggle with their sense of self-worth and identity. Trained counsellors can make a life and death difference at this point by helping victims cope with the psychological impact.
Teachers would do well to stay vigilant and aware of the signs of cyberbullying and digital harassment. They should encourage students to report any suspicious or harmful online activities and ensure that such reports are taken seriously and addressed promptly. It is crucial to foster a classroom environment where students feel safe and supported. Teachers will have to get into the rhythm of promoting inclusivity, respect, and empathy among students and actively discourage any form of bullying or harassment.
Would an evolution of the school curriculum to include discussions on digital ethics, privacy, and the responsible use of technology help students understand the impact of their online actions? Absolutely, and more so when teachers begin to use real-world examples and case studies to illustrate these concepts effectively. Just as important, is for the teachers to engineer an effective communication and collaboration with parents and guardians. Never has the oft quoted dictum of “It takes a village to raise a child” been more true. It has become crucial for teachers to regularly update parents on the school’s policies and initiatives related to digital safety and encourage them to discuss these topics with their children at home.
And technology can help here. It can monitor students’ online activities within the school’s network to help identify potential issues early on. The tricky part in this exercise, however, would be to maintain respect for students’ privacy and act in accordance with legal and ethical guidelines.
The Bengaluru school incident involving AI-generated inappropriate photos is a stark reminder of the challenges posed by advanced technology in the hands of malicious individuals. It highlights the urgent need for comprehensive measures to protect students from digital threats. Schools are being called upon to stay a step ahead by taking proactive steps to educate their communities, implement robust policies, and provide the necessary support to victims. Teachers, as frontline defenders of student wellbeing, play a pivotal role in these efforts. By working together, schools, teachers, parents, and students can create a safer and more respectful digital environment for all.
There is no way to sugarcoat this. We are in a horrific new era of ultrarealistic, AI-generated, child sexual abuse images. The offenders are using downloadable open-source generative AI models, which can produce images to crushing effects. What is even more alarming is the speed of the development and the potential for irreversible mischief it creates.
The author is a generational diversity speaker, author of five generational books and a consultant on working with GenZ; Leading with Social Impact and Suicide Prevention & Destigmatisation. A millennial in spirit, with the benefit of hindsight, she uses her 37 years in media and education to help leverage generational diversity at work and at home. She can be reached at https://www.linkedin.com/in/neerja-singh/