The rapid advancement of deepfake technology is enabling a disturbing new trend of non-consensual pornographic content targeting unsuspecting women and girls. As deepfakes become more accessible and harder to detect, experts warn this emerging form of online abuse could have devastating impacts on victims.
Most recently, American podcaster Bobbi Althoff found herself the latest victim after explicit deepfake images of her circulated across social media. The 26-year old mother of two young children has amassed millions of followers on TikTok and Instagram through her dry humor and candid parenting advice. But last week, Althoff discovered her name trending online for a very different reason.
“Hate to disappoint you all, but the reason I’m trending is 100% not me & is definitely AI generated,” Althoff posted on Instagram after noticing the hoax. The realistic but fabricated video quickly spread as an apparent “leak,” fueled by users trying to capitalize on the momentum and drive engagement.
Althoff joins other female creators and celebrities like Taylor Swift who have recently faced the proliferation of deepfakes portraying them in graphic sexual situations without consent. The swift virality of these videos delivers lasting harm, even after removal from platforms like Twitter that have policies against such content.
“I felt like it was a mistake or something, that it was bots or something,” Althoff later shared in an emotional video. “I didn’t realize that it was people actually believing that that was me until my whole team called me and were like, ‘Is this real?’”
Exposing An Industry of Abuse
While deepfakes have only recently captured mainstream attention, non-consensual pornographic depictions of regular women and girls have quietly persisted for years. Sophie Compton, director of the documentary “Another Body” chronicling one college student’s experience, said an entire commercial ecosystem has built up around the sharing of these abusive creations.
“Sites dedicated to sharing this abuse are able to grow and make money off of violating consent,” Compton explained. “Creators are getting bolder and bolder because they face no consequences. And as creators get bolder, women are being silenced and shamed.”
With the right tools, anyone can now generate realistic fake imagery themselves rather than relying on advanced technical skills. This low barrier of entry combined with a high financial incentive in the form of website subscriptions has spawned an industry where women’s experiences and consent don’t matter.
“It is not just the psychological harm, the intense depression, the anxiety, but also the economic consequences, because it can lead to further harassment online, online and offline harassment, requiring a lot of victims to invest in security systems or change the way that they go to work or go to school,” said Dr. Mary Anne Franks, president of the Cyber Civil Rights Initiative.
Most victims choose not to come forward out of fear of attracting more attention. But even for public figures like Bobbi Althoff, the impacts can be emotionally devastating and difficult to undo once content spreads across social media.
Deepfakes and Democracy
Beyond individual women, experts warn the proliferation of deepfakes poses a serious threat to public trust in information itself. Genevieve Oh, a deepfake researcher who watched the Althoff situation unfold in real-time online, said the videos were framed as leaked personal content rather than AI fabrications.
“Because of the realism, this fake video was portrayed as a ‘leak’ with many online users trying to pile on and post the footage, incentivizing retweets and comments,” Oh explained. This intentional blurring of truth and misinformation means an increasing number of people could be deceived.
The potential for manipulated media to cause real-world harm became clear during the 2016 election when leaked emails and disinformation campaigns influenced public discourse and opinion. Now deepfakes raise the stakes by allowing the creation of sophisticated forgeries almost anyone can fall for.
Earlier this month, fraudulent explicit images of pop star Taylor Swift previewed the havoc deepfakes could wreak on the 2024 election. Lawmakers responded with alarm over how manipulated media could impact the democratic process if deployed by malicious foreign or domestic actors. With visual evidence rendered unreliable, the informational foundations of democracy risk crumbling.
“We’re seeing a trend where deepfakes are getting more accessible to everyday people, and that means they could be used at a large scale to influence the public,” said Henry Ajder, Head of Threat Intelligence at Deeptrace, an AI detection firm. While social media platforms are racing to deploy filters to identify deepfakes, it seems inevitable some will slip through given the pace of technological change.
The Search for Solutions
So how can society respond to curb harm while supporting free speech and innovation? Many experts argue the absence of strong legal deterrents has allowed the toxic deepfake industry to metastasize. Currently, victims have little recourse if non-consensual content is created or shared about them.
“While individual states are moving toward passing bills that would make sharing this type of content illegal, there is no federal legislation protecting victims or deterring bad actors,” Oh said. On platforms like Twitter where deepfakes run rampant, Oh believes stricter policies and enforcement could help stem the tide.
Others caution that overly broad restrictions also risk limiting acceptable uses of the technology, like parody or satire. But Franks contends the lack of regulation does more long-term damage to free speech. “Failing to prohibit abuse silences more people, especially women and minorities, because it makes the internet an unsafe place for them to be,” she said.
In absence of laws, some experts recommend alternative deterrents like exposing the identities of deepfake creators. But this also raises ethical concerns around vigilantism. Ultimately, many agree curbing abuse will require pressure across multiple fronts, from grassroots advocacy to self-regulation within the AI industry. The coming months will test just how prepared society is to confront this evolving threat.
For victims like Bobbi Althoff trying to reclaim their dignity after having it violated online, progress can’t come soon enough. Until then, the relentless march of technology continues to outpace justice. But public awareness and moral outrage is reaching new heights that could finally turn the tide.
