Addressing deepfake porn doesn’t require new criminal laws, which can restrict sexual fantasy and promote the prison system

After deepfake pornography of Taylor Swift went viral on the social media platform X (formerly Twitter), Swifties sprung into action. They organized to report violations of X’s “Synthetic and Manipulated Media” policy and flooded the platform with real images of Swift in an attempt to alter X’s algorithm.

The incident renewed calls for federal legislation regarding deepfake porn. But whether we need to “defeat” deepfake porn by censoring and criminalizing it is up for debate — or at least it should be.

As a criminologist and sexuality studies scholar with expertise in the legal regulation of sex and sexual expression, the push to conflate deepfake porn with misogyny and sexual harm is concerning, as is the call for new criminal laws.

Are deepfakes sexual violence?

Deepfake refers to the use of artificially intelligent (AI) machine-learning applications to generate original but “fake” audio, images or videos that may appear authentic. Deepfake pornography (DFP) refers to products that are sexually explicit in nature.

According to a 2023 report by cybersecurity firm Home Security Heroes, DFP makes up 98 per cent of all deepfake videos online, and 99 per cent of DFP features women. Notably, 94 per cent of these women work in the entertainment industry.

An earlier study of DFP by Deeptrace Labs found that of those in the entertainment industry, most of the 10 most frequently represented individuals were actresses from western countries, followed by South Korean K-pop singers.

Read more:
AI can now create fake porn, making revenge porn even more complicated

Given their gendered, sexual and seemingly “non-consensual” nature, DFP has been widely described as gender-based sexual violence requiring greater civil and criminal regulation of both AI and deepfake porn producers.

Others, however, suggest that deepfake porn may be deeply constitutional sexual expression, and that new laws should be put off until more research about the impacts of pornographic deepfakes on those depicted, as well as on internet users, can be conducted.

Deepfake porn raises concerns about false representations — for instance, falsely depicting an individual as sexually active, into a certain type of sex or involved in the porn industry. Whether this constitutes sexual violence — even if a person is distressed by fake videos of them — is not self-evident.

Many valid reasons exist for why deepfake porn may be created and shared, and why it should not be interpreted or criminalized as “image based sexual abuse” or “virtual rape.” These reasons include, but are not limited to the social value of sexual fantasy — including seemingly “deviant” fantasies — and the need to resist prison expansionism and the carceral state.

Need for new laws?

Deepfakes, as with the cheapfakes that preceded them, can be created for malicious purposes including harassment, spreading disinformation and extortion. In instances where the use of one’s image is deeply upsetting to the individual depicted, legal avenues such as civil privacy, intentional infliction of emotional distress, invasion of privacy laws that address “false light” (making false or misleading claims about a person that cause harm to them) and take-down orders may help address their concerns.

When it comes to deepfake porn and minors, Canada’s child pornography and its Intimate Images provisions likely apply. And in cases where DFP images and videos are used to harass or extort individuals, laws already exist to address these harms. On or offline, however, there are reasons to resist pro-criminalization strategies.

Queer and sex-radical feminists have long established that even though sex and gender are related, theories of gender oppression cannot wholly explain sex and sexual politics. Importantly, anti-sexual violence feminists and anti-carceral scholars have pointed out the limits and harms of using criminal law to respond to sexual violence for sexual violence victims, the accused and society more broadly.

Before we can determine whether sexual violence is the best framework for describing and responding to deepfake porn, we need a better understanding of deepfake porn prosumers — those who produce, consume and share their creations — as well as the importance of sexual fantasy, at the individual and collective levels.

Deepfake porn prosumers

It’s becoming increasingly difficult to interview or conduct research with deepfake porn prosumers given how widely they are described as depraved, as the embodiment of “toxic geek masculinity,” and as driven by an interest in owning women’s bodies.

Research suggests, however, that many deepfake porn creators are hobbyists who are more interested in contributing “to the development of such technology as solving an intellectual puzzle… rather than as a way to trick or threaten people.”

It’s likely that most deepfake prosumers are men, given what we know about the demographics of computer programmers and porn consumers.

Insights from clinical practice suggests that when men do create “fake porn” misogyny rarely serves as a key motivator. Clinical psychologist David J. Ley observes that more of these cases are “driven by feelings of loss, shame, hope, and fantasy than by misogyny and anger.” Similar to Photoshopped “porno collages,” deepfakes serve as a means to explore fantasies that are likely impossible to fulfill.

But is sexual fantasy a valid reason to create and share deepfake porn on public and paid platforms?

Sexual fantasy and deepfakes

Sexual fantasy is more complicated and more important to our sex lives and to our social well-being than people typically realize or acknowledge. For many, sexual fantasy is private and limited to their mind’s eye.

Others, however, see sexual fantasy as something to be manifested as written text, images or digital files, and publicly shared for free or for a fee.

Critical race feminist scholars have demonstrated that sexual fantasy is both a product of and productive of our complex realities, but that a line can also be drawn between fantasy and reality given the important roles that fantasy plays in our individual and collective lives.

At the individual level, a study that surveyed 1,516 adult cis men and women about their sexual fantasies found that more than half of the respondents — 51.7 per cent of women and 61.9 per cent of men — fantasized about sex with a celebrity.

Sexual fantasy research has also helped establish that few sexual fantasies are statistically unusual or rare. This includes fantasies which have previously been deemed perverted or atypical, such as those that involve violence or humiliation. These fantasies are not only common, but are also “unlikely to be revealing of actual behavior.”

According to Ley, reasons for publicly sharing sexual fantasies range from seeking approval, demonstrating technical prowess and playing with taboo to bonding with those who share the same interests or to arouse others in the way they are aroused so as to feel less alone for having these interests and desires.

As legal scholar Andrew Gilden writes:

the actual process of coming to terms with one’s sexual identity often entails extensive fantasizing, experimentation, education, and social interaction. And these processes are often far less romantic, much less ‘dignified,’ and far less ‘PG’ than envisioned by the evolving legal narratives of sexuality.

Consent and fantasy

Thinking about deepfake porn through the lens of sexual fantasy also helps us make sense of lack of consent in DFP. Consent does not factor into people’s sexual fantasies in the same ways as it does their physical sexual relations: I don’t need permission to fantasize about someone, but I do need permission to have sex with them.

Consent is a primarily legal term, that, at its most general, means voluntary agreement to engage in sexual activity. The use of consent language to refer to the creative process of DFP as “image-based sexual abuse” or “virtual rape” shuts down a nuanced conversation about sexual harm and freedom.

Expanding definitions

Ultimately, sexual fantasy cannot fully explain the phenomenon of deepfake porn. But failing to acknowledge the limits of gender-based sexual violence frameworks comes with its own harms, including the ever-growing definition of sex crime and the expansion of the carceral state.

We need to think carefully about the social and cultural motivations and intent of content creators, in addition to the potential effects of their creations. We need to consider whether expanding the scope of criminal law to address emotional harm in virtual spaces will bring about the changes we want to see, including the reduction of sexual violence. We also need to acknowledge that criminal law has largely failed to prevent, and indeed perpetuates, emotional and physical violence at a level that requires great awareness and care.

Concerns about sexual autonomy should inform debates about emerging technologies, but alternative frameworks for making sense of and responding to deepfake porn should be considered before we censor and criminalize deepfake porn producers, consumers and products.