Deepfake – a combination of ‘deep learning’ and ‘fake’ – refers to advanced artificial intelligence used towards the creation of synthetic yet highly realistic images, audio, and video hoaxes[1]. This can be done by creating entirely original material, all of which is entirely false, or by taking pre-existing material and swapping audio/ visual elements with that of another person. Either way, the impact of deepfakes on public trust and wider society has been widely discussed, especially since the proliferation of fake news in 2016 political events[2] (Trump in the US and Brexit in the UK). While deepfakes were once a somewhat intangible threat, to be dealt with in the future, they have now evolved to a level of sophistication that presents genuine, immediate threats to many aspects of society. The likes of OpenAI’s release of text-to-video generative AI, Sora[3], catalyze such risk. In this article, we will explore how deepfakes can erode trust within the legal system, affecting court procedures, evidence gathering, and even free speech. A Brief History of Technology and Court Evidence Types Evidence plays a crucial role in any courtroom but is only ever as good as every one’s ability to trust in it. We started with witness testimony and physical artifacts which are now trumped by voice recordings, photographs, and videos such as CCTV. Technology-based evidence creates more concrete arguments: a video of someone committing a murder is a bit more convincing than the murder weapon being found in their possession. Hence, “historically, audio and video evidence are considered [the] gold standard”[3]. But with all types of evidence, there is room for manipulation. An eyewitness could lie or misinterpret something they see. A murder weapon can be planted, and fingerprints can be tampered with. Once technology is introduced, these manipulations scale in complexity. Voice editing software and photo-editing techniques, e.g., photoshop, can cast doubt on the authenticity of audio and visual evidence. Where with traditional dishonesty there is often a physical trail of evidence to follow, technological manipulation requires a higher level of skill to be able to trace it, e.g., inconsistent time stamps, audio splicing, and minor discrepancies indicative of tampering[4]. With the evolution of technology, the legal system has therefore had to adapt its best practice procedures over time. With regards to evidence gathering and use, this has meant more robust and stringent measures to test authenticity and reliability, as well as a new direction of forensic sourcing/ testing (digital forensics[5]) and chain of custody checks[6]. However, deepfakes now present an even more significant leap in evidence manipulation capabilities. These sophisticated AI-generated media can convincingly depict individuals saying or doing things they never actually did. Unlike previous forms of manipulation, deepfakes go so far as to blur the line between reality and fabrication, posing unprecedented challenges for courts in assessing the authenticity and reliability of audio-visual evidence. Case Studies The most obvious ways deep fakes can be used to help circumvent the law are: The fabrication of evidence to provide alibis for activities and prove innocence.
The fabrication of evidence showing someone’s guilt and involvement in a crime.
Using the possibility of deepfakes to protest the legitimacy of authentic yet incriminating material7. Our first case study looks at fabricating evidence to incriminate someone wrongfully. In 2019, A deepfaked recording was used in a UK custody battle to discredit the father’s worthiness of shared custody. According to the father’s lawyer, Byron James, a “heavily doctored recording” had been presented to the court in which the father is heard making “direct and violent” threats to his wife[8]. However, after further examination, it was found that the recording presented in court had been manipulated to include words that had not been used by the father. In fact, demonstrative of how increasingly accessible editing technologies have become, the mother had used “software and online tutorials to put together a plausible file”[8]. While James had not previously encountered AI-doctored evidence in the courtroom, he has since commented on how it calls into question what kind of evidence can actually be trusted these days. Had they not discovered the tampering, and managed to obtain the original file and metadata, the mother may have successfully persuaded the courtroom as to the father’s fictitiously violent character. While the outcome of, and those involved in the hearing are confidential, it highlights the dangers of taking audio/visual evidence at face value. Not only this, but with the increasingly widespread ease and access to deepfake technologies, versus the relatively older age of judges, the necessary weariness of deep learning technologies may not be fully understood. Our second case study looks at people claiming real videos are deepfakes. In a more high-profile case, Elon Musk’s lawyers attempted to get a lawsuit against Tesla dismissed by claiming Musk had been the victim of deepfake videos. In 2016, Musk was videoed at a tech conference speaking about the extreme safety of Tesla’s Model S and Model X self-driving autonomous features. The videos from this interview have been on YouTube for 7 years now. However, in 2023, the videos resurfaced when a man died after his Tesla crashed while in self-driving mode, and the man’s family and lawyers cited those 2016 claims. Musk’s lawyers tried to deny their authenticity[7]. Being such a public figure, Musk is indeed the subject of deepfakes, which is what his lawyers tried to argue in this instance. The video here was, of course, real though, and the courts did not buy Musk’s lawyers’ claims. However, this highlights how in the age of deepfakes people can not only fabricate but also deny reality. “As people become more aware of how easy it is to fake audio and video, bad actors can weaponize that scepticism”[7]. Especially for public figures, deepfakes offer a layer of protection and immunity behind which they can hide, and avoid taking ownership of reality. Even for non-public figures, instances like this have been seen in courts before, and continue to increase in commonality. Deepfakes and International Regulation Recent regulations targeting deepfakes have emerged globally, with notable initiatives like California's disclosure requirement for AI-generated political ads[9] and the UK's Online Safety Act[10] mandating social media platforms to crack down on harmful content, explicitly including deepfakes. This said, the majority of deepfake regulation currently only really targets deepfakes used with pornographic intent, rather than to circumvent justice systems. While these regulations offer frameworks for accountability and deterrence, their effectiveness hinges on enforcement, resource allocation, skill, and adaptability. Furthermore, although regulation will only grow in necessity for deepfakes under many applications, governments and policymakers will have to consider the potential for negative unintended consequences. This includes impacts on freedom of speech and expression. Vaguely worded laws could lead to censorship or self-censorship, reducing participation in public dialogue and harming democracy. Balancing the objective of combatting deceptive deepfake media with safeguarding human rights will continue as a nuanced challenge, showing the complexities of the ethical considerations surrounding deepfakes, society, and public trust. References 1 What is deepfake AI? A definition from TechTarget 2 Exposure to untrustworthy websites in the 2016 U.S. election - PMC 3 AI deepfakes are poised to enter court proceedings at time of low trust in legal system 4 Authentication of Digital Audio Recordings 5 Understanding Digital Forensics: Process, Techniques, and Tools 6 Digital Forensics and the Chain of Custody: How Is Electronic Evidence Collected and Safeguarded? 7 People are trying to claim real videos are deepfakes. The courts are not amused 8 'Deepfake' audio evidence used in UK court to discredit Dubai dad 9 California Looks to Boost Deepfake Protections Before Elections 10 New laws to better protect victims from abuse of intimate images - GOV.UK Deepfake – a combination of ‘deep learning’ and ‘fake’ – refers to advanced artificial intelligence used towards the creation of synthetic yet highly realistic images, audio, and video hoaxes[1]. This can be done by creating entirely original material, all of which is entirely false, or by taking pre-existing material and swapping audio/ visual elements with that of another person. Either way, the impact of deepfakes on public trust and wider society has been widely discussed, especially since the proliferation of fake news in 2016 political events[2] (Trump in the US and Brexit in the UK). While deepfakes were once a somewhat intangible threat, to be dealt with in the future, they have now evolved to a level of sophistication that presents genuine, immediate threats to many aspects of society. The likes of OpenAI’s release of text-to-video generative AI, Sora[3], catalyze such risk. In this article, we will explore how deepfakes can erode trust within the legal system, affecting court procedures, evidence gathering, and even free speech. A Brief History of Technology and Court Evidence Types Evidence plays a crucial role in any courtroom but is only ever as good as every one’s ability to trust in it. We started with witness testimony and physical artifacts which are now trumped by voice recordings, photographs, and videos such as CCTV. Technology-based evidence creates more concrete arguments: a video of someone committing a murder is a bit more convincing than the murder weapon being found in their possession. Hence, “historically, audio and video evidence are considered [the] gold standard”[3]. But with all types of evidence, there is room for manipulation. An eyewitness could lie or misinterpret something they see. A murder weapon can be planted, and fingerprints can be tampered with. Once technology is introduced, these manipulations scale in complexity. Voice editing software and photo-editing techniques, e.g., photoshop, can cast doubt on the authenticity of audio and visual evidence. Where with traditional dishonesty there is often a physical trail of evidence to follow, technological manipulation requires a higher level of skill to be able to trace it, e.g., inconsistent time stamps, audio splicing, and minor discrepancies indicative of tampering[4]. With the evolution of technology, the legal system has therefore had to adapt its best practice procedures over time. With regards to evidence gathering and use, this has meant more robust and stringent measures to test authenticity and reliability, as well as a new direction of forensic sourcing/ testing (digital forensics[5]) and chain of custody checks[6]. However, deepfakes now present an even more significant leap in evidence manipulation capabilities. These sophisticated AI-generated media can convincingly depict individuals saying or doing things they never actually did. Unlike previous forms of manipulation, deepfakes go so far as to blur the line between reality and fabrication, posing unprecedented challenges for courts in assessing the authenticity and reliability of audio-visual evidence. Case Studies The most obvious ways deep fakes can be used to help circumvent the law are: The fabrication of evidence to provide alibis for activities and prove innocence. The fabrication of evidence showing someone’s guilt and involvement in a crime. Using the possibility of deepfakes to protest the legitimacy of authentic yet incriminating material7. The fabrication of evidence to provide alibis for activities and prove innocence. The fabrication of evidence showing someone’s guilt and involvement in a crime. Using the possibility of deepfakes to protest the legitimacy of authentic yet incriminating material7. Our first case study looks at fabricating evidence to incriminate someone wrongfully. In 2019, A deepfaked recording was used in a UK custody battle to discredit the father’s worthiness of shared custody. According to the father’s lawyer, Byron James, a “heavily doctored recording” had been presented to the court in which the father is heard making “direct and violent” threats to his wife[8]. However, after further examination, it was found that the recording presented in court had been manipulated to include words that had not been used by the father. In fact, demonstrative of how increasingly accessible editing technologies have become, the mother had used “software and online tutorials to put together a plausible file”[8]. While James had not previously encountered AI-doctored evidence in the courtroom, he has since commented on how it calls into question what kind of evidence can actually be trusted these days. Had they not discovered the tampering, and managed to obtain the original file and metadata, the mother may have successfully persuaded the courtroom as to the father’s fictitiously violent character. While the outcome of, and those involved in the hearing are confidential, it highlights the dangers of taking audio/visual evidence at face value. Not only this, but with the increasingly widespread ease and access to deepfake technologies, versus the relatively older age of judges, the necessary weariness of deep learning technologies may not be fully understood. Our second case study looks at people claiming real videos are deepfakes. In a more high-profile case, Elon Musk’s lawyers attempted to get a lawsuit against Tesla dismissed by claiming Musk had been the victim of deepfake videos. In 2016, Musk was videoed at a tech conference speaking about the extreme safety of Tesla’s Model S and Model X self-driving autonomous features. The videos from this interview have been on YouTube for 7 years now. However, in 2023, the videos resurfaced when a man died after his Tesla crashed while in self-driving mode, and the man’s family and lawyers cited those 2016 claims. Musk’s lawyers tried to deny their authenticity[7]. Being such a public figure, Musk is indeed the subject of deepfakes, which is what his lawyers tried to argue in this instance. The video here was, of course, real though, and the courts did not buy Musk’s lawyers’ claims. However, this highlights how in the age of deepfakes people can not only fabricate but also deny reality. “As people become more aware of how easy it is to fake audio and video, bad actors can weaponize that scepticism”[7]. Especially for public figures, deepfakes offer a layer of protection and immunity behind which they can hide, and avoid taking ownership of reality. Even for non-public figures, instances like this have been seen in courts before, and continue to increase in commonality. Deepfakes and International Regulation Recent regulations targeting deepfakes have emerged globally, with notable initiatives like California's disclosure requirement for AI-generated political ads[9] and the UK's Online Safety Act[10] mandating social media platforms to crack down on harmful content, explicitly including deepfakes. This said, the majority of deepfake regulation currently only really targets deepfakes used with pornographic intent, rather than to circumvent justice systems. While these regulations offer frameworks for accountability and deterrence, their effectiveness hinges on enforcement, resource allocation, skill, and adaptability. Furthermore, although regulation will only grow in necessity for deepfakes under many applications, governments and policymakers will have to consider the potential for negative unintended consequences. This includes impacts on freedom of speech and expression. Vaguely worded laws could lead to censorship or self-censorship, reducing participation in public dialogue and harming democracy. Balancing the objective of combatting deceptive deepfake media with safeguarding human rights will continue as a nuanced challenge, showing the complexities of the ethical considerations surrounding deepfakes, society, and public trust. References 1 What is deepfake AI? A definition from TechTarget What is deepfake AI? A definition from TechTarget What is deepfake AI? A definition from TechTarget 2 Exposure to untrustworthy websites in the 2016 U.S. election - PMC Exposure to untrustworthy websites in the 2016 U.S. election - PMC Exposure to untrustworthy websites in the 2016 U.S. election - PMC 3 AI deepfakes are poised to enter court proceedings at time of low trust in legal system AI deepfakes are poised to enter court proceedings at time of low trust in legal system AI deepfakes are poised to enter court proceedings at time of low trust in legal system 4 Authentication of Digital Audio Recordings Authentication of Digital Audio Recordings Authentication of Digital Audio Recordings 5 Understanding Digital Forensics: Process, Techniques, and Tools Understanding Digital Forensics: Process, Techniques, and Tools Understanding Digital Forensics: Process, Techniques, and Tools 6 Digital Forensics and the Chain of Custody: How Is Electronic Evidence Collected and Safeguarded? Digital Forensics and the Chain of Custody: How Is Electronic Evidence Collected and Safeguarded? Digital Forensics and the Chain of Custody: How Is Electronic Evidence Collected and Safeguarded? 7 People are trying to claim real videos are deepfakes. The courts are not amused People are trying to claim real videos are deepfakes. The courts are not amused People are trying to claim real videos are deepfakes. The courts are not amused 8 'Deepfake' audio evidence used in UK court to discredit Dubai dad 'Deepfake' audio evidence used in UK court to discredit Dubai dad 9 California Looks to Boost Deepfake Protections Before Elections California Looks to Boost Deepfake Protections Before Elections California Looks to Boost Deepfake Protections Before Elections 10 New laws to better protect victims from abuse of intimate images - GOV.UK New laws to better protect victims from abuse of intimate images - GOV.UK New laws to better protect victims from abuse of intimate images - GOV.UK

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

AI and the Law: Deepfakes, Eroding Trust, and the Legal Tightrope

Too Long; Didn't Read

Nimit

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

AI and the Law: Deepfakes, Eroding Trust, and the Legal Tightrope

Too Long; Didn't Read

Nimit

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES