Deepfakes to AI Avatars: A Clear Guide to Synthetic Media

In 2017, an anonymous user on Reddit posted an algorithm that utilises existing artificial intelligence algorithms to create fake realistic videos that swap faces. This brought attention to AI-generated synthetic media. In addition, it was open-source, giving people the license to experiment and improve on it.

This reveal was not the first introduction of synthetic media. Its origins can be traced as far back as the 90s, if not earlier, with the use of Computer-Generated Imagery(CGI) and manipulation in Hollywood films like The Terminator.

Since its introduction, synthetic media has progressed over the years. This has led us to this point, where we would discuss the ethical and societal challenges, its applications, future, and how it enhances creativity.

Let’s get started!

Synthetic media, also known as AI-generated media, refers to the artificial production, manipulation, and modification of data and media by automated means. This involves Hyper-realistic video and audio recordings that use artificial intelligence and deep learning to create imitations.

Synthetic media has undergone various transformations over the years. If we look at the movie industry, we will see that this technology was used to create certain life or extra-terrestrial forms that seemed real.

For example, the movie Jurassic Park, released in 1993, compared with the latest Jurassic World released in 2022, shows the difference in how they improved on making dinosaurs more realistic. Although this falls under the Computer-Generated Imagery (CGI), it is also a form of synthetic media. Modern forms of synthetic media are possible due to deep learning algorithms such as Generative Adversarial Networks (GANs) to develop realistic content that may be difficult to differentiate from real or traditional media. It often focuses on visual and audio content; however, text is also a part of this category. Synthetic media encompasses;

Deepfake Videos and Audio: Deepfakes are created using artificial intelligence to generate fake images, audio, and videos. They can manipulate existing content or develop new content to make it appear that a person or an entity is saying or doing something they never did.

Synthetic Speech: Synthetic speech is a computer-generated voice that mimics human speech. It uses text-to-speech (TTS) technology to convert text into speech and AI voice cloning to create a realistic synthetic voice.

The earliest form of synthetic speech was formant synthesis, where the acoustic characteristics of speech were extracted from voice recordings and programmed as rules for recreating voice as digital audio.

This technology produced very robotic speech. However, with the current advancements in AI and deep learning. It is now possible for modern tools to replicate human voices with better accuracy.

Synthetic Images: These are AI-generated visuals or computer-generated visual representations through artificial intelligence. These images mimic real-world images or generate new images instead of the traditional method of using a camera.

Synthetic images include;

• AI-generated art and photos
• Deepfake images of people that do not exist,
• Image manipulation and enhancement can alter or enhance existing or new versions of images.
• Virtual Avatars and Computer-Generated Imagery (CGI) that are used in films and games.

Chatbots and Virtual Assistants: Chatbots and virtual assistants like Alexa and ChatGPT use Natural Language Processing to generate human-like responses. Some virtual assistants use AI-generated voices to mimic human sounds.

The benefits of synthetic media include the following:

Fast Content Creation: Synthetic media makes it possible to create quality content, saving time, energy and resources.

Reduced Cost of Production: Compared to traditional methods of creating content, content generated by AI can bring down the cost of production significantly by reducing or eliminating the need to hire.

Enhances Creativity: Individuals and creators can use AI tools to create unique content- such as visuals, audio or text to express their styles and creativity beyond the traditional means, especially since these tools often require little to no cost to experiment with.

Diverse Use: Synthetic media can be applied across diverse industries, including education, journalism, entertainment, marketing, and influencing.

We live in a world where we consume a lot of media, from social media to advertisements, movies, music, and videos, which can be easily manipulated.

We’ve seen videos of celebrities saying things that they didn’t say, videos on YouTube with fake persons advertising a product, etc. There are also apps where you can use a celebrity voice to sing songs they didn't sing, amongst many other things.

Here’s a picture of President Trump being arrested that caused an uproar on the internet, and some people thought it was real.

Another Example involves a Belgian visual effects artist named Chris Umi, who created a TikTok account posting videos of Tom Cruise, the movie star, engaging in different acts like singing, cooking, and much more. People thought this was the real Tom Cruise, but, the creator Umi debunked it and admitted that he was the one behind the videos.

According to Chris Ume, he teamed up with a Tom Cruise impersonator with a similar face structure, voice, and mannerisms. Ume only deepfakes Cruz's face and stitches it to the video and sound of the impersonator.

He achieves this by training a deepfake model, analysing images of Tom Cruise from various angles and expressions. The AI learns to map the impersonator's movements, recreating Tom Cruise's expression; in other words, when the impersonator is smiling, the model maps Tom Cruise's smile on the impersonators face.

Ume believed that as long as it was stated that the videos were a "Parody" and he was not doing anything to harm Tom Cruise's image, it wasn’t a problem. His act was for entertainment and raising awareness.

In the interview with 60 Minutes, Ume even deepfaked the host's face and made him appear as his young self. It is Incredible to see what's possible with this technology.

As incredible as it is, it’s hard to deny that it is a cause for concern knowing that it will be used to cause chaos, similar to how hackers on the internet look for loopholes in networks and systems to exploit for personal gains.

A few concerns include;

Misinformation: Deepfake images and videos can be used to create content of public figures advocating for the wrong cause. It can be used for political manipulation and defamation of a person's character.

Identity Theft and Intellectual Property Concerns: AI-generated deepfakes can be used to impersonate individuals, leading to fraud, financial scams, and reputational damage. This also raises Copyright and Intellectual Property concerns, as a person's likeness or voice may be used without permission.

Distrust of Digital Content: Deepfakes cause a lack of trust in digital media, as it blurs the line between real and fake content. When seeing is no longer believing," it's hard to trust the authenticity of the content.

Due to these concerns, there has been a call to regulate this issue and methods to ensure that deepfakes or synthetic media aren’t used for the wrong things.

There are no worldwide guidelines or standards against deepfakes. However, countries like the US are introducing legislation to regulate synthetic media and deepfakes, aiming to address the risks related to cybersecurity, user privacy, intellectual property, and election integrity.

This is to safeguard individuals, institutions, and organizations against the misuse of this technology. Some laws introduced include;

• The Deepfakes Accountability Act: In the US, it aims to protect national security from threats caused by technology while providing legal assistance to victims harmed by deepfakes.

• EU AI ACT: This ACT states that systems that generate or manipulate images, audio and video content are required to meet transparency standards. This means that service providers must inform users when they're interacting with AI systems.

• The Competition and Markets Authority(CMA): CMA is a regulatory body in the UK responsible for protecting consumer rights. While there's no specific regulation against deepfakes, certain CMA regulations ensure that consumers are protected from being misled or exploited. These regulations can help protect users against deepfakes and synthetic media exploitation.

There are many ways to spot deepfakes, including;

• Check the fingers and hands - Look for the number of fingers or unrealistic hand structure.

• Inconsistencies in skin tone and texture - Pay attention to skin variations, especially around the face and neck.

• Unnatural Shadows - Shadows may appear in unusual places or behave inconsistently with the lighting.

• Blinking Patterns - Look for unnatural blinking patterns of individuals in videos

• Blurring - In some deepfake videos, you may notice a blur at the point around the forehead where the fake face has been blended with the original.

• Weird or unnatural mouth movements - Mismatched mouth movement may indicate a video is AI generated or altered.

Generative AI tools will become more sophisticated, making it very difficult to detect the difference between synthetic and real content.

New avenues for more interactive, personalized, and immersive experiences will emerge, enabling realistic simulations and digital environments.

This highlights the importance of digital literacy in recognizing the possibility of manipulated media.

Examples of tools/ software that can be used to generate AI voice, Avatars, text, and videos include;

Synthesia

Synthesia is an AI video software. It can be used to create digital avatars and synthetic voices, and you can convert text to video with AI in 140 languages.

Murf.ai

Murf.AI is a free AI voice generator that can be used by brands to create realistic voiceovers for social media ads, commercials, and promotional videos. You can convert text to voiceovers with over 200 voices in 20+ languages, and create voice clones with near-perfect accuracy of the original.

ChatGPT

ChatGPT is an AI chatbot trained to respond to instructions and provide accurate responses.

Writesonic

Writesonic is an AI writing tool used to generate content for marketing, social media, ads, and more, it can create high-quality text in various human-like tones, such as persuasion and formal styles.

Synthetic media is an incredible technology that has transformed the way content is created and presented. Individuals and creators can use software tools to generate content that may consume time and resources when using traditional methods of creating content.

The ease of synthetic media is great for creators and businesses; however, it is accompanied by certain dangers and can be weaponized when in the wrong hands or used incorrectly.

It’s safe to say that, just like everything, synthetic media has both pros and cons, and it’s up to us to use this technology appropriately.