Sora AI: What is it? How to access video generator

The makers of ChatGPT, OpenAI, have announced their latest artificial intelligence engine, which is for creating videos from text prompts.

OpenAI already has Dall-E for generating static images, and now it has moved into videos to help complete its suite of AI tools for all creative purposes.

Sora AI is not yet readily available to use by the public. Still, it has been made available to a select few whose job it is to test the security and stability of AI products, also known as ‘red teamers,’ in February 2024.

But it is already causing a lot of talk on social media, with Sam Altman, the founder of OpenAI, demonstrating Sora’s abilities by replying to people’s prompt requests with the final product, generated by the video generator.

The early signs are that it is equally as impressive as ChatGPT and Dall-E, and it undoubtedly represents a new era of text to prompt filmmaking.

What is Sora AI?

Just as we’ve become so used to generative AI models for text and still images, such as ChatGPT, Dall-E and Google Gemini, Sora AI does the same but for videos.

The AI works in the same way as other generative AI models. It constantly learns from what it sees and consumes, and is trained to provide the most accurate and detailed response to whatever prompt it is responding to.

Sora AI is no different. If you give it a text prompt, for example, “blue boat sailing on the ocean with the sun beaming down”, you will get back a video of just that. You can be as specific or as vague as you like, but the more detail you give the AI model, the better your results will be.

Sam Altman’s Sora AI demonstrations on X in February will give you the best idea of how it works and just how accurate it is with enough detail in the text prompts.

For example, the video below is the result of the prompt, ‘A instructional cooking session for homemade gnocchi hosted by a grandmother social media influencer set in a rustic Tuscan country kitchen with cinematic lighting.’

https://t.co/rmk9zI0oqO pic.twitter.com/WanFKOzdIw

— Sam Altman (@sama) February 15, 2024

How does Sora AI work?

The technology behind Sora AI is the same as the one that lets you search for things on the internet. The more examples the AI sees, the better it will be able to spit the same thing in other images. Eventually, once an AI has seen enough of one thing, it will be able to generate its own version when prompted.

That is, of course, a very simplified way of explaining how generative AI works, but OpenAI has previously provided a more detailed explanation of how its AI model works. Sora AI is trained on publically available and licensed data to know what videos look like at a realistic level. It’s trained to know what it’s looking at and then uses that information to learn how to generate its own versions.

If you ask Sora AI to produce a video of a dog, it will generate results based on all the videos of dogs it has previously seen. It uses visual patches and building blocks to help understand what elements of the video should go where frame by frame. The more it sees and learns, the better and more accurate it becomes.

Sora’s technology is built on a diffusion model, where the AI begins with a messy response before working to improve its output through a series of feedback loops. It also uses transformer technology to use a number of data analysis methods to process vast levels of data, before learning to understand what important parts of the video to keep in, and the least important details to leave out.

What can Sora AI do?

To date, Sora AI is able to produce HD videos of up to a minute in length from text prompts. It is capable of generating ‘real world’, cartoon and CGI-style videos, but currently cannot include audio.

Sora AI can also generate videos from still images, fill in missing frames in existing videos and also stitch multiple videos together. It also has the capability of producing endless loops.

There also examples of it producing simulations of video games, such as Minecraft.

There are reportedly plans to add audio and editing tools to Sora AI, with the latter giving creators the ability to manually fix errors in the AI’s videos. Presumably, the AI will then learn from the manual fixes being made, but that is just a presumption and nothing more at this stage.

There are, however, some limitations, which can be expected. OpenAI has acknowledged these limitations, including people disappearing or transforming into other objects, as well as people moving in ways that just aren’t possible in the real world. OpenAI has confirmed they are already working on fixing these issues.

Once those issues are ironed out and Sora AI grows in sophistication and accuracy, it’s not entirely out of the question that we will be able to use it to create full-blown imaginary worlds, movies and even explore real places in the world without ever needing to visit them physically.

How can you access Sora AI?

Sora AI is not yet available to the public without an invite. Individual creators and testers are being asked to use the AI model and test it so OpenAI can act on feedback to ensure it’s ready for public release.

There is also the very important aspect of security and ethics. Generative AI models have been exploited by criminals and pranksters in the past – such as the sexually explicit deepfakes of Taylor Swift  – so OpenAI is putting a lot of effort into ensuring Sora AI can only be used for good and creative purposes.

That means you will not be able to use Sora AI to generate videos that show extreme violence, sexual content, hateful imagery or celebrity likeness. OpenAI will also include metadata in Sora AI to indicate its videos were generated by AI.

In an interview with The Wall Street Journal, OpenAI’s Chief Technology Officer, Mira Murati, suggested Sora AI will follow the same prompt policies as Dall-E, which means it will refuse to create videos of public figures.

It’s not yet known when exactly Sora AI will be made available to the public, but Murati did hint that it could be nearer the end of 2024. Meanwhile, there are some assumptions that when it does, it will be available as a web app and will eventually include additional features akin to ChatGPT, such as custom bots.

Featured Image: OpenAI // Sora AI