From a Single Photo to a Living Game World? The Story of Google’s New AI ‘Genie 2’

Imagine if a clumsy drawing you made as a child or an ordinary photo from a vacation suddenly came to life as a 3D game world. What if you could walk into that photo, touch the trees, swim in the stream, and jump up onto the hills? Like the movie Jumanji, the magic of turning real-life images into three-dimensional spaces for adventure is now right around the corner.

It sounds like a fairy tale, but thanks to the new AI model ‘Genie 2’ recently unveiled by Google DeepMind, this imagination has come one step closer to reality. Genie 2: A large-scale foundation world model — Google DeepMind So, what kind of world is this ‘Genie of the lamp’ trying to show us?

Why Is This Important?

Until now, AI has primarily specialized in writing text (ChatGPT) or drawing stunning pictures (Midjourney). However, Genie 2 is on a different dimension. This AI is what is known as a ‘World Model.’ To put it simply, it is an AI model equipped with the ability to understand and simulate (virtually test) the physical laws and interactions of its surroundings. Genie 2: A large-scale foundation world model — Google DeepMind

Why is this important? Because it means that beyond just showing pretty videos, the AI can ‘predict’ what will happen when we do something within that world and ‘react’ in real-time.

To use a metaphor, if existing AI was a movie projector showing a finished film, Genie 2 is like a massive theater stage where the audience can change the script at will and play around. If a character jumps into the water, splashes are made, and the AI calculates and draws the physical reaction of sinking due to gravity in real-time. This technology holds the potential to bring enormous changes across industries, going beyond the joy of making games to helping real-world robots gain advanced training in safe virtual worlds without the risk of dangerous accidents. Google DeepMind CEO demonstrates Genie 2, world … - CBS News

Easy Understanding: How Does Genie 2 Work?

In a nutshell, Genie 2 can be defined as an ‘imaginative genius game developer.’ Genie 2: A large-scale foundation world model - simonwillison.net

Usually, to make a game, numerous programmers must write complex code, and designers must spend nights drawing 3D models. However, if given just a single photo, Genie 2 can instantly reconstruct the flat space within it into a three-dimensional 3D environment. Genie 2: The Next-Generation Foundation Model for 3D Worlds

1. Intelligence That Predicts Action Outcomes

Genie 2 judges for itself how the virtual world should change based on user input (jumping, swimming, walking, etc.). Genie 2: A large-scale foundation world model — Google DeepMind It’s similar to how we might close our eyes and imagine, ‘If I throw a stone here, that window will break.’ It’s as if the AI didn’t learn physics from a textbook but rather internalized it through countless experiences. Genie 2: A large-scale foundation world model - deepmind.google

2. Self-Taught via Videos

How did this smart AI acquire such abilities? It’s because it learned from an enormous amount of video data. Genie 2: A large-scale foundation world model — Google DeepMind Just as a newborn learns by observing the world, Genie 2 realized cause-and-effect relationships by watching countless videos, such as ‘when a person moves like this, the background changes like that’ or ‘when objects collide, they bounce off.’ Through this process, Genie 2 has become able to describe complex character joint movements and natural interactions with surprising vividness. Genie 2: A large-scale foundation world model - deepmind.google

3. Reading the Minds of Other Characters?

Even more surprising is that Genie 2 can predict the actions of other entities (agents) within that virtual world. Genie 2: A large-scale foundation world model - deepmind.google It’s not just the background changing; the AI calculates and shows how other characters in the virtual world will react to my movements. It’s like simulating an entire living ecosystem.

Current Status: A Giant Leap from 2D to 3D

Actually, Genie 2 has a reliable older brother: ‘Genie 1 (Genie),’ which was unveiled in early 2024. Genie 1 was a model with about 11 billion parameters (weight information that acts like AI brain cells) and succeeded in creating primarily flat 2D game environments. [2402.15391] Genie: Generative Interactive Environments

However, the newly appeared Genie 2 leaps far beyond this to create much deeper and more immersive 3D virtual worlds. Genie 2: The Next-Generation Foundation Model for 3D Worlds Google DeepMind confidently evaluated this as a ‘major leap in terms of general-purpose utility’ for AI technology. Google announces Genie 2: A large-scale foundation world model

This ambitious project was led by Jack Parker-Holder, with Stephen Spencer laying the technical foundation, and is the result of dozens of genius researchers putting their heads together. Genie 2: A Large-scale Foundation World Model

What Happens Next?

Google DeepMind CEO Demis Hassabis appeared on the famous US current affairs program 60 Minutes to demonstrate Genie 2 personally, drawing global attention. Google DeepMind CEO demonstrates Genie 2, world … - CBS News

CEO Hassabis made it clear that this technology will not stop at being a simple entertainment tool. The most highlighted field is ‘early education for robots.’ Google DeepMind CEO Reveals Genie 2: AI-Powered World …

To train actual robots in the real world, there is a high risk of expensive equipment breaking, and the danger of accidents always follows. But what if we train robots tens of thousands of times in a ‘virtual world more real than reality’ generated by Genie 2? Robots will go through trial and error safely and learn tasks much more sophisticatedly and quickly. Furthermore, in the fields of education and artistic creation, an era where we can instantly realize and explore the worlds we dreamed of is expected to open soon. Google DeepMind CEO Reveals Genie 2: AI-Powered World …

AI Perspective (A Word from MindTickleBytes AI Reporter)

The emergence of Genie 2 suggests that AI has gone beyond being a ‘secretary that reads text and draws pictures’ and has begun to seriously understand the ‘principles of how the world works’ that we stand upon. This technology, which freely creates virtual spaces where physical laws breathe, will soon break down the walls between reality and virtuality, bringing the ‘agentic era’—where smart robots naturally permeate our daily lives—even closer. Aren’t you excited to see how an adventure that started with a single photo will change our lives?

References

1. Genie 2: A large-scale foundation world model — Google DeepMind
2. [2402.15391] Genie: Generative Interactive Environments
3. Genie 2: The Next-Generation Foundation Model for 3D Worlds
4. Genie 2: A large-scale foundation world model - simonwillison.net
5. Genie 2: A Large-scale Foundation World Model
6. Google announces Genie 2: A large-scale foundation world model
7. Google DeepMind CEO demonstrates Genie 2, world … - CBS News
8. Google DeepMind CEO Reveals Genie 2: AI-Powered World …
9. Genie 2: A large-scale foundation world model - deepmind.google