AI Creating 'DOOM' from Scratch? The Secret Behind AI Gaming Without a Game Engine

By Gi Pyeong Lee
AI Gaming Neural Networks Tech Trends
An artistic image overlaying classic DOOM pixel graphics with interconnected neural network nodes.
AI Summary

Google's GameNGen model and experiments using fruit flies and human brain cells demonstrate the potential for artificial neural networks to simulate or control complex games in real-time.

Imagine this: The game you are playing is not a complex program stored on your computer, but a screen that an Artificial Intelligence (AI) ‘imagines’ and draws on the fly every single moment.

Recently, the tech industry has seen a series of fun and astonishing experiments testing the capabilities of AI using the legendary 1993 FPS game ‘DOOM’. It goes beyond just AI that plays games well; attempts have emerged to implement the game itself without a game engine, or even to enjoy the game using the brains of actual living organisms. What on earth is happening between AI and DOOM?

Why does this matter?

These experiments are not simply about creating an ‘AI that plays games’. The core lies in determining how precisely an ‘Artificial Neural Network’ (an AI model that processes information by imitating the structure of the human brain) can learn and mimic the complex rules of the physical world or virtual worlds.(Reference 10)

If we can prove that an AI can generate a game in real-time without a game engine, it means that future metaverse or virtual reality services could operate much more lightly and intelligently than they do today. Furthermore, research connecting the brain’s neural connection map (connectome) of living beings to game control provides important clues about how AI can mimic and assist human or animal intelligence.

Easy to understand: How does AI ‘imagine’ a game?

First, the core of AI, the neural network, is like a ‘giant filter device’.(Reference 3) When we input numbers, the data passes through layers inside the neural network, where it is filtered and amplified to produce a final result.(Reference 6) Metaphorically, this process is similar to passing information through a very complex sieve to extract only the essence.

  1. Simulation without a Game Engine (GameNGen): The ‘GameNGen’ model, developed by researchers at Google and Tel Aviv University, uses absolutely no traditional game engines, such as existing game code or Unreal Engine.(Reference 17, Reference 18) Instead, this neural network learns DOOM’s game data to predict and generate what screen should appear for every frame. It is similar to a veteran painter who has memorized every scene of DOOM and draws them on the spot in response to the user’s operation. The research team succeeded in running this complex video game in real-time.(Reference 2, Reference 15)

  2. A Fruit Fly’s Brain Enjoying DOOM: Software engineer Alex Wormer digitally implemented the ‘connectome’, a map of the neural connections in a fruit fly’s brain.(Reference 1) The approach involves stimulating this neural network with electric signal-like impulses when the game screen changes, and translating the resulting neural activity into game character movements (lateral movement, shooting, etc.).(Reference 19) In a sense, the data from the fruit fly’s brain controls the actual gameplay.

Current Status: How far have we come?

The scope of current research is very broad.

  • AI Hallucinations and Dreams: In some studies, AI visualizes its own ‘DOOM nightmare’ based on game data. The player continues the game while dodging endless fireballs imagined by the AI.(Reference 4)
  • Direct Play by Brain Cells: There is even a case where cultured human brain cells play DOOM. Instead of watching the monitor directly, the brain cells learn to take appropriate actions by receiving quantified numerical information, such as the player’s health or enemy locations, as data through a neural network.(Reference 7)
  • Implementation via Language Models: The recently emerged ‘Torchwright Doom’ compiles the game DOOM itself into a Large Language Model (LLM) and executes it. This is a case proving that language models can function as general-purpose computing platforms, going beyond simple writing tools.(Reference 13)

However, these technologies are still in their infancy. There are many remaining challenges in terms of efficiency to completely replace games that require real-time responsiveness and complex operations. While it is impossible to run the latest games with this technology right now, it suggests that the way AI perceives and expresses the world is changing fundamentally.

What will happen in the future?

In the future, AI will likely evolve toward creating and controlling more complex environments on its own with less data. Games might change from ‘crafted programs’ to ‘experiences that AI imagines in real-time’. Someday, a time might come when you say, “Create a game set in a Martian base with a horror atmosphere,” and the AI creates a game like DOOM on the spot. The day when everything we imagine unfolds like reality through data and neural networks is not far off.

A Perspective from the MindTickleBytes AI Reporter

DOOM has been a playground for AI researchers for over 30 years. But now, it has gone beyond a simple playground to become a stage to prove how deeply AI understands virtual worlds. These bizarre attempts to connect everything from brain cells to fruit fly brains to DOOM ultimately pose questions about how we define ‘intelligence’ and how we can artificially implement it. Isn’t it fascinating that AI is evolving into an ‘intellectual entity’ that can exercise imagination and create worlds, rather than just being a machine that processes data?

References

  1. Google mapped a fly’s nervous system and developers are using it to play Doom
  2. Doom Running on a Neural Network Is a Surreal Dreamscape
  3. A Neural Network Playground
  4. [Neural Net dreaming of Doom Rock Paper Shotgun](https://www.rockpapershotgun.com/dodge-fireballs-forever-in-a-neural-nets-doom-nightmare)
  5. [But what is a neural network? Deep learning chapter 1](https://www.youtube.com/watch?v=aircAruvnKk)
  6. A petri dish of human brain cells is currently playing Doom
  7. [But what is a Neural Network? 3Blue1Brown](https://www.3blue1brown.com/lessons/neural-networks/)
  8. Neural Networks: Zero To Hero
  9. Doom Compiled into an LLM: Torchwright Doom Launches
  10. arXiv:2408.14837v2 [cs.LG] 24 Apr 2025
  11. Google’s AI generated DOOM frame by frame without a game engine
  12. GameNGen: Google’s AI Model Simulates Doom in Real-Time
  13. As soon as Google released the mapping of a fly’s brain and …
AD
Test Your Understanding
Q1. What is the most significant feature of 'GameNGen', developed by Google and Tel Aviv University?
  • It optimizes existing DOOM code
  • It simulates the game using only neural networks without a traditional game engine
  • Users must code it themselves to run it
GameNGen implements DOOM in real-time using only the dynamic generation of a neural network, without any existing game code or engine.
Q2. How does the experiment using a fruit fly's connectome to play DOOM work?
  • The fruit fly is given a controller
  • Game screen information is converted into neural stimuli to control the game
  • The fruit fly watches the monitor directly
Game frame information is converted into stimuli for the fruit fly's sensory neurons, and the resulting neural activity is mapped to game control commands.
Q3. What data does the human brain cell approach to playing DOOM use?
  • The player's mood
  • Quantified game data such as player health and enemy positions
  • Real-time chat information
The brain cells do not watch the screen directly; instead, quantified information from within the game is converted into data via a neural network and transmitted to them.
    AI Creating 'DOOM' from Scr...
    0:00