Building games and 3D worlds with AI

By Oliver Schmalzl

#KI #3D #Spiele #VR #Open Source

15 October 2026

For a long time, building a video game was work for a whole team: models, animations, game world, programming. This week several finds landed in my collection inbox that show how much this is changing right now.

A fishing game by one person

On Instagram, @darkvex.ai shows a cosy fishing game built by a single person. The work is split into steps:

• Tripo AI generates the 3D models.
• Claude and Blender arrange the models and create animations.
• Unity 6 builds the game world and the game mechanics from them.

Boat, river, fishing: all from one pair of hands. How well the game plays is hard to judge from a short video. The scene may look good. The result still has to be checked.

Bringing characters to life

A model alone does not move. For that it needs a skeleton and animations. Mesh2Motion does this in the browser: load a model, fit the skeleton, try out animations such as walking or running, export. The code is open (MIT licence), the included animations are free to use (CC0).

When the AI understands the room

For games and VR, an AI has to recognise not just images, but rooms. The research project OneCanvas by Bartłomiej Baranowski, Dave Zhenyu Chen and Matthias Nießner combines video frames of a room, with depth and camera position, into a single all-round image. A normal image model can read this panorama and answer questions about the whole room, for example where something is. This is research, not a finished tool. But it shows the direction.

Scanning the real world

The opposite way: capturing objects from the real world as a 3D model. There are now handheld scanners for home use, for example the Creality Pika. Whether the quality is good enough for games, you would have to try for yourself.

My view on it

I am currently bringing old games into virtual reality. What is often missing there is exactly what these tools provide: 3D models, animations, rooms you can step into. Some of this I already create with the help of AI.

That does not change an old rule: good games are not made by tools, but by decisions. What should feel good? What can go? No AI answers these questions.

Sources: Instagram @darkvex.ai · Mesh2Motion on GitHub · OneCanvas on arXiv · Instagram @simplifyinai · Creality Pika