I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com
Few things:
1. Opus 5 is just as good for this tbh and cheaper.
2. They don't generate optimized 3d models. They have high poly count for simple geometries.
3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a lot more details like windows, doors etc with tools like Meshy. Can post a tutorial if there's interest.
The models need some more RL to be able to do this autonomously.
what's going on with these overlays?
the readme is presenting those as though they are evidence of the accuracy of the project, but its completely unable to line up the shots
These things always look impressive but I fail to see how they are usable beyond demos or simple games. In my experience the topology is often a mess and texturing is difficult as well
I wish you would provide more information. About time, costs, reliability (how many duds did you have? did you need to do any babysitting or etc) and so on
But it is a very impressive looking demo, for sure.
This demo is based on the wonderful (and actually hand-crafted) Three.js. It is a beautiful demo. The model expounded on the prompt and produced a very convincing web product based on Three.js. Wonder why wouldn't author try to produce this in plain WebGL... my guess - it is much harder and costly to do so.
So, I wonder when is Anthropic (or OpenAI (or... NAME_IT)) going to release one single stable super-working library that does... just anything, that we can then reuse the way these guys demonstrate somebody model's caps, while actually standing on the shoulder of giants.
Because they do stand on the shoulder of gigantic work done by Three.js team. Same goes for demos based on D3, imgui, etc.
This is really cool, but $33 for a single generated world makes it hard to see this being useful for games just yet. Not to mention how this would go in a much larger project
Cool demo! That's funny. I literally sketched out something similar 30 minutes ago on paper. I wonder if there was some recent, semi-related article/link that subconsciously prompted the relevance.
This reminds me of the experience VRML wanted to be 3 decades ago. The only difference is that links went to other worlds living on another server. But that seems trivial to implement with this.
Cool. But the question remains, what is the point of this? I don't mean to pick on this project specifically, but it's indicative of AI usage in general. You end up with something that's like "wow an AI made this" but it's not really anything that interesting or useful in and of itself. I've yet to see a single agent generated piece of software above that bar.
this is really neat. I wonder if people will start building open world games or "AR" games in the vein of Pokemon Go and Ingress based on similar tech. I've had a similar idea for a long time but don't think it was feasible before now due to AI
I don’t really care about the real world, I want someone to make a thing where you take scenes from movies and shows and can then walk around in those worlds constructed by AI, to very high fidelity, and maybe in VR. Someone go do this, I’ll be on the lookout for the hackernews article.
fable 5.1 generated an interactive 3D union square, and the agent filmed its own tour guide vid inside it. you can walk Powell to Stockton, read the actual storefronts, cross a working intersection, watch a cable car go by. went inside Apple and the Nintendo store, lower level included
Fundamentally it’s an image model predicting out frames of a sequence.
It’s not enough to call it just an image model though. Not quite video either because it’s so specific to rendering images of the world.
Thus, “world model”?
The problem I have with “world” is it should imply so much more depth. To me this is actually a “POV image model” or “First-person perspective model” (maybe we call this an FPM).
There could also be TPMs (third-person models).
This maps better to modern game and 3D dev phrasing, where “world model” might imply 3D object collections spatially organized in a cohesive file format - complete with interactivity, audio, and all the baseline constituents of what one might call a “world”.
FPMs and TPMs, think about it. Reserve POVMs for photographic realism.
I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com
Few things:
1. Opus 5 is just as good for this tbh and cheaper.
2. They don't generate optimized 3d models. They have high poly count for simple geometries.
3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a lot more details like windows, doors etc with tools like Meshy. Can post a tutorial if there's interest.
The models need some more RL to be able to do this autonomously.
what's going on with these overlays? the readme is presenting those as though they are evidence of the accuracy of the project, but its completely unable to line up the shots
https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...
https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...
This is the kind of demo that would really benefit from a youtube demo of 2-3 min
Fake edit: there is a longer video here 1 min long: https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...
I would be especially curious to see the NPC person/car logic and if they're on rails or what, that's a pretty good NPC density for a demo.
This isn't what I expected when I heard "world model"!
These things always look impressive but I fail to see how they are usable beyond demos or simple games. In my experience the topology is often a mess and texturing is difficult as well
I wish you would provide more information. About time, costs, reliability (how many duds did you have? did you need to do any babysitting or etc) and so on But it is a very impressive looking demo, for sure.
This is beautiful! I would love if we could model out a whole city, ideally using a much cheaper model.
Nice! I had a similar positive experience treating OpenStreetMaps as the highest tier data source and then using screenshots to tune the aesthetics.
I wonder how high fidelity we can get these views using just ThreeJS and Fable5.1 iteration cycles.
Here is my Palisades Tahoe world that I made in a similar way: https://ski-red-dog-face.vercel.app/
Good work! I'm curious about how much token consumed to have this... Would it be a business secret?
This demo is based on the wonderful (and actually hand-crafted) Three.js. It is a beautiful demo. The model expounded on the prompt and produced a very convincing web product based on Three.js. Wonder why wouldn't author try to produce this in plain WebGL... my guess - it is much harder and costly to do so.
So, I wonder when is Anthropic (or OpenAI (or... NAME_IT)) going to release one single stable super-working library that does... just anything, that we can then reuse the way these guys demonstrate somebody model's caps, while actually standing on the shoulder of giants.
Because they do stand on the shoulder of gigantic work done by Three.js team. Same goes for demos based on D3, imgui, etc.
This is really cool, but $33 for a single generated world makes it hard to see this being useful for games just yet. Not to mention how this would go in a much larger project
I am making a spiritual successor the Rainbow Six Rogue Spear
Opus has been great at building the game engine but it does struggle with world building for me.
Any tips welcomed!
Cool demo! That's funny. I literally sketched out something similar 30 minutes ago on paper. I wonder if there was some recent, semi-related article/link that subconsciously prompted the relevance.
This reminds me of the experience VRML wanted to be 3 decades ago. The only difference is that links went to other worlds living on another server. But that seems trivial to implement with this.
Cool. But the question remains, what is the point of this? I don't mean to pick on this project specifically, but it's indicative of AI usage in general. You end up with something that's like "wow an AI made this" but it's not really anything that interesting or useful in and of itself. I've yet to see a single agent generated piece of software above that bar.
Be nice to turn some of these into Counter Strike maps... mhmm...
Did anyone compare it with gpt-6?
added the hand-painted style kyoto: https://github.com/PhiloLabs/fable51-worlds/tree/main/kyoto-...
this is really neat. I wonder if people will start building open world games or "AR" games in the vein of Pokemon Go and Ingress based on similar tech. I've had a similar idea for a long time but don't think it was feasible before now due to AI
this is not a world model
Started the Matrix
This looks great.
I don’t really care about the real world, I want someone to make a thing where you take scenes from movies and shows and can then walk around in those worlds constructed by AI, to very high fidelity, and maybe in VR. Someone go do this, I’ll be on the lookout for the hackernews article.
Very good
“Fable 5.1 World Modeling” is an incredibly misleading title! This has very little to do with Fable and using an unexpected sense of “world model”
fable 5.1 generated an interactive 3D union square, and the agent filmed its own tour guide vid inside it. you can walk Powell to Stockton, read the actual storefronts, cross a working intersection, watch a cable car go by. went inside Apple and the Nintendo store, lower level included
check source code + more worlds soon
[flagged]
[flagged]
[dead]
I shitpost on this website constantly and trash Anslopic on a daily basis.
I am your biggest hater.
But god dammit if I’m not also your biggest respecter.
reaches out for a handshake
You guys are onto something with this one.
Keep going.
You crazy bastards!
You crazy fuckin bastards you hear me?! Hahahahah
WOO!
WOO!
Y U NO github pages?
Or where can I explore the demo app? It seems like it's just a static page!
All you need to do is create the gh-pages branch! And then you have it at https://philolabs.github.io/fable51-worlds/kyoto-higashiyama
Fundamentally it’s an image model predicting out frames of a sequence.
It’s not enough to call it just an image model though. Not quite video either because it’s so specific to rendering images of the world.
Thus, “world model”?
The problem I have with “world” is it should imply so much more depth. To me this is actually a “POV image model” or “First-person perspective model” (maybe we call this an FPM).
There could also be TPMs (third-person models).
This maps better to modern game and 3D dev phrasing, where “world model” might imply 3D object collections spatially organized in a cohesive file format - complete with interactivity, audio, and all the baseline constituents of what one might call a “world”.
FPMs and TPMs, think about it. Reserve POVMs for photographic realism.