It’s interesting to me that we continue to see such pressure on video and world generation, despite the fact that for years now we’ve gotten games and movies that have beautiful worlds filled with lousy, limited, poorly written stories. Star Wars movies have looked phenomenal for a decade, full of bland stories we’ve all heard a thousand times.
Are there any game developers working on infinite story games? I don’t care if it looks like Minecraft, I want a Minecraft that tells intriguing stories with infinite quest generation. Procedural infinite world gen recharged gaming, where is the procedural infinite story generation?
Still, awesome demo. I imagine by the time my kids are in their prime video game age (another 5 years or so) we will be in a new golden age of interactive story telling.
Hey siri, tell me the epic of Gilgamesh over 40 hours of gameplay set 50,000 years in the future where genetic engineering has become trivial and Enkidu is a child’s creation.
I think that's a bit of a trap. It's not impossible, but by default we should expect it to make games less fun.
The better you make this infinite narrative generator, the more complicated the world gets and the less compelling it gets to actually interact with any one story.
Stories thrive by setting their own context. They should feel important to the viewer. An open world with infinite stories can't make every story feel meaningful to the player. So how does it make any story feel meaningful? I suppose the story would have to be global, in which case, it crowds out the potential for fractal infinite storylines - eventually, all or at least most are going to have to tie back to the Big Bad Guy in order to feel meaningful.
Local stories would just feel mostly pointless. In Minecraft, all (overworld) locales are equally unimportant. Much like on Earth, why should you care about the random place you appeared in the world? The difference is that on Earth you tend to develop community as you grow and builds connections to the place you live, which can build loyalty. In addition, you only have one shot, and you have real needs that you must fulfill or you die forever. So you develop some otherwise arbitrary loyalties in order to feel security in your needs.
In Minecraft there's zero pressure to develop loyalty to a place except for your own real-life time. And when that becomes a driving factor, why wouldn't you pick a game designed to respect your time with a self-contained story? (Not that infinite games like Minecraft are bad, but they aren't story-driven for a good reason).
Now, a game like Dwarf Fortress is different because you build the community, the infrastructure, the things that make you care about a place. But it already has infinite story generation without AI and I'm not sure AI would improve on that model.
I think it's all about how you spin it in, imagine:
- SimCity where you can read a newspaper about what's happening in your city that actually reflects the events that have occurred with interesting perspectives from the residents.
- Dwarf Fortress, but carvings, artwork, demons, forbidden beasts, etc get illustrations dynamically generated via stable diffusion (in the style of crude sketches to imply a dwarf made it perhaps?)
- Dwarf Fortress, again, but the elaborate in-game combat comes with a "narrative summary" which conveys first hand experiences of a unit in the combat log, which while detailed, can be otherwise hard to follow.
- Any fantasy RPG, but with a minstrel companion who follows you around and writes about what you do in a silly judgy way. The core dialogue could be baked in by the developers but the stories this minstrel writes could be dynamically generated based on the players actions. Example: "He was a whimsical one, who decided to take detour from his urgent hostage rescue mission to hop up and down several hundred times in the woods while trying on various hats he had collected. I have no idea what goes through this mans mind..."
I'm not sure if there is a word for it, but the kernel here is that everything is indirectly being dictated by the players actions and the games existing systems. The LLM/AI stuff isn't in charge of coming up with novel stories and core content, they are in charge of making the game more immersive by helping with the roleplay. I think this is the area they can thrive the most.
> by default we should expect it to make games less fun.
How so?
I could totally see generative AI add a ton more variety to crowds, random ambient sentences by NPCs (that are often notoriously just a rotation of a handful of canned lines that get repetitive soon), terrain etc., while still being guided by a human-created high level narrative.
Imagine being able to actually talk your way out of a tricky situation in an RPG with a guard, rather than selecting one out of a few canned dialogue options. In the background, the LLM could still be prompted by "there's three routes this interaction can take; see which one is the best fit for what the player says and then guide them to it and call this function".
Worst case, you get a soulless, poorly written game with very eloquent but ultimately uninteresting characters. Some games are already that today – minus the realistic dialogue.
> I could totally see generative AI add a ton more variety to crowds, random ambient sentences by NPCs (that are often notoriously just a rotation of a handful of canned lines that get repetitive soon), terrain etc., while still being guided by a human-created high level narrative.
Yes, sure, but that's not what I was responding to. AI adding detail, not infinite quest lines, is possibly a good use case.
> Worst case, you get a soulless, poorly written game with very eloquent but ultimately uninteresting characters. Some games are already that today – minus the realistic dialogue.
Some games, yes... why do we want more of those? Anyway, that's not the worst case. Worst case is incomprehensible dialogue.
I actually find the same issue with prequels, especially for the ones that really hit a chord (like the original Star Wars). After knowing what is going to happen in those stories, I just can't get invested in a character who I know either makes it for sure, dies before getting to the "main" story, or doesn't matter because they don't have any connection to my canon of the plot arc. Same-universe spins-offs fit this for me as well.
OTOH, lots of games come with DLC that add new stories with the same mechanics. There might be some additions or changes, but if you really like the mechanics, you can try it with a different plot. Remnant II has sucked a ton of my time because of that.
By definition, an infinite game is as boring as real life. To make it interesting, the engine must be able to search for a good story based on player actions. You can see this today already. Many games will guide the player into one or a few predefined stories. A better game would not have them predefined, but generated on-the-fly based on player actions.
I think less than infinite stories it would be awesome to see infinite paths in a designed setting. Skyrim is fantastic for the many choices and the way they permanently affects the world and trajectory. But there's ultimately a primary overarching story, you just hit it from a variety of perspectives.
Dwarf Fortress is the state of the art in procedural interactive story generation. Youtube channels like kruggsmash show how great it is in that role if you actually read all the text.
But that doesn't translate well to websites, trailers or demos. It's easier to wow people with graphics.
But the narrators aren't narrators; they're just different settings for the relative frequencies of events. Dwarf Fortress is still a more robust "story generator", as the vast majority of what occurs in Rimworld is still basically random events, disconnected from prior events or context.
I guess I see what you mean, but the way they handle difficulty scaling, and the existence of quest chains, by being more fake, is closer to how stories are told, than the more 'realistic' simulation of Dwarf Fortress.
IMHO Humans will still create the overarching stories, what LLMs will do is help fill in the expensive blanks that make adding stories to a world hard.
For example, right now if you save an entire village from an attacking tribe of orcs, only a handful of NPCs even say anything, just a nice little "thanks for saving our town!" and then 2 villages over the NPCs are completely unaware of a mighty hero literally solo tanking an entire invading army.
Why is that?
Well you'd need lots of, somewhat boring but important, dialogue written, and you'd need tons of voice lines recorded.
Both those are now solvable problems with generative AI. AI generated dialogue is now reasonably high quality, not "main character story arc" high quality, but "idle shop keeper chit chat" quality for sure, it won't break immersion at least. And the quality of writing from AI is fine for 2 or 3 sentences here and there.
I'll be soon releasing a project showing this off at https://www.tinytown.ai/ the NPC dialogue is generated on a small LLM that can be ran locally, and the secret of even high quality voice models is that they don't require a lot of memory to run.
I predict that in another 4 or 5 years we'll see a lot of models ran at the edge on video game consoles and home PCs, fleshing out game worlds.
Creativity is the one area where LLMs are completely unimpressive. They only spit out derivative works of what they’ve been trained on. I’ve never seen an LLM tell a good joke, or an interesting story. It doesn’t know how to subvert expectations, come up with clever twists, etc. they just pump out a refined average of what’s typical.
Claude can make some interesting guitar tabs if you prompt it to transcribe an instrument/music that wouldn't normally be something a rock guitar player would be influenced by.
It is like saying the paint brush and canvas lack creativity. Creativity is not a property of the tool, it is a property of the artist.
We also have a very poor understanding of human creativity from selection bias.
Last weekend I found a book at the library that was Picasso's drawings 1966 to 1968. There must have been 1000-1500 drawings in this book. Many were just half finished scribbles.
The average person seems to believe though that the master artist only produces masterpieces because they didn't bother to look at all the crap.
Humans are obviously influenced by others but we can also invent novel things that didn't exist before. LLMs trained on the outputs of LLMs collapse into gobbledygook whereas humans trained on humans build civilisation.
Humans can be said to create from a combination of life experiences, artistic influences, and pure imagination.
LLMs have no life experiences, are only familiar with the most mainstream literary works with the most mainstream internet discussions, and use a fancy RNG formula on the next most likely word as a not so great substitute for imagination.
They're different because they're trying to find the most likely output, and humans usually. You can ask and LLM to make weird combinations and use unusual framings, but it's only going to do so once you've already come up with that.
I asked ChatGPT “ Write a one paragraph pitch for a novel that combines genres and concepts in a way that’s never been done before.”
I’m not going to claim this is Pulitzer-worthy, but it seems fairly novel:
> In Spiritfall: A Symphony of Rust and Rose Petals, readers traverse the borders of time, taste, and consciousness in a genre-bending epic that effortlessly fuses neo-noir detective intrigue, culinary magic realism, and post-biotechnological body horror under the simmering threat of a cosmic opera. Set in a floating, living city grown from engineered coral-harps, the story follows a taste-shaper detective tasked with unraveling the murder of an exiled goddess whose voice once controlled the city’s very tides. As he navigates sentient cooking knives, ink-washed memory fractals, and teahouses that serve liquid soul fragments, he uncovers conspiracies binding interdimensional dream-chefs to cybernetic shamans, and finds forbidden love in a quantum greenhouse of sentient spices. Every chapter refracts expectations, weaving together genres never before dared, leaving readers both spellbound and strangely hungry for more.
Well, all of the conceptual elements it used are in the training set; it just combined them in ways that don't even make syntactic sense. Yes, I know we "just" combine ideas too when we're creating. My point is that I don't think it was producing new concepts, just slamming words together in grammatically acceptable ways. Do any of its absurd phrases mean anything to you? They don't mean anything to me. I could create something conceptually sound based on its absurd phrases, but that's still me doing the work where the LLM is acting as an algorithmic name generator.
I'd be curious if it could explain those concepts and use them in consistent ways. If so, I'd be curious how novel it could really get. Is it just going to be repackaging well-trod scifi and fantasy devices, or studied philosophy? Or could it offer us a story with truly new understandings? For example, to my knowledge, House of Leaves is something truly novel. It's probably not the first book with intentional printing errors, or with layered narration, or with place-horror, etc. But I think House of Leaves is pretty widely considered a sort of "step forward" for literature, having a profound impact on the reader unlike anything that came before it.
(A really serious discussion will require analyzing exactly what that impact is and how it's novel.)
they also struggle to know when to break the rules of english, make up words, introduce pun, bounce between tones, write with subtext, introduce absurdity, allude to other ideas etc.
I'd say its less the work they have been trained on, and more what they have been reenforced to do, which is stay on topic. it causes them to dwell instead of drift.
No Man's Sky is kind of what you're looking for, except you may notice its quests (and worlds) become redundant quickly...I say quickly, but that became the case for me after like 30 hours of game play.
That's the kicker, LLM driven stories are likely to fall into the same trap that "infinite" procedurally generated games usually do - technically having infinite content to explore doesn't necessarily mean that content is infinitely engaging. You will get bored when you start to notice the same patterns coming up over and over again.
Procgen games mainly work when the procedural parts are just a foundation for hand-crafted content to sit on, whether that's crafted by the players (as in Minecraft) or the developers (as in No Mans Sky after they updated it a hundred times, or Rougelikes in general).
Yeah, generative AI can create cool looking pictures and video but so far it hasn't managed to create infinitely engaging stories. The models aren't there yet.
I'd argue that the same principle applies to pictures, there are many genres of AI image that are cool the first time you see them, but after you've seen the exactly the same idea rehashed dozens of times with no substantial variety it starts wearing really thin. AI imagery is often recognizable as AI not just because of charactistic flaws like garbled text but because it's so hyper-clichéd.
I wonder if there's some threshold to be crossed where it can be surprising for longer. I made a video game name generator long ago that just picks a word (or short phrase) from each of three columns. (The majority of the words / phrases are from me, though many other people have contributed.)
I haven't added any words or phrases to it in years, but I still use it regularly and somehow it still surprises me. Maybe the Spelunky-type approach can be surprising for longer; that is, make a bunch of hand-curated bits and pick from them randomly: https://tinysubversions.com/spelunkyGen/
"There’s no question in my mind that such software could generate reasonably good murder mysteries, action thrillers, or gothic romances. After all, even the authors of such works will tell you that they are formulaic. If there’s a formula in there, a deep learning AI system will figure it out.
Therein lies the fatal flaw: the output will be formulaic. Most important, the output won’t have any artistic content at all. You will NEVER see anything like literature coming out of deep learning AI. You’ll see plenty of potboilers pouring forth, but you can’t make art without an artist.
This stuff will be hailed as the next great revolution in entertainment. We’ll see lots of prizes awarded, fulsome reviews, thick layers of praise heaped on, and nobody will see any need to work on the real thing. That will stop us dead in our tracks for a few decades."
there's only really like seven basic plots; man v man, man v nature, man v self, man v society, man v fate/god, man v technology so we should probably just stop writing stories anyway
It would not surprise me if most people could not tell whether some story about the human condition is human or AI generated. Excluding actual visual artists that have specific context of the craft, most people already can't tell AI art from human art when put to a blind test.
As far as I know know AI art can't really follow instructions so it's actually very, very easy to tell the difference if you aren't biasing the test by allowing vague instructions permitting random results to be considered acceptable.
"Here's a photo of me and my wife, draw me and my wife as a cowboy in the style of a Dilbert cartoon shooting a gun in the air" can't be done by AI as far as I know, which is why artist are still employed throughout the world.
Last time I checked GenAI it wasn't able to handle multiple people, but giving Midjourney a picture of yourself, and asking it to "draw me as a cowboy in the style of a Dilbert cartoon shooting a gun in the air" is totally a thing it will do. Without a picture of you to test on, we can't debate how well the image looks like you, but here's one of Jackie Chan: https://imgur.com/a/6cBrHWd
Are you saying you can upload a picture to mid journey that it will use as a reference?
Jackie Chan is not a good example because he's a famous person it may have been trained on. I used myself as an example because it would be something that is novel to the AI, it would not be able to rely on it's training to draw me, as I am not famous.
IMO this will be the differentiating feature for the next generation of video game consoles (or the one after that, if we’re due for an imminent PS6/Xbox2 refresh). They can afford to design their own custom TPU-style chip in partnership with AMD/Nvidia and put enough memory on it to run the smaller models. Games will ship with their own fine tuned models for their game world, possibly multiple to handle conversation and world building, inflating download sizes even more.
I think fully conversational games (voice to voice) with dynamic story lines are only a decade or two away, pending a minor breakthrough in model distillation techniques or consumer inference hardware. Unlike self driving cars or AGI the technology seems to be there, it’s just so new no one has tried it. It’ll be really interesting to see how game designers and writers will wrangle this technology without compromising fun. They’ll probably have to have a full agentic pipeline with artificial play testers running 24/7 just to figure out the new “bugspace”.
Can’t wait to see what Nintendo does, but that’s probably going to take a decade.
QAing a game built on a framework where fundamental mechanics are non-deterministic and context-sensitive sounds like a special kind of hell. Not to mention that once you find a bug there's no way to fix it directly, since the source code is an opaque blob of weights, so you just have to RLHF it until it eventually behaves.
Seems like there's already a lot of slop on steam and I really doubt it will be difficult for quality content to be highlighted even if the amount of games increases 1000x or more
That has been the case since art was first industrialized with the printing press. Most of them don’t survive but a significant fraction, if not the vast majority, of books printed in the first century were trashy novels about King Arthur and other fantasies (we know from publisher records and bibliographies that they were very popular but don’t have detailed sales figures to compare against older content like translated Greek classics). Only a small fraction of content created since then has been preserved because most of it was slop. The good stuff made it into the Western canon over centuries but most of the stuff that survives from that time period were family bibles and archaic translations.
I don’t see why AI will be any different. All that’s changed is ratio of potential creators to the general population. Most of it is going to be slop regardless because of economic incentives.
Are game ratings reliable on Steam? If yes, then it will be easy to avoid the slop. Or are they overrun with clickbots, like Amazon, where people give five stars for some crap product?
Thanks to high bandwidth Internet, YouTube and smartphones is easier than ever to produce and distribute high quality video. So much good stuff coming from it.
Expect something similar if video games, interactive 3D is cheap to produce.
Filtering is a much easier problem to solve and abundance a preferable scenario.
We already have deluges of free, and almost free, publicly available assets. Getting Over It, a game that deliberately used those, had a running author's commentary on the this phenomenon and in short no, endless assets does not translate to endless creative works; it's seen and treated as trash that nobody wants to use.
Given we have engines that can render complex 3d worlds, can maintain consistency far longer than a minute and simulate physics accurately, why put all that burden on a GenAI world generator like this?
It seems like it’d be more useful to have the model generate the raw artifacts, world map, etc. and let the engine do the actual rendering.
I’d prefer we trained AI politicians by watching Team America World Police and statements made by George Bush and the neocons after 9/11. Maybe AI politicians could learn from their mistakes and stop involving us in foreign proxy wars in the first place. Especially ones that could escalate to nuclear armageddon.
> I want a Minecraft that tells intriguing stories with infinite quest generation. Procedural infinite world gen recharged gaming, where is the procedural infinite story generation?
You're not gonna get new intriguing stories from AI which only regurgitates what it's stolen. You're going to get a themeless morass without intention.
I also find it amusing how your example to Siri uses one of the oldest pieces of literature when you also tire of stories heard a thousand times before.
if you do basic chatgpt prompts in late 2024 asking for dynamic story telling, sure, you'll get what you said. it's super dismissive to think that wont get better over time, or that even with the tools today, that you can't get dynamic and interesting stories out of it if you provide it with the proper framework
> it's super dismissive to think that wont get better over time
When did we start thinking this way? That things HAVE to get better and in fact to think otherwise is very negative? Is HN under a massive hot hand fallacy delusion?
If you compare the historical rate of improvements in computing power and algorithms vs rate of improvements in building scale, you'll find one is a whole lot more likely to reach its goal, even if the rate of progress slows significantly.
Lots of people want that AI grift money and need to be pollyanna true believers to convince others that models that don't know truth are useful decision makers
Actually, all you need to do is to apply structured randomness to get diversity from a LLM. For example in TinyStories paper, a precursor of the Phi models:
> We collected a vocabulary consisting of about 1500 basic words, which try to mimic the vocabulary of a typical 3-4 year-old child, separated into nouns, verbs, and adjectives. In each generation, 3 words are chosen randomly (one verb, one noun, and one adjective). The model is instructed to generate a story that somehow combines these random words into the story
You can do the same for generating worlds, just prepare good ingredients and sample at random.
A story is not just words crammed together that sound plausible. Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots? That paper sounds like it has a scientist’s conception that a story is just words, and not complex trade offs between the start of a story and its end and middle, complexity and planning that won’t come from any sort of next-token generation.
These are “stories” in the most vacuous definition possible, one that is just “and then this happened” like a child’s conception of plot
> Is the AI going to know about pacing? About character motivations? About interconnecting disparate plots?
For LLMs like GPT-4, this all seems reasonable to account for and assume the LLM is capable of processing, given appropriate guidance/frameworks (of which may be just classical programming).
The talent isn’t fungible. The people who make amazing synth patches are rarely the same people who write amazing songs. The people designing great fonts are not the same people who write great books.
We should celebrate creation of tools and capabilities, while also acknowledging that there are many layers yet to be completed before your very cool ad hoc video game request can be delivered.
I'm skeptical- of course the technology could improve, when I look at LLM story output it isn't very well written in terms of creativity. This makes sense when it is basing it's output on variations of "most likely token but with some RNG built in." You end up with stuff that's much less surprising and original than what a good human author would invent.
Star Wars isn't great- but if a single company didn't own Star Wars we'd likely see some companies competing with really great Star Wars stuff- and others sucking at it. The issue is economic.
is PvP multiplayer considered a "story"? Is a football game a "story"? I guess if all you consider for story is "things happen", then a PvP match can be a story, but that's stretching what I would consider "story" for a game. That is the story of the match, but it's not in and of itself a plot story.
Consider EVE Online. The stories it generates are Shakespearean and I defy anyone to argue that they have no plot.
I would go further and predict that stories generated by sufficiently advanced AI can explore much more interesting story landscapes because they need not be bound by the limitations of human experience. Consider what stories can be generated by an AI which groks mathematics humans don't yet fully understand?
Unexplored comes to my mind, it differs from other games with procedural generation in that it generates a graph for the gameplay first and builds the levels around it. It's not necessarily ground breaking, but it has a special feel to it as objects are placed with purpose.
That’s actually a use case I can understand- and what’s more I think that humans could generate training data (story “prototypes”?) that somehow (?) expand the phase space of story-types
Ironic though - we can build AI that could be creative but it’s humans that have to use science and logic because AI cannot?
> Are there any game developers working on infinite story games? I don’t care if it looks like Minecraft, I want a Minecraft that tells intriguing stories with infinite quest generation.
There are a lot of great storytellers who don't have the technical/design skills to bring their ideas to life. AI generation is going to make that part easier, which is a good thing.
“The inhabitants kept addressing the player and begging him to not shut down their world, so this patch raises punishments for breaking the 4th wall to eternal torment for all the perpetrator’s descendants and that seems to fix the problem for now.”
> It’s interesting to me that we continue to see such pressure on video and world generation, despite the fact that for years now we’ve gotten games and movies that have beautiful worlds
Those beautiful worlds took a lot of money to make and the studios are smart enough to realize consumers are apathetic/stupid enough to accept much lower quality assets.
The top end of the AAA market will use this sparingly for the junk you don't spend much time on - stuff the intern was doing before.
The bottom of the market will use this for virtually everything in their movie-to-game pipeline of throwaway games. These are the games designed just to sucker parents and kids out of $60 every month. The games that don't even follow the story of the movie and likely makes the story worse.
Strangely enough this is where the industry makes the vast majority of it's day-to-day walking around cash.
Are there any game developers working on infinite story games? I don’t care if it looks like Minecraft, I want a Minecraft that tells intriguing stories with infinite quest generation. Procedural infinite world gen recharged gaming, where is the procedural infinite story generation?
Still, awesome demo. I imagine by the time my kids are in their prime video game age (another 5 years or so) we will be in a new golden age of interactive story telling.
Hey siri, tell me the epic of Gilgamesh over 40 hours of gameplay set 50,000 years in the future where genetic engineering has become trivial and Enkidu is a child’s creation.