TSE: Story Pipeline
May 14, 2014
6 minute read

When I started coding TSE, I made a conscious choice: I decided that, for this game, worse is better. For example, I think Java is really a terrible language in a lot of ways, but for the purpose of this game:

  • I knew how to write Java
  • I knew how to debug Java
  • I knew how to distribute Java applications

Java is full of warts and generally impossible to write productively without using an IDE - but it’s okay. From the age of 14 to 17, I’ve spent way too much time learning the ins and outs of the language and mastering Eclipse.

So for example, I could say that:

  • Java generics are non-reified and generally confusing
  • Lambdas are Java 8 only (and I’m targetting Java 6)
  • The lack of operator overloading means an orgy of String.equals()

To emphasize the first point, consider that:

parseArgs(Collections.<String, String> emptyMap());

Is valid Java. And that doesn’t even begin to scratch the surface.

But for the purpose of this project, none of this matters. Java is weird but, it’s weird I know. So it’s all cool.

LD29 story strategy

So when I had to choose a storage format for the story, I could have gone with something like XML (well-supported in Java), or Twine’s source format (used by other games), or a custom binary format for packing efficiency and load performance, but instead, I used something that is terrible in many ways, YAML, which has the following characteristics:

  • YAML is human-readable
  • YAML is human-writable
  • As a side-effect, it generates nice git diffs

There was one problem though: libgdx doesn’t have YAML loading support out of the box. So, during the jam, I could’ve looked for a way to load YAML from Java, and I could’ve lost up to a day getting it to work.

Instead, I started looking for a command-line YAML to JSON converter, and quickly found a JS one. And then when I ran into issues with it, I found a Python one harder to set up but much less daft.

And then when I got tired of forgetting to run make in the assets directory, I found a watcher program called when-changed that ran it for me. And then an Eclipse plug-in to automatically refresh the workspace when files are modified externally.

In retrospect, doing it earlier would have saved me a lot of frustration. But frustration is a very good indicator that something really, actually needs to be improved. Had I done that earlier, it might have replaced some other thoughts that was currently occupying some brain real estate, and I would have lost them.

Or maybe we would have abandoned the project because I had spent so much time tuning the tech and none focusing on the creative part. Or we would’ve released a game that wasn’t enjoyable because I had no time to focus on user experience (if you think it’s bad now, you have no idea where it came from).

And releasing something technically impressive but seldom enjoyable is something I’ve done before and that I’m shit-scared of doing again. That fear acts as a healthy inhibitor to prevent me from spending too much time in my ivory tower!

Post-LD29 progress

During LD29, it quickly became clear that editing a single 1KLOC file was hardly convenient. It works, and has served us well, but if we are to spend a few months adding more content, we are going to want a better tool.

And that’s where worse is actually worse. If the whole game was done with Twine, we would already have a nice, web-based story editor (and the legacy desktop one) with graphs and everything.

But we’re not using Twine. And our story engine does things Twine doesn’t have built-in (such as being place-aware, having an inventory and stat system, etc.), and our engine does NOT do things Twine does because we don’t need them. And as cute as the Twine web editor is, it would quickly become cumbersome to edit something as massive as our story.

In fact, a web-based editor is not a bad idea at all - the web is still very much a document platform, and has become apt at editing as well. But maybe the manually-organized spatial graph approach does not scale well? Maybe we’d rather have an automatically-generated, filterable graph and story-wide local search?

Maybe we could even add a condition editor in there easily? And some kind of ‘savestate crafting’ system so we can easily test various playthrough scenarios? And maybe it’d be cool to be able to see what we’ve changed in the current edition session, compared to, say, the last git commit?

Hacking all that on top of Twine’s web editor is surely not impossible, but it would take me a really long time. For example, their editor uses Backbone, which is a fantastic piece of software that I have absolutey no experience with.

Instead of having an ad-hoc language, our story engine uses a simple document structure easily represented as a JavaScript object tree. And we know JavaScript can parse and emit JSON easily. On the other hand, parsing YAML and emitting YAML is kind of painful in the browser.

But wait, maybe the game can act as some kind of API server, serving the story as JSON and accepting a modified JSON version, that it would convert back to YAML and store. And that’s exactly what I did. I implemented a YamlParser that implements libgdx’s BaseJsonParser, which constructs a JsonValue-based DOM.

Since I was already using libgdx’s JSON facilities, in a single line of code, I was able to let the game parse the story.yml file directly, making the whole yaml2json dance obsolete. The next step was to let the web-based editor upload an updated version of the story as JSON and save that back to story.yml, in YAML. Since snakeyaml’s emitter looks like libyaml’s, I already knew how to do that.

Finally, since we now have YAML to JSON and back conversions all over the place, I needed a way to make sure that no data was lost or altered in the process. Writing a simple test that loaded all the game’s .yml files, converted them to json, parsed that, and emitted yaml again - and then compared the doms, quickly reassured me that everything was working fine.

To try things out, I regenerated story.yml as if it had been opened by the editor and saved without any modifications, and as expected, the diff was beautiful:


Recently, I’ve come to think of parts of life as spending part of a capital.

Staying up all night is spending part of a health capital you’ll have to top up at some point. When you’re being a dick to a close friend for any reason, eventually you run out of patience capital. When in a creative discussion, you pick your battles because you have limited political capital.

And when coding something, there’s a fine line between “sticking to your guns” (the things you already know well) and getting lost in a maze of technology you’re barely getting started with.

The more experience you have, the more you realize how little you actually know. But also, the better your bullshit radar becomes. And with countless years spent recklessly trying to will things into existence ex almost-nihilo, you get some sort of instinct of how much you can afford to learn in a given project.

This article is part of a series about the development of The Stanley Enigma