TSE: Discovering CMake
Jun 8, 2014
9 minute read

In the last article, we talked a bit about the TSE story editor, and about why I made the choice to write the prototype in Java.

Recently we’ve come think that Java is nice for semi-rapid development, but libgdx is still immature as a 3D engine, and that prompted me to start looking for other solutions to release the final version of the game.

Does that invalidate the point I’ve made about Java? No! It still allowed us to quickly prototype what we wanted out of the game. Now that the basis of the game is pretty much set in stone, it’s not a crime to look at other languages to implement the end result.

Ogre3D

I remember Ogre3D from the olden days when I was still maintaining Xith3D. Their demos looked cool, but it was C++, and I didn’t want to have anything to do with that at the time.

Nowadays I’m more open-minded, and I gave it spin. Here’s what I liked:

  • Many things in the engine is stored in simple text files: materials, compositors (for full-screen effects) - this alleviates the pain of recompiling to iterate on the game

  • While not as cutting-edge as commercial engines such as Unreal, CryEngine, well, the Ogre folks have been doing 3D for a while and they’re not afraid of the future.

Since Ogre3D uses cmake as their build system, I had to learn that, too.

CMake

CMake is a generator - in a sense, rock contains a subset of the functionality of CMake, since it outputs Makefiles. But rock also knows how to run the build directly, which is not CMake’s role.

CMake acknowledges the fact that:

  • The dev machine and the build machine are not necessarily the same (and may have different setups)
  • To target different platforms smoothly, one must adapt to various build systems, compilers, etc.

Whereas rock launches gcc (or clang, which is a gcc-like as far as CLI interface is concerned) on each OS and takes care of platform-specific stuff, CMake will happily generate, for instance:

  • Makefiles or ninja build files, for Linux
  • XCode projects, for OSX
  • Visual Studio projects, for Windows

Whereas XCode is a layer cake on top of something vaguely gcc-ish, Visual C++ is an entire beast entirely, with a completely different compiler (that doesn’t support C99, but does support recent versions of C++, for example).

I’ve stayed away from Apple and Microsoft tools as long as I could, but with CMake, there’s no need - it’s possible to set up a relatively painless build environment for all three platforms.

The basics

You can think of CMake as a mostly declarative DSL to set up all the various properties that make up a project or a library (or multiples of each). It is, however, mixed with various imperative statements to perform commands on each platform (in a relatively platform-independant way, sometimes) so that the properties are set to meaningful values.

CMake ships with hundreds of FindXXX.cmake files in most of its distributions, that help find common libraries such as QT, Gtk and the likes. In Ogre-land, however, we’re on our own - most of the dependencies we need are better built manually.

Fortunately, the ogre contributors provide an ogredeps repository where all the basics, from Freetype to Freeimage, are provided. And the ogre default source distribution contains FindXXX.cmake scripts for those libraries as well.

Compiling a CMake-powered project is a two-step process. As far as I can tell, most builds can be done out-of-tree - and it’s just cleaner to do so. A common way to do it would be to create a ‘build’ directory inside whichever project you’re building, run cmake .. in it, and then run the build command that’ll use the build files cmake just generated.

On Linux, it could look like:

mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local ..
make -j4
make install

(Where .. refers to the parent directory, and -j4 is here to compile with 4 threads in parallel)

Variables can be passed with the -D flag. There’s no ad-hoc command-line option such as --prefix because, as opposed to autotools, there’s really very little imposed structure in CMake projects. This makes it very flexible but also very messy when you’re trying to figure out the best practices.

On OSX, it could look more like:

cmake -G Xcode -DCMAKE_INSTALL_PREFIX=/usr/local ..
xcodebuild -target install -configuration Release

And on Windows, one could use the CMake GUI to generate Visual Studio build files, and then from the VS command-line utility, run:

cmake -G "Visual Studio 10" ..
msbuild /p:Configuration=Release INSTALL.vcxproj

I’m too ashamed to post my own CMakeLists.txt files on this blog (plus a lot of it is very project-specific), but it took me maybe a week to really familiarize myself with the tool and its various features.

One problem CMake is tackling since version 2.8 is packaging (or bundling). It means different things on different platforms, but mostly it’s:

  • Copying assets
  • Copying required libraries alongside the executable (via fixup_bundle)

fixup_bundle is an interesting beast - it does some of the things I’ve described in my Cross-platform game distribution article, sort of like a cross-platform dylibbundler.

On Linux, it’ll use ldd, on OSX otool -L, on Windows dumpbin, to find out dynamically linked libraries that an executable depends on. Then it decides whether to copy them and where to copy them depending on various settings and heuristics. I wouldn’t say it’s a joy to use, but when it works it’s a beauty.

CMake also handles icon files, and various App bundle properties (name, vendor, version) that you need to set if you want to do things properly on OSX.

Linux-specific notes

Anything command-line is pretty much guaranteed to work beautifully on Linux with very few hassles - which is not something I can say of OSX (even with the fantastic packaging work of the brew crew for example) or, god forbid, Windows.

However, rewriting the game in C++ from vim is no fun. I’m not that experienced with C++ yet, despite having read a lot about it, written an alternative, made several student projects, and played a whole lot with its older brother C.

So I’ve started looking for IDEs. I started by testing out KDevelop. While it did handle CMake-based builds, it was too unstable for my likes, and overall quite clunky to use.

I remembered a Rich Geldreich post (of the VOGL team, Valve’s OpenGL profiler/debugger) about QT Creator and how it was pretty much the best option for linux as a C/C++ IDE - turns out he was not lying!

Since I still use Linux as my main development platform, QT Creator serves my auto-completion and debugging needs very well. It’s flexible, fast, well-designed, and supports FakeVIM which is almost enough to fool me when I’m too tired to notice the missing keybindings.

I was originally using the CMake’s Makefiles generator, but now I’m using the ninja build system, which feels more turn-of-the-century than its distant cousin.

OSX-specific notes

I have a love-hate relationship with OSX - my MacBookPro is my most versatile portable device and often sleeps with me, but there’s just many things about the Apple way that doesn’t appeal to me.

The way things are now, OSX is stuck in an eternal conflict between the Unix way (executables and dylibs) and the Apple way (.Framework bundles and .App bundles). I eventually gave up compiling ogredeps because it was too messy. Instead I just grabbed one of the prebuilt packages from Sourceforge.

I still hate everything about XCode. Even at version 5 (Apple just announced XCode 6 with Swift support at the time of this writing, which is.. not a priority for me), it’s still slow to start, slow to react, and while I get the intention behind the strongly-opinionated design of its interface, it just doesn’t resonate in me. It’s probably perfect for Apple-only dev. I’m just not that kind of guy.

CMake’s fixup_bundle was designed for OSX first, so it works well there without many surprises, but CMake will get paranoid about which Frameworks it is allowed to copy or not - based on some heuristics (path prefixes) it will decide that something should be copied, or that it should be present on all systems.

For example, if you install Cg.framework (nVidia’s Cg shader toolkit) system-wide, it’ll think it ships with every Mac and that you definitely don’t need it in your app bundle! (which is, of course, false)

So, as with everything that has the slightest bit to do with C++, nothing quite works out of the box. You need to get your hands dirty and surf from mailing list to forum to 90s-design-documentation to eventually get what works for you (and sometimes settle on an ugly, but working-for-you-right-now kind of solution).

ninja is not an option on OSX with Ogre 1.9.0, as some variables are not escaped/replaced correctly and any build attempt fails miserably. So, I’ll be rolling with xcodebuild until that’s fixed (possibly with Ogre 2.0.x).

Windows-specific notes

Ye who likes command-line, flee while you still can. Windows’s cmd.exe is a celebration of everything that is wrong with text user interfaces. Sure, there are alternatives, but that’s what Visual Studio offers you if you want to run Microsoft’s xcodebuild equivalent, msbuild, from a command line, rather than open up the full IDE each time.

I use a Git bash to get a few unix commands, and for all my git needs. I also run cmake from there. In fact, I only use the Visual Studio command shell to run msbuild.

One annoying habit of CMake users on Windows is to postfix debug dlls with _d. So you’ll have OgreLayer.dll or OgreLayer_d.dll - same goes for config files, etc. I get the idea, but it does complicate CMakeLists.txt quite a bit sometimes.

I’m not using fixup_bundle on Windows yet, rather, I’ve manually listed all necessary DLLs for CMake to copy, following up on the Ogre3D sample CMake project. I might migrate to it later, if it’s not too much hassle. I don’t think everyone knows that fixup_bundle works on Linux and Windows too!

Conclusion

Getting up to speed with CMake took a lot more effort than I had anticipated, but I’m happy I’ve done so - it allows me to have a shared C++ codebase that compiles on all three target platforms with their native, first-class citizen toolchains (ninja/gcc, XCode/clang, and MSVC).

Now that I have a base game skeleton working everywhere, I want to reimplement TSE with these tools up to feature-parity with the Java version. As always, I’ll blog about it right here.

Thanks for reading!

This article is part of a series about the development of The Stanley Enigma