Lifting the Limitations with Open-Source

Originally posted on Gamasutra

Last month, Tag Games launched our game development engine, at Pocket Gamer Connects in London. The engine is called ChilliSource and is available under the MIT license as a free, open-source solution for creating games.

At Tag, we have long understood the benefits of using and contributing to open-source software, but I was surprised that many developers I speak to see open-source solutions as a risky choice, or know little about the advantages of using open-source software. Having been involved in the ChilliSource venture from day one, it feels like a good opportunity to share some of the reasons Tag use open-source solutions (focusing on game engines) and ultimately what lead us to open up our internal technology in this manner.

Background: The trouble with closed-source…

To date, ChilliSource has been used by Tag Games to create eleven titles on a range of platforms. However, prior to ChilliSource, we used a popular closed-source engine to develop most of our games. We didn’t really have any major issues with the engine itself, but its closed-source nature provided some concerns.

So what do I mean by ‘closed-source’ and ‘open-source’? Really the difference between closed and open source is in the licensing approach; and there are lots of different licenses! For example, it’s possible to have software where you can see the code but not edit it, or edit it but not re-distribute it, or do what ever you like with it. In the context of this blog I’ll make the distinction that you can view, edit and update the code of an open-source project (i.e. MIT license) but cannot do the same on a closed-source one.

With closed-source software you are basically at the mercy of the software vendor. Often this is fine because it is in the vendor’s interests to keep the customer happy with new features and bug fixes; but the problem arises around the issues of time and priority. The features and fixes that the vendor will focus on are those requested by the majority of the community (that’s where the money is right?); this could mean the bug fix you require is at the bottom of the pile. This might be, at best, a minor inconvenience or…it could be you have a release deadline in 2 days, a whole bunch of advertising booked and absolutely no way of fixing your bug in order to meet that deadline. Worst of all the third party company could have their software acquired, potentially stopping support or limiting deployment platforms for your future games.

We’ve had a few issues like this at Tag, where the support team of a third party technology couldn’t possibly work to the timescale we required, and particularly in the case of an engine that’s enabling all of your games (and possibly all of your company’s revenue), this can be a scary prospect. This became such a concern for the company that the team were tasked with finding an engine that would allow us to address these issues ourselves. Unfortunately at that time smartphone gaming was still in its infancy and there were no open-source solutions available that met the requirement of iOS and Android support, plus 2D and 3D rendering. As a result ChilliSource was born.

So what are the main benefits of open source?

Reason one: Taking control…

Control is the main reason that Tag moved away from a closed-source engine and I believe, above all else, that control is the main benefit of open-source software.

Handing control to your programming team means that you are no longer dependent on third parties for new features and bug fixes. If iOS has a new killer framework your team want to take advantage of to wangle a front-page feature on the App Store, then you can. If a bug is found in your live app and it needs to be fixed and updated ASAP, then there is no need to waste time to-ing and fro-ing with remote customer support. Open-source enables you to work to your own timelines and your own priorities

Control also has more far reaching implications than just bug fixes, it can allow you to add new platforms and open up untapped markets. Niche markets can potentially be quite lucrative for small, indie studios but are maybe not recognised by large vendors. Ultimately you get a say in the direction of the engine that you use to make all your games – that’s got to be a good thing.

ChilliSource is written in C++ and one of the main benefits of using a global language, in combination with open-source, is the wealth of available libraries and modules that can be plugged-in. You don’t have to be tied to a particular physics engine or networking library, for example; you can use tools that you are already familiar and comfortable with.

With open-source solutions no one can seize this control away from you. The source does not belong to an individual or company, it cannot be acquired; it belongs to you and the community.

Reason two: Seeing is believing…

Being able to edit and update the source is really powerful but just being able to see it also has huge benefits.

A common misconception is that open-source software is of poor quality. I’m sure there are many poor quality open-source projects out there but I’m equally sure many poor quality closed-source projects also exist. At least with open-source software you can audit it, perform your own code reviews and judge the quality for yourself. Just because a project is open-source does not mean that any old code can be contributed. Most open-source projects have strict guidelines and review processes that must be met before code can make its way into the main branch (however you can always fork the source and manage the code using your own guidelines and process if you wish).

Having the ability to examine the code means that, for example, if you want to know what type of encryption is used in the data store or what networking protocols are used in the multiplayer system, then you are not dependent on the documentation (which can often be non-existent or worse out of date); just open up the source file and see for yourself.

Reason three: Did I mention it was free?

Open-source software is free. I could have lead with this reason and that might have been enough to convince many of the merits of open-source (sometimes there are costs for services such as support but the software is free to use). One of the biggest expenses at Tag is kitting out new employees with expensively licensed software; this can be a huge barrier for smaller, indie companies often preventing them from growing and expanding. Switching to suitable free, open-source alternatives can save money and be hugely rewarding if you’re one of the passionate and dedicated developers contributing their expertise and time to free, open-source projects.

Reason four: Sharing and improving…

One of the main reasons for releasing ChilliSource to the wider industry is that we wanted to build a community. Opening your technology to the community allows you to engage with developers with different experiences, specialisms, needs and ideas and can drive your project forward in ways that you would never have thought of had you kept it in-house. It also gives you the opportunity to hire people who are already familiar with your tools and technology, rather than spending time and money training them.

Knowledge sharing is invaluable – whether you are a student looking to get your hands on source code for learning, a developer looking to make use of a product that takes advantage of an experienced and active community or a contributor looking to give back to the community and get feedback on their code. This is the power of an open-source community where knowledge, features and bug fixes are shared and everyone benefits.

Conclusion: Final thoughts

The games industry is hugely diverse with many talented and dedicated individuals, but it is also constantly evolving and changing. It is important for a company like Tag Games to stay agile and be able to adapt to changes. This need led us to take control of our own middleware by creating ChilliSource, but in order to tap in to the experience and diversity of the game development community we needed to open up this technology and share it with the community. We now have the ability to listen and act upon the ideas and work created by other developers using ChilliSource which will prove beneficial to us all.

If, like us, you want to have a say in the tools that you use to make your products (and ultimately your revenue) and you want to benefit from the experience, knowledge and dedication of a community; then I would really recommend investigating open-source as a viable alternative to licensed products.

Scott Downie, Lead Technology Programmer, Tag Games

Letting Go of OOP is Hard

Let Go!

I read a lot of blog posts these days about the death of OOP, with flocks of programmers abandoning their objects in favour of DOD, reactive or FP (and probably other acronyms I’ve never heard of). I’ve been dabbling quite a lot recently with both DOD and FP and it’s easy to see why many people have embraced that style of programming, but when it comes to doing my day job I still always think in objects; I can’t help it and I also know I’m not alone.

Now, I still believe that object-orientation has a part to play in system design; as do all the other acronyms. There are no silver bullets in programming and all approaches have their own strengths and weaknesses; it’s about picking the best solution for the problem at hand.

The problem is, that despite all the recent bluster, I still don’t think many people know that there are any alternatives to OOP, let alone what their respective strengths are. I watched Mike Acton give a presentation on DOD at a C++ conference and the audience questions at the end led me to believe they all thought he was an absolute nut job. One of the questions that I ask graduates during interviews is what programming methodologies they like and what the strengths and weaknesses are and 90% of graduates answer OOP, strengths: encapsulation, abstraction, etc and weaknesses: O_o. Seriously, most of them have never even considered that there are any other types of programming, let alone weaknesses of OOP; to them OOP is programming. When I prompt graduates about the difficulties of multi-threading OOP applications they shrug “threading is hard”.

Even when you know and have used other programming methodologies it is still hard not to default to OOP. The worst thing is most of us have twisted OOP so far it doesn’t even look like OOP anymore - flat inheritance trees, composition, immutable objects, etc.

I like OOP but I think I, and others, like it too much and a large part of the blame falls on universities (and like so many of the worlds ills - Java). Most uni courses that I’ve seen (including the one I took) have 6 months of imperative training, maybe a little assembly and then blam! straight onto the objects for 4 years. All the course examples are OOP, frameworks are OOP, even marking guidelines deduct points for lack of OOP. Instead of all this brainwashing, teach a variety of techniques and allow students to work out what they like and don’t like for themselves. Ultimately it’s about picking the most appropriate techniques for getting the job done and getting home in time for tea - we can call it objective-oriented programming.

Now if you don’t mind I’ve got to go and send a empty message to this bin, it’s leaking its encapsulations all over the floor…

CodinGame

I stumbled across this great programming site last week called CodinGame. It’s basically a site full of fun programming challenges ranging from AI to maths to data structures.

One of the best things about the site is that it supports a whole bunch of languages including C++11, Scala, Java, Python and C# to name a few. This is brilliant as it allows you to implement solutions to the same problem in different languages and see which is fastest, shortest, simplest, etc. It also poses an added challenge of picking the right language for the problem.

I’m seriously toying of the idea of introducing it at work - perhaps picking a challenge a week and then getting the team together to discuss their implementations.

The only downside is that it’s pretty addictive…

Functionally Challenged - Dabbling with FP

For a while now I have been tempted by the dark art that is functional programming. It’s high order functions and powerful one liners hold a certain allure for me and last year I even started playing around with Haskell.

I wasn’t ready.

This year I’ve decided to give FP another crack and have even bought myself a book - “Functional Programming in Scala” (this is an excellent intro to FP and although the examples are in Scala it focuses on teaching core FP principles) and took a Coursera course, of the same title, which turned out to be a great accompaniment to the book. So that’s how you know I’m serious.

I wanted to learn FP for a few reasons - firstly: I like learning random new things, secondly: everyone is banging on about it and thirdly: I wanted to see if it could improve my C++.

The Good…

Here are some of my favourite aspects of FP:

  • Everything is immutable and pure (well almost otherwise it would useless).
  • State is explicitly modeled.
  • Almost all programs can be modeled using a selection of higher order functions such as map, fold, etc.
  • Functions are not understudies to classes and can be composed, chained, passed around - brilliant.
  • Generic programming is simple and achievable with little boiler plate code.
  • Pattern matching.
  • Lazy evaluation: No boilerplate for caching the results of expressions! Don’t pay for what you don’t use!

One of the most impressive aspects of Scala is that you can build almost every language construct from first principles - you can create your own booleans and ints and even create your own if, then, else statements. Completely useless but really shows the power of the language.

The Bad…

Things I didn’t like so much:

  • Apparently FP programmers like single letter parameter names (because everything is so generic and based on maths notation). I couldn’t bring myself to do this.
  • And…that’s pretty much it.

In truth because I was using Scala (which is a hybrid OOP/FP language) it was much easier to incorporate state at the edges of my program than it would have been using a pure FP language like Haskell (and state is pretty important - put it this way I’m glad my bank is interested in the state of my account).

One of the issues I had with Scala was trying to determine when to use FP and when to use OOP. Usually I ended up with case classes acting as simple structs holding a collection of data. My big dirty secret is that sometimes I would even use an imperative for loop (what’s the harm in a wee ++i) in place of for comprehension on a range.

And the C++

Unfortunately there is little scope for me using Scala in my day job so I had to salvage what I could and bring that across to C++ - this wasn’t as much as I’d hoped. I’ve always thought of C++ as multi-paradigm (and it is) but even C++11 struggles to do proper FP - you have to pick and choose your battles.

One of the cornerstones of FP is immutability and Scala has a whole bunch of immutable data structures. Updating the contents of a data structure produces a copy of it, and because this happens all the time in Scala, and because the structures are immutable; sometimes a copy isn’t even made. Seriously, if you want a substring of a longer string you basically get a reference to part of the original string - it’s never going to change. The compiler is set up to perform these kind of optimisations - C++ isn’t. If we made a copy of vector, array or map every time we updated an element then we’d be in trouble (imagine creating a new texture every time you updated a texel!).

C++11 has introduced anonymous lambda functions which really help with the functional style and several higher order functions such as std::transform(i.e. map), std::accumulate(i.e. reduce), etc. but could really do with some more convenience functions for, among other things, composing functions i.e.

def compose[A, B, C](f : B => C, g : A => B) : A => C = { a => f(g(a)) }

Also (and I’m by no means the first to mention this) C++ is in desperate need of a D-lang style “pure” keyword that prevents any non-local state from being modified.

That being said there’s always learning to be had:

  • Stop making unnecessary setters! I do it all the time - “Oh here is a getter without its partner. Must be lonely. Stick a wee setter in there. Job done. Now how do I make this thread safe…?”.
  • Sometimes a copy is good. Although it’s usually too expensive to copy a data structure on every modification perhaps it makes sense to batch modifications together and then make a single copy updated with all changes. This pairs up nicely with the data oriented approach of “where there’s one, there’s many” and actually FP and DOD make good bedfellows.
  • Higher order functions are useful. I should make more use of std::algorithm to make my code more generic and less verbose (although the lambda syntax in C++ is still a little verbose compared to Scala i.e data.map(x => x + 2) vs std::transform(data.begin(), data.end(), newData.begin(), [](auto x){return x + 2;});)
  • Model state explicitly. It’s much easier to reason about data and state (not to mention multi-thread it) if it is visible and explicit and covered in warnings and alarms rather than encapsulated in an object that you have no idea is thread safe or not. Concurrency isn’t that hard is access to state is carefully managed.
  • Void functions are (usually) stateful functions. This is so blindingly obvious but I had never really considered it to be the case. Reducing the number of void methods or functions will reduce the amount of hidden state and side-effects in your application.

Heading over to the dark side

I’d really like to take on a project using a FP language and hopefully I’ll get a chance in the course of my work to experiment more with FP. If anyone else is interested in learning FP I’d really recommend approaching it without any OO prejudices and learning it from the ground up. Embrace its strange syntax, higher order functions and immutability - don’t fight against it and try and enforce an imperative style. Accept that creating and managing state should be a chore - this will help you mimimise it. Don’t be put off because it is different. Different isn’t bad it’s just different.

I’ve already started learning about functional reactive programming (again via Coursera) so there is a good chance my next blog post will be about that.

P.S.

I’ve also attached my first effort at a functional program and my revised effort based on the huge lambasting I got on “CodeReview” (cheers guys).

Getting My Hands Dirty with Open-Source

It’s going to be an interesting year at Tag Games as we are planning on open-sourcing some of our in-house tech (hopefully I’ll be able to go into more detail in the next few months). I’m both really excited, and slightly apprehensive, at the thought; as a great deal of the code and the architecture is mine and this will be the first time that I’ve had any code of real worth available for external criticism (or external love).

So far the process has made me look at my code in a completely different way, re-evaluating every design decision I’ve ever made in case I have to justify it to some guru hacker. It’s also really strange thinking of the code as the product rather than the means. The best thing so far is being able to learn Git on company time!

As part of open-sourcing we will be looking to actively engage with a good core community that will help drive the direction of the technology as well as make use of it. The community aspect is what concerns me most as it is vital and difficult to encourage and sustain a vibrant and contributing body of developers. However it shames me to admit that I have never contributed to an open-source project and expecting developers to contribute to mine is perhaps slightly hypocritical. So I have decided to do something about that and starting next week will be looking to regularly contribute to an open-source project (that gives me the weekend to actually decide which one). This will a) allow me to no longer be a hypocrite, b) give me something to actually blog about rather than the usual pish and c) provide a good opportunity to sabotage the opposition.

Hopefully this will give me a good insight into how an open-source community works and will help me develop a thick-skin as I suspect I’ll need it over the next year!