My Disappointing Automation Experience


This entry is adapted from a post I wrote on the Unity forums expressing disappointment.  I wanted to write a high-level overview of something I did that worked really well, but it was not meant to be.  Continuous integration isn't all that exciting for someone who isn't a developer, but it does help me with a lot of housekeeping so I can focus on making a better game.  However, there were hurdles; for now I've shelved this effort and am resuming work on the actual game.  Read on for more.


This past week I've been trying to set up a continuous integration (CI) pipeline for Chromavaders. I'm currently using the Personal edition of Unity 2019.2.2, but will upgrade to Unity Plus if I need to. I want my pipeline to automatically do these three things, so I don't have to do them by hand:

  • To ensure that the tests pass when I make changes to the code
  • To ensure that my game builds on all the platforms I support
  • To deploy to all platforms and storefronts I plan to support

I looked into Unity Cloud Build, but unfortunately that's not an option for me, for these reasons:

Instead, I decided to try out the GitHub Actions beta for these reasons:

  • I already have a paid GitHub plan, so I get 3000 minutes included per month
  • It supports Windows, macOS, and Linux, so I can build for any supported Unity platform on the same pipeline
  • If I want to add some build step external to Unity (like importing files or generating code), I can do that at my leisure

GitHub Actions has been a great experience for me; it's doing everything I want it to and doesn't get in my way with technical limits. Seriously, give it a try...with a project unrelated to Unity. Here's why.

Unity recently published some improvements to their test framework, and has expressed a renewed interest in CI. These new efforts led me to believe that I could set something up that would at least fit my requirements, even if there were some warts along the way. I was mistaken.

Unity's licensing mechanism is 100% getting in the way of me building the CI pipeline that I want. Currently, a Unity license must be obtained manually, either in the editor or through the process described here and here. For manual use (i.e. when actually designing a game), this is fine; I had no issue installing and licensing Unity for my development machine.

The problem is that you still have to do this when installing Unity on a CI runner. Most CI services don't make promises about a runner's identifying details (network address, hostname, serial numbers, other OS identifiers). As a result, a Unity license for a CI runner could become useless at any time without warning. In fact, that's exactly what's been happening with my own pipeline. This undermines every effort the Unity developers make to offer a smooth automation experience. What's the point of automated builds, tests, or deployments if I have to manually license Unity every time I run my CI pipeline?

I don't have a problem with downloading and installing Unity on the CI runners, nor do I with actually invoking it on the command line. It's rougher around the edges than I'd prefer, but at least it fits my needs. All I want is to be able to use Unity for automated builds, test, and deployments in a CI runner without any manual intervention; the best way I can see that is to remove or totally automate the licensing process for CI pipelines.

Or, if Unity's legal team would get nervous, perhaps they could consider special CI-only builds of Unity that don't need to be licensed, but can only be run headless? Such a build couldn't be used for designing content, and thus would not be worth pirating.

I love Unity and it's generally a bliss to work with, but in this case I'm incredibly disappointed. I know the dev team can do better.


No new screenshots this week because I spent my time working on my attempt at a CI pipeline; it's external to the game, but still in its service.  You can always try out the new build I recently released and take screenshots of that.  Or, if you really want to see a picture, here's my cat Pepe.


Get Chromavaders

Comments

Log in with itch.io to leave a comment.

(+1)

I had a similar experience with Unity Cloud Build. While it can be great for getting started with a game. You usually have to use Unity Pro to do the build in a cloud somewhere like Travis CI then use something like Node.js to automate the distribution. It's a surprisingly complex and frustrating experience for complex projects.

The frustrating thing is that I almost have everything I need, but the engine goes out of its way to prevent that.  Just let me write shell scripts (or Powershell, if I must) and I'll take care of the rest...

(+1)

A great read, I love the way you describe your journeys. Even though it failed, it gave me great insights in CI with Unity builds. I also hadn't heard of Trimmer yet, what a great tool!

Thanks for taking your time to document this!

Thanks for reading (and playing)!  One of the Unity devs responded to my forum post.  If something comes of it, I'll post a follow-up.