I've alluded to my experience playtesting Chromavaders in the past, but I figured this week month <ins>now</ins> would be a good time to actually tell you more about it, for two main reasons:

• The NYU Game Center just finished moving to a new building, and the semester there started recently.
• A new group called the EGD Collective just launched, and it just held an inaugural event featuring game demos.

Both of these are important milestones for their respective groups, and good opportunities for me to figure out how to make the game better.

One Less Thing to Worry About

I've previously written about testing Chromavaders.  However, the linked post primarily describes my process for testing the technical parts (the code) of the game, as opposed to the design parts (the rules and difficulty).  Automated tests do not come at the expense of playtesting.  In fact, quite the opposite; by having a test suite to warn me when something breaks, my playtesting sessions are easier and more informative.

There are two main reasons:

Productivity

By minimizing the possibility of unexpected bugs or regressions (i.e. things that once worked but broke), I can spend less time apologizing for bugs and more time discussing the tester's feedback and opinions.  This also puts me more at ease, as I don't have to worry that some surprise bug will leave a bad impression on the player.

Don't forget, multiple people play Chromavaders at playtesting events.  A surprise bug could prevent me from getting useful feedback and make the entire night a waste.  And if I'm not around to explain the bug (like if people are playing online after seeing my game in a stream), a bug might be perceived as a half-baked feature or a design flaw.

Flexibility

The whole point of these playtesting events is to get ideas that will help me make the game better.  However, there's no guarantee that this ideas will actually do that; while events I go to tend to have a lot of developers or gamers (i.e. people with stronger and more nuanced opinions about game design), they're just as fallible as I am.  So I have to experiment with their suggestions and come back next week.  Of course, when I do so, the game should still work.  Hence the tests; automated tests give me guard rails to prevent me from breaking things that I don't plan on changing.

How to Test the Fun

So what happens when I actually go to a playtesting event?  The specifics vary based on the event, but it's usually some variation of this:

Preparing the Hardware

Build my game and make sure it runs on my playtest laptop.  I don't playtest on the same machine I develop on; the playtest laptop is cheaper, to mitigate risk.  If my dev machine is lost, broken, or stolen, all work has to stop until I can get a new one.  But the loss of my playtest laptop would be more of a nuisance than a tragedy.  Plus, ensuring that my game runs on a weaker computer helps me determine how much optimization I need to do.  (So far, Chromavaders has run on my playtesting laptop without a hitch.)

Arrive Early

Arrive at the event early, so I have a lot of spots to set up shop.  I want to pick a location that'll draw as much traffic as possible.  If there's a TV available, I always claim it.

Wait

Usually I get people lining up within a few minutes.  If people ask me about what I'm showing before I finish setting up, that often means I'm in for a good night.

Observe

Watch intently and take notes about how they play, or about questions they ask.  Or about bugs; those usually happen, too.  The nature of my game is a pick-up-and-play, so I try to spend as little time as possible explaining the game; if I need to do that too much, it means I need to clarify explanations in-game.

This would be different if I were creating a deep, progress-heavy game like an RPG or an adventure.  In fact, the playtesting process as a whole would be very different -- that's part of the reason such games are difficult to make.

Talk and Listen

When they're finished, ask about their experience.  This is a freeform conversation, not a formal interview, so I generally just open with "what do you think?"  If a playtester needs some prodding, I generally ask specifically about what they liked or didn't like.

I don't take every item of feedback literally, however.  If I did that, I'd risk spinning my wheels on inane changes that don't wind up adding value to Chromavaders.  So I look for patterns; naturally, if multiple people are saying the same specific thing, I should take it seriously.

But sometimes the underlying pattern is less clear.  People might all be saying different things that all happen to point in the same direction.  Or they might be highlighting different problems that can each be solved with one common solution.  I have a new build of my game coming out very soon, and will give you specific examples then.

If a player isn't able or willing to give detailed feedback, they might not be interested in the game.  But this is less of an issue at playtesting events, where everyone there wants to show off or try out weird games in development.  At conventions that attract a wide variety of players with different expectations, some of them are just not going to care enough about Chromavaders or its genre to have any interesting thoughts about it.  I don't take that personally; you can't please everyone.

Play Other People's Games

Everything that I'm doing is built atop a culture of people willing to help each other out, despite being potential competitors.  That comes in the form of advice, tools, open-source libraries, technical explanations, or feedback on one another's games.  At the kinds of events I go to, many of the people who playtest my game are themselves aspiring developers.  It would be massively hypocritical of me to not show interest in other people's.*

Making time is trickier than it sounds, because my priority is still on Chromavaders.  I always have to be there to take notes and watch people play, and talk to them about it.  That is why I show up in the first place, and I can't lose sight of that.  Most of the events I go to are recurring, therefore if I can't get around to a game odds are pretty good I'll be able to try again next week.

It's a lot easier to do this if I have help; when I was at Play NYC, my brother helped me man the kiosk so I could go around and playtest or otherwise give feedback on other people's games.  Naturally, we swapped roles so he could do the same (or in case one of us wanted to get lunch).

*For this reason, I'd also like to split off parts of Chromavaders into a bunch of smaller libraries and distribute those freely.  But not until Chromavaders is close to being released, as I want such code to be battle-tested first.  When the time comes I'll be sure to let you know.