4 Myths about Playtesting:

I am fanatic about testing my games, and for the past 6 months, I’ve run 12 – 25 tests in per month (not including playing solo or with co-designers on projects). My background is in design and design research and I run user studies for a major retailer. User experience design all about analyzing behaviors and using data from studies to drive change, and techniques from UX research have been incredibly valuable in my playtesting and design process for tabletop games as well. I wanted to start my posts on board game design with a series on how to improve the quality and quantity of your playtests.



Myth #1:

“Start testing with your family and friends, then strangers.”

You see this often listed as a first step in posts about playtesting, and there are many successful designers who got designs published starting this way. But you’ll notice, almost every time you see the advice to start testing with friends, all of them follow up with: “but then, make sure you get your game tested by people who don’t know you.” I think most people reach for their family and friends to test because that is what is easy/familiar, not because it’s necessarily the fastest or best way to get a game tested to completion. My view is you should jump straight to testing with strangers. I’m not saying NEVER show your game to to family and friends, just that their feedback isn’t as useful.

  1. Your family and friends are unlikely to be honest with you.
    You’ve probably heard this before, but no matter how many times you tell them that you want honest feedback, they will be biased and try to avoid hurting your feelings.
    1b: “My friends aren’t like that, we’re all mean to each-other.”
    It doesn’t matter. Everyone has subconscious biases, and people that know you will have a harder time separating a critique of the game from a critique of you. Just because your friends are more honest than most (or you perceive them to be) doesn’t mean that their feedback is as unbiased as a stranger’s.
  2. Your social and family circle is too closed.
    Even if everyone is different, the fact that you are friends and family means you likely: come from similar backgrounds, are more likely to share political and social views, are likely to have had similar levels of education, are likely to have the same primary language, and many more hidden factors. In short: even if you have a diverse group of friends and family, you think more alike and have more in common than you and a stranger. Your friends and family are less likely to catch things during testing that you haven’t thought of.
  3. Testing isn’t a central part of your relationship, so you are changing the nature of your social dynamic.
    Let’s say I loved singer-songwriter music, and I have a friend who’s a really good musician. I’ll go to her shows. and if she invites me over to listen to a preview of her new album and give her feedback, I’d do it. But if EVERY time I came over, she wanted to play her album, I’d start to get annoyed.

Myth #2

“You shouldn’t make your prototype look nice”

This is very common design advice in the board game community, but is only true to a point. Spending a small amount of extra time (for me, this is typically ~2hrs per iteration, maybe a little less) to make your prototype look better is worth it. Now I’m not saying go out and commission artwork, or spend an extra 100 hours designing the perfect custom font for your card titles, but it is worth the time to go slightly beyond bare bones and make sure your prototype is clean and professional. Much of the job of game components is communicating information, and your prototype needs to do that as effectively as possible. Learning graphic design makes you able to communicate information more clearly. So do it.

  1. Having clear card, board, and other component layouts decreases teaching time and confusion during the game.
    How players feel about the experience of a game is connected tightly with it’s pacing and narrative arc, and stopping to parse or interpret information during gameplay messes with that pacing. If your players need to spend an extra 10 min per game reading your handwritten cards or parsing the confusing chart you’ve laid out on your game board, you’ve not only lengthened your testing time, but you have negatively impacted their play experience.
  2. Having a clean looking prototype attracts testers to your game.
    People want to pick up and play games that feel nice. My next post will be about recruiting play testers. This is way easier if you can show a nice looking prototype. It doesn’t need to look like a final product. It just needs to show clear effort and professionalism.
  3. The key is being able to do this quickly.
    You don’t want to get bogged down making your prototype pixel perfect, but it’s worth cleaning it up. Learning fundamental graphic design and layout principles is worth the time investment, because they are foundational skills you can apply across all of your prototypes, rulebooks, sell sheets, publisher pitches, email templates, etc.

Myth #3:

Playtesting is about making a series of small iterations.

This one is half right: because the design process is all about iteration. My issue is people often approach their game testing as though they are conducting a quantitative study, and each small change represents an experiment against a control of the previous version. I think it’s more helpful to approach playtesting using qualitative research techniques, using technique from usability studies or ethnographic research to base playtest structures on.

Realistically, you will never have enough tests to statistically finish your game using a “scientific” A/B testing method. It’s better to make lots of changes at each iteration during earlier parts of the design process to see how the core experience and feeling changes, then make smaller incremental changes toward the end of testing. 

The majority of the balancing of your game should likely be mathematical + a little bit of manual tweaking. If you just essentially use “trial and error” over and over, you don’t have enough real data to conclude your game is balanced.

That being said:


Myth #4

Your game needs to be balanced


There are many different definitions of balance but for simplicity’s sake, let’s say a game is balanced when players of equal skill stand a statistically similar chance of winning. If a game has multiple core strategies or paths to victory, balancing between those strategies means that each on of those paths stands a statistically similar chance of winning if executed well.

Your game doesn’t need to be balanced to be fun. In fact, the PERCEPTION of balance/fairness is more important than whether your game is actually balanced. Players will continue to have fun as long as they believe that they have a fair chance. For example: let’s say you have an asymmetrical game. You’ve extensively balanced the skills and abilities of the three factions so they all stand an equal chance of winning. However, the counterplay from Faction C that allows them to stand up to faction A is not intuitive, and requires a few playthroughs of the game to understand. If the first 4 times a player plays your game, faction C loses, they’ll conclude the game isn’t balanced and shelve it. Similar to this are games where there are prescribed opening moves. If people take said “standard” moves, then the endgame is balanced, but if anyone deviates, they essentially hand the victory to another player. Here, your balance is getting in the way of players having fun and meaningfully participating during the duration of the game.

Some designer/publishers don’t consider balance while learning the game a real issue, and focus their development only on expert players who can really plumb the depths of tactics. Ignacy Trzewiczek for example, tells a story in his book about an army for Neuroshima Hex that has absurd win-rates (Steel Police) until people learn to play against it, at which point he concludes that it is in fact balanced. That was expansion content, but again I think you have a problem if it requires weeks of playing constantly for players to achieve the level of knowledge to make the game balanced. In my view this represents a flawed design.

Final Thoughts

Playtesting is an incredibly important process. Hopefully, some of the lessons I’ve listed here help you in getting your games testing and getting the most out of those tests.

More coming posts in this series:
Recruiting Playtesters
How to run Playtests like a Researcher
Getting Better Playtest Feedback
Interpreting Playtest Feedback like a Researcher

Interested in how I run large scale tests? Join the remote testing group for Mission Control.

Interested in staying up to date on developments at Brieger Games? Sign up for our mailing list.

3 thoughts on “4 Myths about Playtesting:

  1. Great article.

    The line between ‘friends’ and ‘strangers’ is a tricky one – any playtest group will probably quickly become a group of acquaintances, if not friends. I think I agree though.

    Absolutely agree with point #2 – spend as much effort/time/money as you’re willing to throw away in a week’s time. Was this inspired by the recent FB thread?

    #3 – I’ve met only one person who did the kind of tiny changes you’re suggesting. I think it’s all down to personal style and am less confident about this point – I feel Dave C was getting frustrated by the extent of Andy Y’s changes for Towers. I think either approach could be correct. With subjectivity, ‘scientific’ research is basically impossible. ‘Balance’ at a high level of play is really the only thing that can be mathematically determined.

    #4 – I find this an odd story and I’d like to hear the context. Ignacy has a flair for a dramatic tale.

    The key question for ‘Steel Police’ would be, “who is this for?”

    If it took the pro-level players so many weeks to learn to play with it, then presumably the less enfranchised folk stand no chance. Of course, it takes a long time to exhaust the base content, and so presumably this is for the benefit of the core fans, rather than an expansion they sought to market to a wider audience?

    Another question to ask would be, “Is it fun when you’re losing as this team?”

Leave a Reply

Your email address will not be published. Required fields are marked *