Retrospective: Devlog Day 4

This post is the fourth in a series recounting the week of development for Brackeys Game Jam 2022.1, based on the Devlog notes I’ve kept throughout the week! For some context on how I came up with the idea for the game and some early sketches, begin by reading the first entry!

Thursday morning I was itching to see whether there was any feedback that got added overnight. Like before, I had a whole list of things to work on at the top of the morning. By far the biggest question was: How are people playing my game?

It is worth mentioning at this point that, although I was not yet aware in the morning, during this day my productivity was significantly affected by the news of Russia invading Ukraine. I processed my grief and empathy and approached the situation with stoicism, evaluating the things that were within my control. Plus, because we currently live in Romania, so close to the conflicts, it was difficult to cultivate the motivation and enthusiasm to build a game.

Barely crickets

Assuming that most of community is from the west, and given that Discord is mostly buzzing while it’s night time where I live, there would have been a bunch of people playing the game. In reality, most of the community was hard at work making their own games and, in all honesty, I haven’t made enough connections yet to have enough people interested in what I have built.

All I had to look at were the default Analytics on itch.io, and they were surprising! Here are the stats as of 8pm EET on the 24th.

| Date       | Views | Plays | Referrers |
| ---------- | ----- | ----- | --------- |
| 2022-02-23 |    15 |    12 |         5 |
| 2022-02-24 |    44 |    38 |        44 |

This mostly leaves me guessing, but here’s what could be inferred:

• on the 23rd, 10 people came from a directly link, safe to assume it was Discord, I would assume between 7–10 also played the game this way
• on the 24th everyone was referred by itch.io and 6 of them viewed the page and didn’t even play

This doesn’t tell me enough about whether people get stuck, how many levels they played, etc. It does seem a bit odd that some people clicked through to see the game’s page and haven’t played, and maybe that’s a signal to make the game’s page more engaging.

Higher resolution data

To get the sort of information that I wanted, I needed to log a few pieces of data from within the game, such as which levels are completed, whether people prefer the shortest path, or whether they always get the coin, and more specific level information later on.

While working on Camden Election Fallout I tried Google Analytics on for size and was very disappointed with the type of reporting done there. The product at its base is mostly focused on marketing and revenue and offers little in the way of understanding anything else. (There might still be some hope with Firebase Analytics, though that might be an exercise for a different time.)

I had also briefly looked into Game Analytics, which promises a free tier and touts itself as being made specifically for games, but in reality, after diving a bit deeper, it seemed to be nothing else than clone of Google Analytics, with more insights about revenue and click through rates than how a game would actually be used by the players.

So I had decided to go with a more bare-bones approach, using the free tier of Cloud Firestore, a Firebase service I haven’t actually used before, but is essentially a JSON document store that I could download and play with as a dataset later on. Plus it seems to offer a setup for Unity as well, so it might be a good option game development in the future as well!

👉 Are you interested about the details of the Cloud Firestore setup? Let me know in the comments and I’ll write a separate devlog to go into depth!

At first I was thinking of building a high-resolution series of events, to really understand how long people take to solve each maze, but then I looked at the amount available in the free tier of the service and decided to be more reserved. I have settled on the following schema:

{ pid, rid, levelId, inventorySummary, actionSummary, timestamp }

• pid — a pseudo-random player identifier, completely anonymous
• rid — a pseudo-random run identifier; a “run” is defined as starting the game from the title screen and playing until the game is finished or the page is closed
• levelId — the internal ID of the level, LEVEL0, LEVEL1, …
• inventorySummary — a codified list of items in the player’s inventory
• actionSummary — a list of the last option in each choice
• timestamp — a way to sort all of these events for a replay of the run

This took a little over half a day to learn and code, but the payoff wouldn’t come until the next day, when I’d actually get to look at player data!

The stats!

By 8pm the next day I already got a bunch more information about how people were playing the game. Let’s ask some questions on the data!

What is the highest level each player has completed across all runs?

• LEVEL5: 23
• LEVEL4: 5 👈 This means that 5 players looked at LEVEL5 and might have not been able to solve it!
• LEVEL3: 1
• LEVEL2: 1
• LEVEL1: 1
• LEVEL0: 1

Most players (~72%) completed all levels, which is encouraging. I’m worried about those who drop off before completing level 4, however. Maybe they were disappointed to find such an unpolished game. It’s till difficult to tell.

During each round where LEVEL5 was completed, how many coins did the player have?

• 22 players had 2 coins
• 1 player had 0 coins

Well, I guess we’ve got a speedrunner on our hands! :-)

How’s it looking today?

Now that the ~~game~~ prototype has been out for a while, let’s look at the stats since launch!

What is the highest level each player has completed across all runs?

• LEVEL20: 8
• LEVEL18: 1
• LEVEL16: 1
• LEVEL15: 1
• LEVEL14: 1
• LEVEL11: 1
• LEVEL10: 4
• LEVEL7: 3
• LEVEL6: 3
• LEVEL4: 7
• LEVEL3: 4
• LEVEL2: 1

A total of 35 players have each completed a run since the jam submissions was closed, and ~23% of them went all the way up to level 20! We’ve got something there! Even the fact that about half of the player complete level 10 is encouraging enough data for me!

During each round where LEVEL5 was completed, how many coins did the player have?

• 15 players had 2 coins
• 6 players had 1 coin
  • 4 of these got it in LEVEL4 and missed it in LEVEL5
  • 2 of these only got it in LEVEL5
• 3 players had 0 coins

Already we can learn a lot about how players interact with the levels, and formulate new and better questions for future play testing, specifically around the decision to pick or not pick coins.

For example, when watching ScalphaMission play the game on his Twitch live stream, I noticed him being confused by the fact that he was stuck picking up the key, then noticed that you get to keep the key when restarting, but was too eager to open the lock and missed the key!

You can watch ScalphaMission play Intuition on Twitch, as well as other awesome games! Then play and rate NYX, his submission to the jam!

Come back again tomorrow to read about why I chose and stayed with the minimalist graphics style for the game, and what features and polish I was able to cram in before the deadline!

As always, if you’re enjoying this type of content, give a shout-out, please! “Like” button at the top, comment box below, and… maybe go ahead and play the game yourself! Cheers!