The semester report for late 2015

Tree graphics from the second half of the yearThis site looks at locations’ TripAdvisor performances from time to time; it’s interesting to reflect on how far the industry has come. The Timeline shows that the number of known open exit games in the UK doubled (or, on every occasion but one, more than doubled) over the first half of 2013, the second half of 2013, the first half of 2014, the second half of 2014 and the first half of 2015. (Those numbers: 1 to 3 to 7 to 14 to 30 to 64.) Over the second half of 2015, the number did not double again but rose from 64 to 95. Maybe it’s only 94. It’s in the very high double figures, at least.

It’s worth occasionally looking at trends in popular reviews of exit games in the UK and Ireland, taken from TripAdvisor statistics. This site is using a little more reserve than once it did with regards to what it says, bearing TripAdvisor’s terms of use in mind; the aim is not to laud or criticise particular sites in this regard, more to look at the bigger picture. Besides, if you run a site and care about your performance in this regard, it’s probably not difficult to work out which site is which from context. As usual, there’s more than a hint of truth in xkcd’s snark about online star ratings; in this world, anything other than full marks (and, especially if you’re on eBay, several pluses and stars) is a “diss that don’t miss”. It’s not necessarily a healthy state of affairs for anyone who cares about subtlety, graduation and shades of light and dark – but, with this in mind, are five-star ratings quite as common as they used to be?

Here’s some raw data, aggregated over the universe of TripAdvisor reviews for exit games in the UK and Ireland that this site was able to find.

Time period Number of reviews Number of 5* reviews Proportion of 5% reviews
To end of June 2014 1665 1532 92%
Second half of 2014 2240 1998 89%
First half of 2015 4248 3900 92%
Second half of 2015 6697 6127 91%

From these figures, there has been no measured change in quality over the last six months; a z-test does not suggest any even slightly meaningful degree of significance in the change from the first half of 2015 to the second. These statistics make the considerable (and untestable) assumption that the standard required for a 5* review is the same as it ever was.

It may be closer to comparing like with like to only consider the 52 sites that have been open since before January 2015, where we have meaningful numbers of reviews (10+, and even that’s a stretch) for H1 2015 and for H2 2015.

Site location First half of 2014 Second half of 2015
  Reviews 5% reviews Prop’n 5% Reviews 5% reviews Prop’n 5%
Scotland 66 52 79% 115 95 83%
South 79 70 89% 75 65 87%
Ireland 24 18 75% 20 16 80%
Midlands 11 1 9% 15 0 0%
Midlands 109 108 99% 113 107 95%
North 84 82 98% 69 64 93%
South 22 21 95% 23 15 65%
South 64 60 94% 92 88 96%
Scotland 203 167 82% 103 82 80%
Scotland 103 98 95% 98 96 98%
Scotland 119 113 95% 163 151 93%
Scotland 66 64 97% 148 142 96%
Scotland 122 109 89% 106 90 85%
Scotland 14 12 86% 35 30 86%
Scotland 41 41 100% 111 109 98%
North 36 35 97% 23 23 100%
Scotland 11 11 100% 48 45 94%
North 200 194 97% 176 163 93%
North 156 150 96% 93 86 92%
North 80 80 100% 95 94 99%
North 72 66 92% 104 97 93%
North 28 27 96% 23 19 83%
North 40 37 93% 147 132 90%
North 21 18 86% 89 77 87%
London 189 178 94% 177 163 92%
London 220 207 94% 235 213 91%
London 71 54 76% 62 47 76%
London 23 22 96% 23 23 100%
London 20 19 95% 35 33 94%
London 59 50 85% 115 97 84%
London 31 29 94% 128 114 89%
London 62 60 97% 181 178 98%
London 75 71 95% 202 191 95%
North 31 30 97% 32 32 100%
North 172 155 90% 211 184 87%
North 76 69 91% 258 216 84%
North 190 169 89% 254 195 77%
Midlands 13 13 100% 15 15 100%
North 41 37 90% 37 31 84%
North 65 62 95% 38 29 76%
North 41 41 100% 59 57 97%
Midlands 82 76 93% 148 137 93%
Midlands 46 46 100% 162 158 98%
Midlands 27 20 74% 46 38 83%
South 121 112 93% 104 91 88%
South 71 70 99% 74 74 100%
North 141 134 95% 185 161 87%
North 39 37 95% 34 33 97%
North 210 203 97% 198 190 96%
North 95 90 95% 197 186 94%
Ireland 36 36 100% 85 85 100%
Ireland 22 20 91% 53 48 91%

The first column is classified as Scotland, Ireland, London, and (referring to different parts of provincial England) North, Midlands and South. Be aware that Ireland refers to both Northern Ireland and Ireland as such; this is poor practice that the site does not usually follow, but the alternative would be to let at least one site be individually identifiable, which this site considers to be an even worse alternative. There is further ordering in the table which this site chooses not to make explicit but is not hard to deduce. (If you run a site and can’t work out which site you are, you could always ask.)

So, only among these 52 popular and well-established sites:

Time period Number of reviews Number of 5* reviews Proportion of 5% reviews
First half of 2015 4040 3744 93%
Second half of 2015 5432 4905 90%

That is a statistically highly significant difference over the whole of the population and some individual sites have similarly statistically significant differences as well. Does this mean that those sites have got worse over time? Not necessarily; it may just mean that people are holding well-established sites to an even higher standard still to the one that once they did. Run your own tests!

There’s one other comparison worth running:

Time period Number of reviews Number of 5* reviews Proportion of 5% reviews
52 well-established sites 5432 4905 90%
The other 40+ sites 1265 1222 97%

That too is a statistically highly significant difference. Does this mean that new sites are better than well-established sites? That could be one conclusion. Does this mean that people are holding these less well-established sites to a less exacting standard? That could be another. Certainly some new sites have got off to remarkably accomplished starts. This is excellent news and this site looks forward to seeing whether they can maintain their remarkably high ratings over time. Every site was a less well-established site before it became a well-established one.

There is one very important assumption that this analysis makes: that the reviews that people leave are a genuinely representative sample of participants. Different sites seem to perform more or less effectively at converting participants into reviewers and it is not clear why. Looking at the geographic locations of reviewers, it’s also sometimes possible that more than one member of the same team might choose to leave a review for some games, though there’s not necessarily anything wrong with that; it’s conceivable that different members of one group might leave – say – both a ***** review and a *** review, rather than the group leaving a single **** review. It’s not unknown, in the wider world at large, for there to be such things as fake reviews; this site isn’t aware of it having happened in the exit games it covers, but this site has chosen not to look too hard.

In conclusion: in aggregate, the statistics suggest that exit games were awesome up until the first half of 2015, and have been just as awesome in the second half of 2015 as well.

1 Comments

  1. Great (geeky!) article. So many things I could say.

    Better reviews for newer games: That doesn’t surprise me. People review bad experiences uniformly I think, but they’re more likely write a review for a good experience if there aren’t already lots of good reviews. Similarly, the companies involved are much more likely to push the Trip Advisor reviewing when they don’t have many.

    I do think that as escape rooms get established they’ll lose a bit of their gloss. Firstly they become worn – more damage to the venue generally and the rooms themselves. Secondly they hosts probably go downhill a bit. Often they recruit people to host who just don’t have the enthusiasm the owners do. That’s not a given – most owners probably aren’t naturally great hosts, so they could recruit better hosts, but I suspect that means spending more money.

    The slowdown (in terms of percentage growth) is a shame, but not a massive surprise. It’s still a similar numeric growth though, which is good, and in terms of numbers of games (which as existing players is probably the most important number) I think we’re getting continued good growth – those extra venues all bring new rooms, but the existing venues do too.

    Thanks for compiling!

    Reply

Leave a Comment.