After looking at whether there was a cultural gap, I next consider the difference between New and Experienced tracks. Here, I’m only going to look at those puzzles where there was a difference in content.
“First sale”, “Advertising”, “Viral marketing”, and (after compensating for the London difference), “Buyout” will be control puzzles. On these four identical puzzles, Experienced were significantly better players. The gap is least if New scores are increased by 4 or 5 points per puzzle.
Again, I think that a Student t-test is an appropriate method, comparing New to Experienced. My chosen data sets are the raw scores for teams completing each individual puzzle. Teams dropping out part-way through the day will contribute data for puzzles they solved.
“Collecting Ingredients II” involved solving cryptic clues. These cryptics were different between the tracks – New had a word in a sentence; Experienced had a partial anagram. Experienced needed to fully resolve the sudoku, New only needed the middle three letters, but I think that’s a marginal difference. The evidence suggests that Experienced teams performed worse than New, and the discrepancy goes away by allowing 2 points to Experienced. Given the advantage of Experienced teams on identical puzzles, we might add 6 to Experienced scores.
In “End of the First Day“, Experienced had a longer route to the answer, more letters to decrypt from ternary, and weren’t given the partial hint that there was decryption (but didn’t need to solve that partial hint). Many sides who got this puzzle got it very quickly (though I’ve discarded a team credited with the answer in 40 seconds), and it appears New sides were behind by 1 point. However, we might expect New to be further behind on equal puzzles, so add another 3 points to Experienced scores.
For “Practicing the Sales Pitch“, New had a much clearer hint that semaphore was involved, *and* a time limit more generous by 10 minutes. Experienced were clearly behind, by about 8 points. For our notional parity, try adding 12 to Experienced scores.
In “Mass Production“, there was a small difference – New were explicitly told there’s a word chain. Don’t think that’ll have altered the scores, and this proves to be the case – Experienced were up by 3 points. I can’t reject the hypothesis that there was no additional advantage to either track, so no change.
Finally, for “Memoirs“, New were told that these were rebuses, and given one of the least obvious initial answers, but penalised with a 10 minute shorter time limit. Experienced appears to have benefitted by about 9 points, so we might deduct 4 from their score.
Solely to compensate for differences in the puzzles, I suggest that New teams might deduct 17 points from their score and compare with Experienced sides. That would put the top New team, Colleen Werthmann, in a tie for 31st place; six New teams crack the overall top 100.