Once again my friends and I are organising the annual CiSRA Puzzle Competition, which has just opened for team registration. Part of the process of running such a competition is of course creating the puzzles, which is a lot of fun.
Another part, for the sake of the competition format, is writing a series of 3 hints for each puzzle. The puzzles are worth points, and the point value decreases as more hints are released over time. We’ve actually spent many hours discussing (or arguing may be a more accurate term) how to approach the writing of hints.
A major problem with trying to run a competition like this and writing suitable hints is that we get feedback from participants criticising the structure of the hints. In particular, people strongly dislike it when they feel they have done most of the hard work involved in solving a puzzle, and are stuck at the final step, and we release the first hint, and it tells them about one of the initial steps of the puzzle – a step they already figured out with their own effort.
This is understandable. We’ve felt exactly the same frustration ourselves when participating in other puzzle competitions. When you’ve put in hard work and figured something out by yourself, the last thing you want to see is a hint that lets the other teams who haven’t figured it out yet catch up to where you are. What you really want is a hint that helps you. Ideally a hint that helps you and nobody else!
We could write hints so that the first hint helps people stuck at the final hurdle in the puzzle, and only later hints give away the earlier bits of the puzzle. But what does that mean? It means we’re helping the good teams. The teams full of strong puzzle solvers, who are already ahead of most of the pack, because they’ve probably already solved a bunch of other puzzles that other teams are still struggling with. And we’re not helping the weak teams – the ones who really need a kick start to even know what the first step in the puzzle is. If we did this, the effect would be to boost the strong teams and give nothing to the weaker teams. It would be making the already strong teams “win more”.
On the other hand, we can structure the hints so that the first hint helps people with the first step of the puzzle, the second hint helps with an intermediate step, and the final hint helps with the final step. What does this approach do? It helps the weaker teams to get a leg up on puzzles that they were really having difficulty just starting on. It doesn’t help them with the rest of the puzzle.
Now if we assume the stronger teams have figured out steps 1 and 2 by themselves, maybe 20 hours ago, and a weak team is now given a helping hand with step 1 by the first hint, while the first hint doesn’t help the stronger teams… do you know what? The stronger teams will still generally solve the puzzle first! They still have the advantage of being better puzzle solvers, and the extra advantage of 20 more hours to cogitate on the final step. Hints that help the weaker teams just level the field a little bit. They don’t help weak teams to beat strong teams.
And levelling the field a bit is useful. Imagine you’re on a team that is struggling to figure out how to even start a puzzle. A day later the first hint is revealed, and it’s a big hint for the final step of the puzzle – a step you’re not even up to yet! It helps the strong teams, not you! That’s discouraging – you’re falling even further behind. You may as well give up on this lousy competition.
So hints that step through the puzzle in order are useful to encourage teams to stay in the competition. They flatten the point spread a bit. And they don’t really hurt the best teams – the ones who are (understandably) a bit frustrated that the other teams are “catching up” thanks to hints. Because, I’ll say it again, the best teams are going to solve those hard puzzles before the weaker teams anyway. When the last hint comes out that gives the game away, the strong teams will still beat the weak teams who have been guided through the early steps by the first two hints.
So that’s the main principle we use when writing hints. We sometimes cop flak from strong teams who feel frustrated, but we’re okay with that. A second principle is that we also try to mitigate that a bit by making hints double-barrelled if we can. The first hint must provide a somewhat strongish clue about how to start the puzzle, but may also provide a more or less cryptic hint to getting past a later roadblock step. It should be slightly cryptic, to give us room to make it more obvious with a later hint.
I actually argued against doing this, but some of the other puzzle organisers were so adamant that we had to give something to the teams stuck on the last step that we adopted it as an option.
Anyway, the basic point of this post is that figuring out the philosophy of puzzle hint writing is difficult and full of contradictory pulls and opinions. It’s something we’ve actively spent time arguing over, to arrive at what I’d describe as a slightly uneasy truce, rather than total agreement. Further, this serves as an illustration that what many people may think of as some rather trivial consideration can be extremely complicated and involve deep questions of game-theoretical philosophy. If you’re serious about organising something like a puzzle contest, you’d better be prepared to think deeply about stuff like this.
Tags: puzzle
Obviously there’s a counterargument. :)
The first thing to mention is that, if the hint helps with the first part of a puzzle, and gets weaker teams past the first hurdle while leaving stronger teams stuck where they were before, then it hasn’t actually helped anyone solve the puzzle. It’s just gotten a few more teams stuck at the same later point. That can potentially make people wonder why we’re bothering to put out hints at all.
The second problem – which we’ve come across occasionally in other puzzle comps that will go unnamed :) – is when the first hint is so obvious that everyone who looked at the puzzle in the first day has worked it out already. That’s really annoying.
But, having argued^H^H^H^H^H^Hdiscussed it for a while now, I’m pretty happy with the current approach.
I’ve followed the CisRA puzzle competition for some time now. I do usually attempt at least the first few puzzles, but they typically are too difficult for me. Of course, they are designed for teams, rather than an individual. I too know the fustration of being given a hint that I already knew. On the flip side, there’s nothing like being given a hint and being unable to work the hint out, especially when the puzzle is in hurdle form. If this hint is supposed to help me solve stage one, and I can’t, are stage two and three going to be impossible?
Anyway, I do look forward to the puzzles again this year. At the very least, I enjoy marvelling at the solutions once released.
Hints could be made strategic, with stronger hints worth more than weaker hints. That is, stronger hints could diminish the value of the solution more than would weaker hints. A team could decide how much to “spend” on hints. (I’ll add that I am unfamiliar with the rules and format of the CisRA competition; I’m just tossing out a hypothetical idea here.)
I remember ‘way back in the olden times, when I was in grade-school. In some of the mathematics books, the problems to work out would sometimes have a completely useless (to me, anyway) hint, like “HINT: Use pi = 3.14”.
I am part of an intermediate team that is doing better this year – 10 out of 12 solved so far – because we have more free time to dedicate than previous years and a great new team member.
I commented to another team member on Tuesday morning that the hints were guaranteed to be frustrating – we were well advanced in one puzzle and could not get started on other; whatever the hint level, it wasn’t going to help with both. So, I was well aware of your impossible dilemma with choosing hints, even before I read this page.
(As it happens, neither hint helped us to progress at all, and we are forced to wait for the second round of hints due in a couple of hours.)
However, I do want to address your claim that better teams will still solve puzzles faster even if the hints are given easy-to-hard. That may be true, but they won’t be able to solve the puzzle until it has been reduced in value twice. They may be half-an-hour faster than the weaker teams, but they won’t be a day faster to get them the additional point that they “deserve” for making more progress than the weaker teams.
I don’t object to the current approach – I can see there is no ideal solution (except, of course, emailing me personally with hints suited for me, and not giving hints to anyone else!)
Finally, thank you, thank you, thank you for the puzzle competition! We are most appreciative of the work you have put in.