Further Hugo Thoughts

[drplokta]

I've been reflecting on my proposal for making the Hugos less gameable. I think the principle is correct, that the number of nominees should automatically increase in categories where an algorithm detects slate-like behaviour. I now favour reducing the number of nominations per nominator to four, and having between five and ten nominees per category. That would prevent any two slates from taking over a category, assuming the algorithm increased the number of nominees to ten for that category (unless they were very well-organised in getting a large number of nominators nominating different subsets of the slate, which would at least remove all deniability about what they were doing).

But I don't think my algorithm for detecting slates is optimal. The problem is that the numbers that stand out as a slate in categories like Best Novella and Best Short Story would look perfectly normal in Best Dramatic Presentation: Long Form or Best Professional Artist, because there are many fewer eligible candidates, and so the most popular ones inevitably get more nominations.

So I'm after a piece of data that's not in the posted summaries. How many items in total were nominated in each category in previous years, especially 2013 and 2014, going all the way down to items that received only a single nomination (although it would also be good to know the number that got five or more nominations)? Does anyone have access to that data, and are they able to share it?

Comments

[netmouse.livejournal.com]

I am not sure why you want to try to 'detect' a slate looking only at the nominations for a single category. All the statistics in the nominations are available. The program can compare nomination patterns between entire ballots and determine whether or not some are more identical than is statistically probable using similar algorithms to how they detect cheating on tests. The problem is comparable-- you want to detect whether someone is just copying someone else's answers or are using their own thoughts as input. Once the system has detected identical ballots it can then figure out what percentage of all nominating ballots in each category is on those identical ballots and increase the number of nominees accordingly.

[drplokta]

I don't believe there is any chance that the business meeting will pass a Hugo process that goes "...and now put all the nominating ballots through this complex algorithm designed to detect collusion...", and so I don't intend to propose any such thing. Just looking for identical ballots is too easy to work round -- tell everyone who is in on the collusion to leave off one item from the slate, chosen at random. I'm thus looking for a simple algorithm that can be expressed in at most two paragraphs of text that can detect slate-like behaviour. The nominating numbers for this year are obviously screwy to human observation, so it should be possible.

[jamesb.livejournal.com]

Have you come any closer to something you could propose, or be happy with?