Rating Algorithm spinoff discussion

[Dragavan]

Let's move this out of the Top Writers Section thread since this really has nothing to do with it.

Usoki's idea for clipping people's votes who are outside the average isn't really the best way to go at all... and can be gamed easier than most other modes of dealing with vote gaming (which is the main reason to even have an algorithm in the first place).

Personally, I always liked places that have complex weighted algorithm to do theirs. The main thing is that they can't be an algorithm that they publicly tell anyone what it measure or how it works. The more details you give out about it, the easier it is for those who want to game it. I think the IMDB does a fairly good job with this. I like giving more weight to people who have been around longer, voted more often, voted a wider range of stories (both in quality and authors), and possibly other things (written stories, donated, or whatever you choose). How much weight each of these things is where you need to tweak things to get the right balance.

Yes this will tend to create an "elite" class of members, whose votes are worth more than others, but that's how these systems work. And anyone can make theirs worth more by being more involved in making the site better (reading and voting more over time). It's not going to be an uncrackable elite class, or a glass ceiling of any kind, just something that you have to actually be involved to become part of. You have to care. That way people who jump on, vote their friend's stories up, and leave, will never be worth as much as a vote as the person who reads lots of stories over the years and votes every one fairly. It will also weed out the people who only ever vote thing either 10 or 1, or always just vote stories they like 9 or 10, and make their ratings become slightly watered down.

But, the main thing that's really needed to make any good weighted algorithm work well is having a LOT of people voting... we don't really get those kinds of numbers on most of the stories. With the low numbers we get on voting, no method is really going to work well and be mostly free from gaming. If we go with a weighted system it's going to either greatly skew things because the low numbers are affected by highly weighted system, or it's not going to change anything because of a lightly weighted system. We just don't have the numbers to make it work really well.

[Usoki]

But, the main thing that's really needed to make any good weighted algorithm work well is having a LOT of people voting... we don't really get those kinds of numbers on most of the stories.

This is really what it comes down to, at the end of the day. I can sorta see End's frustration with weighted voting, in that it harms good people who haven't yet proven themselves to be good voters. And whatever stories you vote on as a weakling, you can't go back and revote with more weight, so there's a clear advantage to voting on crap until you have enough weight to be impressive. But, really... to me, we just aren't big enough of a site for it to matter.

It makes sense that big sites with lots of people would have impressive and complex voting algorithms. And it makes sense that they wouldn't tell you how they work, so that no one tries to abuse them. But we're not a big site. The main extent of voter cheating is pretty much limited to two cases: people who make their friends rate their stories 10s, and people who get mad at a person, and rate the story a 1 in anger. As it stands now, there IS no way to really fix the first case, and the second case can only be fixed by deleting ALL of the existing votes.

Would my suggestion work for large sites? Not even remotely. But it would work for us. It would help fight the first case, and it would pretty much eliminate the second case. I, for one, would love to get rid of a random '1' vote without having to clear out all of my ratings. It'd be different if there were a stated reason- but there's no comment whatsoever. And then there's all of the 1 ratings that MRH has lashed out with over the years. I guess this method could be easily abused, since it's so simple... but who would take the time and effort to abuse it?

Ehh, I dunno. This conversation seems to come up every few years, and we always bicker about the same few ideas. I think the other thing which gets mentioned is some sort of moderator who can go through and delete ratings. And then people cry censorship and cheating and whatnot, to which we reply that of course whoever has this power would need to be impartial, and... yadda yadda yadda.