Abusive Ratings

[march5th00]

At Chooseyourstory.com we have a few things in place to take care of abusive ratings. First off, the higher your "experience points" (points earned for writing great stories / contributing to the site) the higher your rating is weighted (up to a certain point - it's never very high). Members automatically are weighted higher than non-members or anonymous ratings. Also, any rating outside of the standard deviation is automatically discarded. The system seems to work. We'd be happy to work with any of you guys on this if you'd like.

[donteatpoop]

I say any rating can be submitted for review, high or low. There are those Gary whatever stories that got tens, and no justification of the rating (such as Cab) only serves to help in the rating being deleted.

So here's what I think we have so far:

Abnormal Ratings are submitted to the council (likely by council members).

Council discusses the rating and what merit it may have, comments are taken into consideration. (discussion may take a few days, possibly a week so that everyone in the discussion has at least read the story and rating in question)

If the decision is to delete the comment/rating, then it is deleted.



I also propose an option for people who rate and wish to change their rating.


All of this, of course, only if severenz is cool with it. Perhaps it can be done with the site revamp?

[Dragavan]

I agree with most of what End said, but I want to comment on this part.

Cab isn’t a “vocal” member but he’s been here a long time and written a few stories. He went and rated Drag’s From Darkness Story a 5. Which is pretty low considering that it’s a solidly written story. However perhaps Cab found it boring, dunno, he never gave a reason. But in any case that’s his right.

This is the one thing about the current rating system that bugs the hell out of me. The lack of comments as to why AND the inability to ask (as they often have their messaging blocked). If I get a lower than normal rating (or even a higher one) I really want to know why so I can learn from it. I don't care so much that I got a 5, I care that I don't know why.

I am not really strongly for a complicated weighting system, but a simple one that gives a slight edge to votes from people who are writers of stories on the site (at least two large enough to show up) and are regular voters (having given a large number of them to a variety of authors), weeding out extremist voters (those who mostly only give 1s or 10s). I don't think the weight should be that heavy, just a slight edge in the way it ranks stories. I don't think activity level in the forums or recently logging into the site should have anything to do with it. I also don't think that ANY of us should know the actual criteria and algorithms of the system, since certain people who like to crunch numbers could use and abuse it. But I would really be just fine without any kind of weighting system.

As for putting a "report abuse" button on here, it would just complicate things and cause all sorts of extra debate, work, and stress. If we had an active site with a staff (and possibly money), I could see this being useful, but not for a one man show like this. Sev doesn't need more crap to deal with. Even if there was a "council" doing most of the debate and work, sev would still have to be more involved to deal with complaints, dispute, and other problems that grew out of the council and removals than he is now. I don't think that's really fair.

I also propose an option for people who rate and wish to change their rating.

This is something I really wish we could do. Most of the stories here are unfinished and still getting ratings. What if they change and improve a mediocre story? It still has a 5 from me and I can't change that. What if they totally screw up a great starter for a story later in it and make it turn into crap by the end? It still has a 9 from me and I can't change that. What if somebody (not mentioning any names here) goes on a bender and jokingly starts rating good stories with 1s, totally screwing up their position in the ratings, but later sobers up and realizes what they did? The story still is stuck with that 1 and no matter how many duplicate accounts they make to rate it with 10s, getting themselves banned (again) in the process, will actually repair the damage. The simple ability to change your own rating of a story would take care of all of this.

[Locke]

I'm not sure if site membership is currently required to rate. That's not something I would want (insofar as ratings matter to me I'd be against it), but some may want to consider it.

The ability to change ratings would be a nice feature. As to any weighting system, in principle I agree with End and Nappi: let anyone who wants to rate have a go at it. Site membership and participation do not necessarily make one more qualified to rate (though they may add bias one way or the other). Whoever happens to be reading is a member of the audience the author is trying to reach, and his perspective, even if very different or limited, is valid. Especially for pure statistics, I'd personally rather know exactly how well a story is being recieved by my readership (especially if I'm trying to cater to that readership) in general than have weight given to relative friends, skewing the results.

I don't care much about ratings one way or the other (save that the proposed changes set a slightly bad precedent by introducing restrictions) so whatever's decided is fine with me. As there is some concern over abuse, though (and there will always be abuse, if a relatively small amount) I propose an entry in the comment field where a user rates a story be required before the system accepts a rating, and the required length of the comment set at least a few lines. True, this will will result in a slight drop in overall rating of stories from those who won't take the time, and now and then a determined spammer/flamer will just post a longer nonsense comment, but by and large it will improve the feedback an author recieves, and even in the case of apparently extreme ratings the author will know the reason why.

That to me is the point of a rating system, after all, to determine how well one's story is being recieved - not necessarily to have the most and the best (even if abuse occurs and someone rates with the comment "furries" copy/pasted a hundred times you can disregard it, compute the averages yourself and concentrate on the ratings that actually tell you what someone thinks of the writing).

To augment the longer and hopefully more helpful comments, break down the rating into seperate categories instead of a single nondescriptive 1-10 (examples: "spelling/grammar," "overall quality," and of course "I liked it/I didn't like it" as the reader's opinion/enjoyment of the story is something we want to measure in addition to its literary merits. You can probably think of better categories, but these illustrate my purpose). The categories would be averaged for the purpose of the overall story rating, but remain individually visible in the comments section. This provides more and better feedback to the author as well as further spotlighting potential extreme or abusive ratings.

To counterbalance the slight possible drop in ratings on the site because of the somewhat more in-depth comment system, a comments box could be placed after every "the end" or a room of the author's choice where a reader could rate right there on the page. All this could someday evolve into a private statistics page only the author can view with a breakdown of ratings by room, site and non-site users, etc, but especially for the near future that's overkill, prohibitively labor-intensive and not likely.

I'm just throwing out ideas. My proposed solutions (if you decide abusive ratings are a real problem and merit a solution) vary in complexity (though the best and most helpful is in my opinion the simplest: the mandatory and longer comments) but none really require an administrator's touch after implementation and most importantly none are restrictive or give "weight" to some raters over others. If you do make changes to the system, I would prefer they stay true to those tenets. Make sure your motives for any proposed change are better feedback for the author (and to a small extent ease of navigation for the reader as some do browse by ratings) and not "I want to have the most/best." Whichever system is in place, your writing should take care of that aspect anyway.

[Nappi]

HOLY MOSES!!!Dragavan has got some serious competition in the long-winded posts department these days..heheh