🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Score voting system.

Started by
24 comments, last by BitMaster 9 years, 7 months ago

As an EU resident I have no clue what you are talking about.

- so, if your native language is English, then it is difficult for you to understand the English language with imperfect style, sorry.. blink.png

(c) 2000 by "vvv2".
Advertisement

One of the biggest things we wanted to award points to was the contributing of articles to the site.

I see no problem with that. I do however think the rating increase should be given at a reasonable point in time, for example when an article passes a first peer review hurdle. For example just yesterday I reported a spam bot which managed to gain dozens of rating points for posting pure spam articles over several weeks. That wasn't even someone trying to game the system, it was just spam.

But yeah, the numbers to accumulate infinitely but nobody has gotten close to overflowing an integer yet. And to be honest, the people with insanely high reps do a ton for the community.

Overflowing is not my problem. The linear accumulation is. The old Elo-like system allowed people to have outstanding rating without endlessly accumulating values out of normal reaches. Even linear accumulation, although a downgrade to the old system, would be acceptable if not for some of the really stupid things to get reputation (see below).

Sure you can kind of cheat your way through it by doing the little things that earn rep, but ever since we made the switch we saw a huge increase in participation in things like post upvoting/downvoting and article authorship.

That's a stupid argument. The old system only allowed you to have one vote per person. Allowing votes per post is indeed the only sensible contribution I'm granting the new system. Of course this system will have more voting participation, in the old system you voted once per person unless they did something incredibly stupid later on or became an upstanding member. As I said above, I would have no problem honoring things like article submission.

Rather than gut it I'd like to look for ways to build on it and improve it. I think those who dedicate a lot of time deserve more recognition than they get.

I don't see how my suggestions are gutting the system. Having participation is good, but having a grade-school-like system ("everyone gets points just for being there!") is attracting the wrong kind of participation. The useful and valuable members don't need to be bribed into participation. The ones you attract with points for everything are often the ones you don't want participating as much. It devalues rating for everyone. Does the upvote I got mean I have actually said something valuable or did someone just stumble over the post by accident on his daily upvote clickery?
Personally I tend to avoid voting because getting a point for every up-vote feels like cheating. Usually I try to find matching pairs of something valuable and something stupid.

Again, I did not list my points above because those are the rating modifiers I'm aware of. I know there more ways to modify the rating, I'm just objecting to these.

As an EU resident I have no clue what you are talking about.


- so, if your native language is English, then it is difficult for you to understand the English language with imperfect style, sorry.. blink.png


I'm not a native English speaker. Also, since you seem to be aware your choice of words is not ideal you might have taken this opportunity to actually explain better what you mean.


Overflowing is not my problem. The linear accumulation is. The old Elo-like system allowed people to have outstanding rating without endlessly accumulating values out of normal reaches. Even linear accumulation, although a downgrade to the old system, would be acceptable if not for some of the really stupid things to get reputation (see below).

Just what is the issue though with out of normal reaches though? I think the number of users with insanely high ratings is fairly small relative to the whole membership list. And there are trivial ways to get rep, but there should be a balancing act going on where more legit ways to gain rep outweigh the trivial stuff. I could eliminate the trivial ways, but the reason they were introduced was to provide alternate ways to improve your ranking outside of relying purely on others to upvote you.

Overflowing is not my problem. The linear accumulation is.

What's the issue with accumulating a maximum of 365 reputation per year though? And that presumes someone really logs in each and every day, which I guess very few people actually manage (sure, you could probably use a script for that, but let's stay realistic, nobody is seriously going to do that).

It will take someone about 11 years of logging in every day to reach a reputation comparable to yours. That's quite a bit of work for a number that doesn't mean an awful lot. It's not like your boss will pay you 200€ extra if your rating is 2000 or 3000, or like you're going to get hired somewhere because you have a rating of 2500 at some internet site.

Upvoting increases your rating by 1 as well, there's a limit of 5 a day of that though?

Maybe you shouldn't get a rating boost for upvoting someone's post at all? (And a rating hit for downvoting?).

EDIT: Well maybe you should still get a rating hit for downvoting to discourage downvote sprees. But as mentioned before people then uprate to cancel that out anyway.

And while the number in itself might not have much value, being one of the top-rated posters might be.

"Most people think, great God will come from the sky, take away everything, and make everybody feel high" - Bob Marley

Upvoting increases your rating by 1 as well, there's a limit of 5 a day of that though?

I'm pretty sure if there is such a constraint, it's broken. I stumbled over another rating whore today. The only effects on his rating for weeks have been log-ins and upvotes of all posts in two or three threads per day. Nevertheless his daily rating could change by at least ten points.

Maybe you shouldn't get a rating boost for upvoting someone's post at all? (And a rating hit for downvoting?).

EDIT: Well maybe you should still get a rating hit for downvoting to discourage downvote sprees. But as mentioned before people then uprate to cancel that out anyway.

I'd once again like to peddle the idea of the old Elo-like system. If you have to give rating modifications for up-votes (I still maintain there is no real need for that), then make it count as a win against a value very close to the baseline. You get a self-rating effect then but it will be vanishingly small the higher your rating is and you can no longer rise to ridiculous values.
I also think it would remove the requirement to pay yourself for a downvote. People who go on down-sprees are practically exclusively people on the extremely shallow ends of the rating system (often considerably below baseline). Their effect on anyone is unlikely to be high using the Elo-like system and any rating loss would be quick to be counterbalanced.

And while the number in itself might not have much value, being one of the top-rated posters might be.

The number should give an idea how trustworthy information in the post is. If I run across a post which contains information I do not have enough knowledge of to evaluate the correctness I at least have something. And if I check the the rating history (which is basically one click and a single glance) I have even more indication about the person in question: is it a slow and steady growth or is the person prone to volatility and misinformation?
Or at least in theory that's possible. The person I stumbled on today does have a "slow and steady growth" that looks comforting on first glance. Only I do remember their time on the forum and know not of a single useful post they made and several that would put me on my toes with anything they said.

The number should give an idea how trustworthy information in the post is.

[...]
Or at least in theory that's possible.

That's what the advocates of voting systems assume, but it's not true. Only in theory, like you said.

You only need to look at another popular developer site which is all about voting up and down. You'll see not few 3-line questions that are quite low quality and are tagged with a popular tag like "optimization" that have 30k views and scores of +15 or +20 or even higher and then you say to yourself: This question is valuable and shows research effort? What the fuck?

Then you find questions that do show research effort and are well-worded, and they have 500 views and a score of 4 or 5, simply because they don't have an "interesting" catch-the-crowd tag. So although they have a much higher vote-per-view rate, they end up scored "worse".

Or take the matter of "downvote sprees" and "counter votes" which is also prevalent on this forum. The purpose of a vote is not to counter someone else's opinion because you don't agree with that person, or to harrass someone, but to express whether you find a post valuable in the first place.

But people (some, at least) simply don't care what's intended. That is why vote systems don't work well, in practice.

How often have you been disappointed by a product you bought at Amazon although it has 5-star reviews? And how often do you see 1-star ratings on products that are perfectly OK, only because the person giving the rating is too darn stupid to use the item correctly?

I recently saw a 1-star rating on single-use pipettes which are not very durable. Well, you genious, they're single-use. They're not meant to be durable, you use them once.

You only need to look at another popular developer site which is all about voting up and down. You'll see not few 3-line questions that are quite low quality and are tagged with a popular tag like "optimization" that have 30k views and scores of +15 or +20 or even higher and then you say to yourself: This question is valuable and shows research effort? What the fuck?
Then you find questions that do show research effort and are well-worded, and they have 500 views and a score of 4 or 5, simply because they don't have an "interesting" catch-the-crowd tag. So although they have a much higher vote-per-view rate, they end up scored "worse".

Sometimes easy answers (like in For Beginners) appear to be getting unjustified high ratings but I'd say the current policy of giving yourself rating for voting something like that up is partly to blame for that.
Besides, I don't see the point of the rating system as a way to put the 'usefulness' of every member down to an exact number. The point is largely to allow a separation of people who participate in a constructive and civil manner and those who don't.

Or take the matter of "downvote sprees" and "counter votes" which is also prevalent on this forum. The purpose of a vote is not to counter someone else's opinion because you don't agree with that person, or to harrass someone, but to express whether you find a post valuable in the first place.
But people (some, at least) simply don't care what's intended. That is why vote systems don't work well, in practice.

I cannot say I have noticed many downvote spress and when I did it was practically always either new members (suggesting it's a duplicate account and would have to be subject to moderator action anyway) or people very low in the ratings to begin with (as I said before: an Elo-like system would filter those votes out mostly).

How often have you been disappointed by a product you bought at Amazon although it has 5-star reviews? And how often do you see 1-star ratings on products that are perfectly OK, only because the person giving the rating is too darn stupid to use the item correctly?
I recently saw a 1-star rating on single-use pipettes which are not very durable. Well, you genious, they're single-use. They're not meant to be durable, you use them once.

I never buy Amazon products by review stars. When I use reviews there at all I have a look at the one-star and two-star reviews. Then I mentally remove all of those which are written in metaphorical crayons or people who obviously don't know what the hell they are talking about there or is completely irrelevant. If the remainder contains significant issues, these can be quickly double-checked using other sources.

This topic is closed to new replies.

Advertisement