Discussion about new upcoming rating system

Doug,

I would LOVE a +/- system for media. There is a dutch website called dumpert.nl, and they use a "kudos" system there (similar to reddit's up down). Content that is good and received well by their community gathers positive points. Stuff that stinks, gets a -1. When sorted by the kudos, the top rated videos of the day are laid out right in front of you with a net gain of +1's to -1's. Very easy to see what is trending, what is going places, etc. This would be fantastic for our photo and video sections. Exactly like you said, no one person has the power to change a rating drastically. It will get the rating it deserves over time (be it in an hour, a day, a week...)

That being said,please please PLEASE do NOT do this to the forums.

I hate reddit, not for their members, not for their content, but for their layout. NS has THE best forum layouts in the game. I've been here a few years, and have been on countless other forums. Its layout, its simplicity, just everything about it works, and works well. Reddit is just miserable (for me at least), because logic tells me to read things in chronological order. They decided to let everyone as a group determine what I should read instead. Well, I don't want members to determine what I should see as the best replies, I want to decide that myself! And I do that by reading through, seeing the posts in the order they were replied, and rewarding the person that made a good comment with karma, or a nice reply to their post. Same for those that decide to comment in a negative fashion.

I can't read youtube comments for the same reason. You get some of the "member favorites" up top, then the latest replies, and the older stuff becomes archived. It's a fucking mess. TGR is a mess too with their VBulletin layout. There is too much information for each thread in their subforums, and each reply is huge. NS is great: A nice sig, a nice member identification on the side and signature, and clean, tidy subforum listings.

The karma system could be replaced with a thumbs up or down system, though. Again, no one person can greatly effect a member, and it shows true colors more easy than 1-10. I mean hell, I look at karma ratings the same way it would be in a point system. lots of positive (~6.5+ exchange rating) and they are probably a good contributor. Middle of the road (close to 0) (~4.5-6.5) and they contribute O.K. Lower or negative points (sub 4.5) and I take what they say with a grain of salt.

A complete overhaul of the forums would take a lot of time and man hours. Not to hate on what you guys have done, but there have been a few issues with what we have, that have gone quite a long time without answer or solution. I'd hate to see a large project be taken on, only to have more issues (what project doesn't?!) that take months and months to address.

The current forum system we have right now works really, really well. Why don't we address the small things to make it flawless and not worry about completely redoing every aspect of the site. Start with the Media sections, maybe karma system, and thats it. But for the stuff that ain't broke, don't fix it!

Jamie

PS: I love what you guys have done here and I mean no personal harm with what I said. I know you guys work your fucking asses off day in and day out to make this place run, and I applaud you!

this and i think this is a good idea

i think doing that to the media sections is a really good idea.

also, what jamie said is spot on. i hate the layout on every other forum, but NS's just looks right. you guys do a great job

upboats for everybody!

Finally!

agree with everyone else...

I've been browsing ns for almost a decade (took awhile to join) and the one thing that seems to have never changed much is the forum layout. I think there's a reason for that, and that reason is that it's absolutely perfect. Keep it that way.

Karma changes? Im in the top three, there's your proof of a broken system.

#7 reporting


yup, something's wrong

but seriously don't change the forums

First off, don't worry. This isn't coming to the forums yet. I'll address, but please can we keep the conversation on topic? This is an announcement about media first and foremost which needs this the most.

Forums

We have some dreamland ideas about forum ratings and such, but please fear not we no matter what, forever and always will have the classic forum layout available by default.

Even if we come up with the best solution for conversational ranking of all time, every single core user will be able to browse forums and comments in the exact form as they are today. We certainly have no intention of breaking something that works so phenomenally well, and even if we thought we were doing something better and you hated it, we'd revert any change we made.

We've done it before with 'latest added' video/photo pages, and we'll do it again. Please give us credit for how much we listen and react to your suggestions.

Media Rating Converstion

Yes, all previous ratings will completely change. We have done the math and come up with a fair way to convert all historical ratings, but your previous rating will be tossed into the ether never to be seen again. When we did our research, it turned out that there were just a few malicious voters ruining the system for everyone, so basically almost every rating was tainted. The ones that weren't were drastically higher than all the rest. The system was fucked.

The new system completely accounts for this, so what you see will be a much more fair ordering of media. Things will shift as more people can easily rate stuff up and down, but we're confident this is going to be a lot better.

Closing Comments

The key issue that we're trying to address is that Newschoolers is growing at an insanely exponential rate. With such a flood of new people, both the ratings systems and the chronological views have become increasingly difficult to keep resonating properly. Both have flaws... take B/S/T - everyone complains about constand 'bump' posts. There's a strife of issues everywhere which need to be addressed (and yes we do listen and actually have about the next 3 years worth of programming time mapped out in project management software) and we're addressing them.

Some issues are bigger than others, and some have drastically larger consequences to fixing than others. We are always looking both big picture, and doing our best to keep the core user of newschoolers at the front of our minds.

So far in the back end the media changes look good, so I'm pretty sure we're going to give the launch order pretty soon. I'm going to do a final audit with the staff, and we'll let you know where its going.

Stoked

yay, this should be good for ns

Launched.

https://www.newschoolers.com/readnews/5352.0/Changes-to-NS-Rating-System

Not feeling the new rating system. Actually liked the old one better than the like or dislike ones. Which this is. I guess it's good in one way because the immature kids can't completely rape a 10 star edit with a few 1 stars. Idk, maybe it'll grow on me



I'll give it a shot though

Looks good so far!

So i have a question about the "score" is that the amount of up votes? down votes? or total votes? Cause the #1 all time has like 277 and then the next one has like 400 something, so im guessing thats not a number that means anything other than the number of votes.


I like the new system alot. Being someone who produces alot of content for NS, im excited to see how it works. Ive had many videos that were like 9.8/10 that within like 5 votes would be like 9.1/10 because kids would just give it a 1 and completely fuck and chance of top rated. I know that happens to way more people than just me (which is why all time top rated use to be all the same videos, if you look now its pretty mixed up). It got really frustrating to see a video with 4000 views, 50 ratings at like 10 ten when i had a video with 20k view, 300 ratings and it wouldnt make all time top rated cause like 5-10 people rated it a 1. I think this new system will make it so top rated is actually top rated and not just a video that slips under the rader with a good rating (as your example showed in the news release).


Anyway, i think its sick doug and im stoked to see it in action.

The score that is displayed is simply your total up votes minus your down votes. This is to visually get a quick idea of what is worth watching in the 'latest added' views. Those views are important for getting ratings onto new media.

The ranking order in top rated is based on a different calculation, so that is why something with a higher displayed score might not have as high of a rank as something with a lower displayed score.

Dig?

Yeah for sure, i was just curious to what the "score" meant. Stoked to upload a video in a few weeks.

great job bishop. i like the new system way more. seems fairer to everyone involved

So basically the top rated section will encompass a mixture of the videos "score" and view count?

Hell yeah this is a sick change.

No, it is an equation where the rating order is based on:

ranking score = lower bound of Wilson Score confidence interval for a bernoulli parameter

http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

^^Read there for more. It essentially takes into account more of a hive mind philosophy, and assigns accuracy to smaller data sets.

Old school mathematicians basically had figured out this problem like a hundred years ago. We on the internetz just needed to listen.

Couldn't the like or dislike deal still give you stars or some kind of rating that could be measured against other videos? Even if it varied slightly from the data that ranked you?


Would be nice to be able to see something and no where it's ranked.

We actually eperimented with this.  The problem is that the actual rating that gets produced by the algorithm Doug mentioned is between 0 and 1.  Even if we multiply this resulting score by 10 to make it out of 10, the results don't really work well with a star rating.

For example, lets say a new picture has 2 up ratings and no down ratings, it would have a rank of about 0.2.  So if we multiply this by 10 and render the old star rating, you'd have 2/10 stars.  Members would immediately be confused about why it has 2 out of 10 stars from only positive ratings.  This display would give the impression that the picture isn't highly rated, when in fact it simply doesn't have enough data to be ranked very highly yet.

The bar display solves this by simply showing the ratio of Up to Down votes, and the Score gives you an idea of how many ratings make up that ratio.  We found this to be far less confusing and easier to understand at a glance. 

How they hell do you know dumpert.nl?

The new system is certainly better, but I think a 1-3 rating system would he best. We need something between the plus and minus, for things that are just average.

haha I met/know a kid from holland, he always links me to dumpert videos

There is something, which is not voting.

Look into the math behind this. All ratings systems on the internet are completely broken, this is a really solid way to take all the issues with ratings into account.

You either rate it up, rate it down or don't rate it. That is a three point scale.

TOO MUCH FACEBOOK!!!

except completely different

is this why there is currently no comment/pic/views karma? it seems most peoples karma has flatlined

New system looks dope.

Maybe the next up grade could be a favorite system? I'd see a system where every member can choose for exemple 5 favorite picture, video or even thread. They could be displayed on every one's profile (or private) and you could change it if you see a new favorite picture/video/thread. There could be the 'top favorite' picture/video/thread.

so do these ratings go negative? and if so what has the lowest?

Not at all to whine, but I noticed this since the new system was put into place. Do we still receive our daily couple of karma points for having people look at our videos and pictures and rate them? I think that was a good think to incorporate media contributions into your overall score....just wondering if its still happening.

I was thinking when looking at the top rated section, and I don't know if others agree but it seems to me that the videos should now be ordered by the amount of positive score they get. They should not be ordered by the ratio of good to bad. Just my 2 cents

No hate, but...is this more important than the search bar ?

They aren't ordered by that ratio. They're ordered by a fancy algorithm which to the extent of research we've performed is the best possible way to do this. Read my posts in various places for better explanation.

Yes.

We are starting a revolution in media, and this had to come first.

Search is on the list, but there's vast changes necessary to really fix that issue. We've begun a plan, but its a deep rabbit hole.

It definitely still is, but the calculation has changed overall. Since we put out a new ratings system, it drastically affected the way that media karma is calculated. Everyone will notice some drastic changes, but it will be a more accurate representation of what is going on.

Has anyone told you how to search ns with google? its really easy.

Put in "site:newschoolers,com" then whatever you wanna search and bam, pops up. very easy.

is there a way to see how many down votes there are like on youtube? It's cool to see the number of up votes and the green/red bar. But when I see the red on the bar I want to see the exact number of down votes. Is it there and I'm not seeing it?

Seems good, don't change the forums as said above.

The score that is displayed is the number of upvotes minus the number of downvotes.

We liked the idea of one number, as it made finding current good stuff within the 'latest added' view really easy.

looks good!

I actually like the new rating a lot more than the old one because it makes a lot more sense to me, because i always found myself using the star system as a like or dislike and never used the in-between options so i think it is a big improvement

agree.

Also how come videos with lower numbers than others are getting top rated all time? Doesn't have anything to do with comments right, because it has less of those as well?

Score does not equal your ranking order. Videos are ranked on a more fancy and more accurate algorithm.

I understand that the score is the number of up votes minus the number of down votes. I also agree with your point in only displaying the score on the thumbnail versions in all the different categories a viewer can be in such as latest added, top rated, etc to keep the clutter down with one number to accurately display a videos quality. But on the actual video page, where the video can be voted up or down, I'd like to see the actual number of up votes (without the downs subtracted) and then the actual number of down votes similar to on youtube (except they don't do a score, only total likes and total dislikes). I find it annoying that I can see the red bar on the graph but can't find out the actual number of down votes. I made a sample version to help explain what I am getting at.