Member of the Month

KilaTsunami
October, 2024

Online

Welcome to the Newschoolers forums! You may read the forums as a guest, however you must be a registered member to post. Register to become a member today!

Downrate and GTFO.

profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
Well, so as it looks, there are a few people out there really ruining the party for everyone.

I am making the official announcement that we are coming up with a new ratings system based on simple up/down voting, and we are going to scrap the entire historical system in favour of the new one.

However - this is going to take us probably a month or so to get done. Its no tiny task to completely re-think the way that the site calculates ratings, and change up all the systems which look for top rated media. It is going to be worth it.

In the meantime though, downrating will NOT BE TOLERATED. I had Paul build myself and the admins a tool which allows us to quickly check the ratings of individual media items and users. We are able to in a few seconds call up every rating that you have ever done on the site. Sure you think that you can get away with saying 'well I thought that photo sucked' but we can INSTANTLY tell when you're lying. The reason is that we have ratings habits of all times of you, and can look at the patterns.

Take for example, user tpills.

His friend uploaded this photo:

Which is decent, and people liked it. I liked it. I rated it well, as I thought it was a worthy candidate.

However, that day there were other really worthy candidates currently listed in 'top rated of the day'

In order to make his photo advance better, user tpills went through the top rated and did this:


I understand why. Hell one of my own photos was a candidate that day, and I was really hoping to see it advance. I couldn't believe the stab in the heart I felt when I realized that I was doing well, but someone went through and gave me a 1 which dropped my rating. I can imagine how horrible it feels for all of you when this happens, seeing your photo or video doing well and then all of a sudden struck down.

Upon confronting user tpills, he stated that his simply rates stuff a 1 that he doesn't like. What is interesting is that is true of most of you on here... you rate stuff either a 10 or a 1. An immense proportion of the votes on the site are either 10 or 1, with very little in between... though a smattering of votes between 7 and 10.

So you're all begging for an up/down system and we'll give it. But we need to do it right.

What I am offended to see is the fact that when people put their own photo in our system, they all of a sudden try to cheat.

We here at newschoolers try to give exposure to the unknown riders with potential. We try to build something that is representative of what the people want, and it gets pissed all over it not by only trying to game the system but by then pretending like its 'just your opinion'.

Frankly this is the reason that traditional media usually doesn't listen to the masses. Jerks come in and game it for their own reasons.

Noting all of that, we will build something that is impossible to game. We'll use the mechanics that Reddit uses, and get something in place that is really fancy and will make this a thing of the past.

I hate that it has to come to this, but I guess it is what it is.

Until then - DO NOT DOWNRATE. I will find you, I will hunt you down, I will ban you and I will publicly embarass you.

Vote stuff what it deserves. Its not hard.

Sorry for filpping out, but this shit is ruining what we're trying to do here, which is give exposure to the up-and-coming athletes, photographers and videographers.


Jan 4 2012 12:12PM
0 0
profile picture
fujarome
Posts: 8630 - Karma: 1,561
what an asshole....


I've never put any of my own stuff up but unless it's a truly terrible picture, I never rate low. I probably fall in that category of people who rate 7-10. Most of the stuff people upload is just that good to me.


but huzzah for a new system! Looking forward to it.
Jan 4 2012 12:49PM
0 0
profile picture
I_Am_Mod
Posts: 13214 - Karma: 3,753
usually i don't even bother rating stuff i don't like ....
Jan 4 2012 1:02PM
0 0
profile picture
freestyler540
Posts: 13082 - Karma: 18,719
This is great news. Its just funny it took someone to downrate your stuff for you to take action...

Thanks Doug and Paul for the update in the rating system!
Jan 4 2012 1:03PM
0 0
profile picture
Blurst
Posts: 13577 - Karma: 10,169
Ya this happened to all my videos last season, same dudes would rate them at 1 and get all their friends too as well. My videos all started out at around 1 out of 5 ratings.
Jan 4 2012 1:05PM
0 0
profile picture
t_rob
Posts: 6433 - Karma: 11,458
The thing is that uprating is the exact same thing as downrating. That about that for a second.
Jan 4 2012 1:08PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
Think about this for a second, giving a 10 to an item with many votes around 9-10 (as it seems to be good) will not raise the score significantly.

Giving it a 1 will lower the score significantly.
Jan 4 2012 1:21PM
0 0
profile picture
t_rob
Posts: 6433 - Karma: 11,458
Yeah, obviously. However, that was not what I was trying to get at. Think about it again.
Jan 4 2012 1:30PM
0 0
profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
Well TBH that is why I add stuff. I need to experience what you guys experience to really come across issues. Its way easier when I can follow something through the system.

That is the hardest part about developing NS - its easy to forget what the end user experience can be like because we are so close to the back end all the time.


Jan 4 2012 1:48PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
Well stop being so coy and just say what you are trying to get at.
Jan 4 2012 2:24PM
0 0
profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
No it isn't, because most people cluster their votes around the top. So most votes on the site are between 7 and 10.

Now what you are saying is that someone voting their own media a 10 when it clearly sucks that is upvoting. Agreed... but it doesn't last long when it really sucks. The problem is that downvoting something that is actually good pulls it out of the top rated lists, because its hard to get in there. If there was a worst rated list and people wanted to be in that, then upvoting would really screw them over.

When 10 people rate something a 10 and one rates it a 1... then that totally messes up the average. Something that got a whole bunch of 9s would probably beat it, even though that second one would be the superior media item.


Jan 4 2012 2:52PM
0 0
profile picture
t_rob
Posts: 6433 - Karma: 11,458
QUESTION: On an average day, how many pictures get enough votes to become picture of the day? Is it possible that not a single picture would get enough votes without solicited ratings?


Math Time!:

We are talking about pictures that are in contention to become picture of the day so we can assume the the rating is going to be somewhere in between 8 and 10. The actual algorithm is some combination of rating and amount of ratings. There could be more components and what the value of each input is isn't going to be given out anytime soon. No matter how many ratings a picture has, it is going to be most effected by the rating farthest from the mean. This is going to be the 1 because 8 is farther than 1 than 10. We both agree on that. I just said it to put down an intro to the people who have no idea what we're talking about.

(Sorry, I'm not trying to pick a fight but I most likely sound like a Giant Douche. Mike, I really respect you BTW you're one of my favorite members.)

What is also in the indisputable math department but I've never seen brought up explicitly is than the more ratings a picture has, the less susceptible the mean is to movement and also the less power a singular ranking has. The pictures in contention for POD have a good amount of votes. There is a minimum amount of votes needed to become POD. I don't know what it is. That is why tons of 1's are needed to bring a rating down.

That is also why the top rated pictures of all time don't change... ever... until someone dies. (That's a sketchy debate I think should happen some time.)


Argument time:
Question to picture of the day winners: Did you get other accounts to 10 your photo?

I 10 most my own pictures. I don't deserve them so I assume other people do this to their photos. I get pm's asking me to rate up edits all the time. Everyone knows this happens but we don't talk about it. I don't see a difference in getting a bunch of accounts to 10 a picture in attempt to secure POD than 1 bombing another picture. This sometimes happens at the same time but who knows for sure how often.

By rating any picture, it changes that picture and every other pictures in contentions likelihood for becoming POD. Votes are not independent of other pictures. That is why I don't see a difference in between a 10 spree and a 1 spree. Every vote is at the expense of another picture nomatter what that vote is.
Kapish?

Do you see how upvoting could theoretically be a problem?


Jan 4 2012 3:52PM
0 0
profile picture
eheath
Posts: 61593 - Karma: 122,968
It'd be interesting to see which members ALWAY rate my stuff 1s, i get a few every time i post a video on NS, even a video where i received a large amount of praise for. i guarantee some of them would be quite well known, possibly even people I know.
Jan 4 2012 3:58PM
0 0
profile picture
t_rob
Posts: 6433 - Karma: 11,458
I opened the reply and didn't check your post before a replied. Agreed on most points. I think you're overlooking how much upvoting occurs. Maybe if the top 10% of the highest and lowest votes were thrown out of the rating, it would fix this. Also throw out a users availability to rate their own media as well. Upvoting happens. Downvoting happens. It's easier to track but before it is too late most of the time.
Jan 4 2012 3:59PM
0 0
profile picture
JStrathern
Posts: 8747 - Karma: 1,536
Jan 4 2012 4:31PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
This is where you lost me - me can no head do math good.

But on a serious note, I still don't see upvoting being the major issue, mean relations or not as we don't know the algorithm. It must be a potpourri of Views + Comments + Ratings.

For example, if a video I posted had 1000 views and an average of 9.88 from 15 votes with a few comments, I feel it still wouldn't take the cake over a video from the same day with 6000 views, 200 ratings with 9.87 average and over a 100 comments.

Jan 4 2012 4:48PM
0 0
profile picture
PJS.
Posts: 11943 - Karma: 895
tpills just got fuckin served.
Jan 4 2012 5:52PM
0 0
profile picture
Shza.
Posts: 425 - Karma: 92
I think the upvote, downvote thing is an awesome idea. It works wonderfully on reddit, and no one ever complains about it there.
Jan 4 2012 6:02PM
0 0
profile picture
skitheWest.
Posts: 2304 - Karma: 1,276
so is this stuff just for media? or are you getting rid of the karma system as well?
Jan 4 2012 7:16PM
0 0
profile picture
Deluge
Posts: 4401 - Karma: 805
.
Jan 4 2012 7:25PM
0 0
profile picture
joris.blanc
Posts: 914 - Karma: 6,584
Its horrible when this happens. I'm happy to read that this will change (or at least limit it).

I'm not the biggest fan of the like/dislike system, I really appreciate to be able to rate a 8 or 9 it the pic isn't perfect in my opinion (don't know if landing on snow or airbag, little stuff like that). I will kinda miss that, but if this new system can limit the downrating I'm all for it.

I just didn't really understand if the old ratings will be converted into the new system or not and if yes, how will it be (ratings 1-5 = dislike and ratings 6-10 = like)

Will be interesting to see the new rating system !!!
Jan 4 2012 8:58PM
0 0
profile picture
Piste.Off
Posts: 11586 - Karma: 1,860
Just wondering, is this going to change the karma system at all?
Jan 5 2012 12:37PM
0 0
profile picture
Trace.
Posts: 1460 - Karma: 635
This, if you don't have anything nice to say then don't say it at all. I don't think I've rated anything under a 7 unless it was just mindnumbingly bad
Jan 5 2012 1:08PM
0 0
profile picture
TACO-DOG.exe
Posts: 11988 - Karma: 12,261
does this go with rating everything 10? because that's also a little unfair
Jan 5 2012 1:11PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
There's discussion in the thread about that a bit upwards. But still, I think that it goes like this:

1. You rate something a 10 and it goes up
2. People will notice it
3. If it deserves to be there, it will stay up

Doesn't work the same way if people give you a 1 as the first score, it's pretty much gone forever.
Jan 5 2012 1:24PM
0 0
profile picture
84891jnf92048
Posts: 11552 - Karma: 2,550
and what about current ratings on images and videos? will they stay, and if so, how?
Jan 5 2012 6:03PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
Learn....to.....READ.
Jan 5 2012 6:10PM
0 0
profile picture
84891jnf92048
Posts: 11552 - Karma: 2,550
yeah that helps man! thanks!
Jan 5 2012 8:48PM
0 0
profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
No you're totally right that this happens as well... but because most people rate items from 7-10 anyway, it doesn't throw off the calculation as much when you uprate with a 10. We've designed the system around the idea that most ratings are betwen 7 and 10. But 1 just throws a monkey wrench into things... if it doesn't deserve a 1.

Mainly the reason that upvoting doesn't mess with stuff as much is the difference between what it deserves and what it gets. You'll never get yourself on the homepage with a total shit picture and upvoting alone. Shit pictures get low ratings from people honestly rating them as shit and they drop off the face of the earth super quick. The thing that picks the media of the day/week/month takes into account that higher rated items remain such over time and with lots of ratings. Its very strict that way, and actually doesn't always pick something of the day. The true definition of POD is:

'The best photo of the last 365 days that hasn't been picked as Photo of the day already."

It weights you getting a bunch of ratings quick. So yes, up voting here can help you get on the homepage. However, with the amount of people voting (and our quick ability to change the criteria) you'll almost always get beaten by something that is actually good. People flock to good things quickly, and you can't top the masses liking your photo.

Finally, its very easy for us to limit voting to IP addresses. I think that is a change we'll immediately implement, so you can't vote by making multiple accounts.

Above and beyond all else though, what this has taught me is that the current system is broken. Take tpills for example... he's mad at me because I ousted this in public. I however contacted him and in a very friendly manner asked him if he downrated... all I wanted was to fix the system. But he continued to lie to me like I couldn't figure out what was going on. As I dig deeper I see that this is prevalent absolutely everywhere on the site... and as sad as it is to say, you can't trust the masses to do the right thing.

I however don't believe that what we're doing is wrong. What we're doing here is creating a media system which is powered by up-and comers and curated by the very people that like it. We're creating a true media source by the people for the people. We NEED a system that takes into account the people that try to break that beauty for their own gain.

As people have mentioned, upvote/downvote is a good system. However, simple upvoting/downvoting doesn't work either... you can game the shit out of it. So we went deep into research about how some of the really big boys do this, and found an excellent solution. I'll re-post this from another thread:

"I actually really like that idea, but I think that it might lead to less ratings and more fights. As well, if someone just wants to be a dick and not give a fuck they could simply downrate, post their name and say 'go fuck yourself'. Same issue happens, and we still have to police through gestapo tactics.

I'm pretty sure that the right idea here is to go with up/down voting, and then calculate the score as the lower bound of Wilson score confidence interval for a Bernoulli parameter.

The formula looks like this:

Click for larger image

Then we'll be fucking bad-ass."

THe solution here is to use extremely complex math that takes into account people gaming the system. We can add a few touches to it that are specific to the Newschoolers ecosystem, and then make algorithms that choose top rated stuff based off of this math vs. the old one.

Will the old method of top rated be perserved? No. However, its broken, and too many people are gaming the system. We need a new system, and to build it we have to first break the old one.

What we're doing here is special. This will send ripples throughout the universe of ski media. We'll encounter a few bumps, and we're going to scar a few members who are acting against our community.

We will however prevail, and we will take the power from traditional media by force.

We are newschoolers.

Expect us.






Jan 6 2012 11:06AM
0 0
profile picture
Asterisk
Posts: 954 - Karma: 199
Damnit. I just rated a good video a 1 completely by accident when it was first posted. Anything you can do about that?
Jan 6 2012 11:07AM
0 0
profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
Shit... no not right away. I'll try to get Paul to fix it.

Thanks for being honest though.


Jan 6 2012 11:11AM
0 0
profile picture
Asterisk
Posts: 954 - Karma: 199
This will definitely fuck it. I was the fourth rate. Thanks
Jan 6 2012 11:16AM
0 0
profile picture
t_rob
Posts: 6433 - Karma: 11,458
I'm unfamiliar with reddit. I just stopped by their website for a little knowledge. It seemed like overall hit or miss which I do not think NS media is. NS media is a lot classier and more diverse than reddit. NS media is everything from work no name kids that could just as easily appear in a photo annual to today's POD which is a 2 year old wearing a tall hoodie to some chick snowboarding down a green circle. There are some well composed photos and there is some straight crap. On NS we are ranking the coolness of the picture whether it is the skiing or the photo itself.


Does NS get a large enough sample size to run the wilson confidence lower bound thingy. The current top pick on reddit was uploaded an hour ago and has 346 comments. NS doesn't get that kind of traffic. NS doesn't think that number of comments expect in the all time top pics.

I'm also starting to come terms with going from a ten point rating system to a 2 point system. Lets say a picture should be rated a 7 and everyone believes on this impossible fact. In the current 10 point system I have the power to throw in a 7. However, for something to be a 7 in a 2 point system, we have to rely on 70% ups and 30% downs. (That's not actually right but its close enough) So what would I vote because I think somethings a 7? Up or down? Like I said, I think NS media is more complex than reddit and it could be hard have a 2 point voting system.

On the other hand, the 10 point scale is so broad it's ridiculous. Has anyone ever rated something a 2 ever? I usually only rate things 5,7,8, or 10. There is a lot more of a difference between a 7 and an 8 than a 1 or a 2. Then maybe a 2 point system would be better. Blah.


EFF IT. Run a trial run for a week and go from there.
Jan 6 2012 12:11PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
I think one thing that should be taken into account in the new system is to change your vote afterwards, such as with karma. Many times I've meant to click 10 or 9, but accidentally clicked on something lower as my mouse is so damn fast and it can't be reset.
Jan 6 2012 1:30PM
0 0
profile picture
JStrathern
Posts: 8747 - Karma: 1,536
YES this always happens to me
Jan 6 2012 1:34PM
0 0
profile picture
.RYAN66
Posts: 5381 - Karma: 235
It is so great to see that you guys are trying to make the sight better, but idk if this is it. Like said earlier I only rate something of I think it is cool. The only time I give a 1 is if I think something is straight up stupid.
Jan 6 2012 1:35PM
0 0
profile picture
freshlyDIPPED
Posts: 1160 - Karma: 121
Thanks Bishop for keeping it real! Downrating is for cheaters.
Jan 6 2012 1:38PM
0 0
profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
Reddit is like the craziest site on the internet. The key though is about their ratings system, its excellent. Sure we don't have the same content, but that is where you have to tune it to be specific to our site.

Right now the POD issue is whether or not we include what people LIKE or what is a GOOD PHOTO. Currently, its what people Like. This is sort of a separate issue, because it can be solved fairly easily. For instance, photos like today's aren't actually supposed to show up, because the 'people' category isn't supposed to be up for consideration (whoops), just like Gear. The 10 month old kid wearing the tall Hoodie, was a photo though that people liked.

That is actually insanely easy to fix, and has nothing to do with ratings. What we might have to do is start reading the Exif data, or measuring the quality. I could see you rating the photo like thumbs up for different things - photo quality, riding talent, awesomeness - that sort of thing.

Also in terms of the sample size, well sure. The lower bound wilson score can be used for samples of any size.

Note also that your analysis of how to calculate thumbs up / thumbs down is also not correct. You can't just measure ups and downs and calculate it like blam.

Read this post by one of the Reddit guys - http://www.evanmiller.org/how-not-to-sort-by-average-rating.html - That is an excellent explanation of the ratings problem, and where we're starting our philosophical change of our ratings mentality.

Finally note that yes, we lose the wider point scale per person by 2 point rating system, but what we gain is the fact that knowledge really comes out of the masses. No one person here should be able to make a decision - everything should be based off of what we all think and vote for. Like a democratic media empire basically. 2 point system takes away huge amounts of ability to cheat, but also relies on the masses collective intelligence.

Funny enough though, the masses intelligence if you make the data input simple enough and to follow a solid set of rules is quite good. Especially here.


Jan 6 2012 3:05PM
0 0
profile picture
Milksteak
Posts: 9948 - Karma: 833
Excellent.
Jan 6 2012 4:20PM
0 0
profile picture
I.B
Posts: 8137 - Karma: 24,351
with the guy that fucked up his rating, that has happend to me before, like i've meant to hi 8 or 10 but i acutally hit 3-4 and i can't redo it. A pop up window would be nice, but a pop up window that could be made so if you do'nt want it, you don't need it. But at least it's there.

Jan 6 2012 4:50PM
0 0
profile picture
Mike-O
Posts: 14593 - Karma: 1,089
No no no.

Just the ability to change your rating. No cluttering pop-ups needed, selectable or not.
Jan 6 2012 4:56PM
0 0
profile picture
corona
Posts: 12794 - Karma: 32,720
I think 3 years ago I rated 5 or 6 pics in the top rated section 1's just because I wanted to see the numbers change. It was just one of those days.
Jan 6 2012 4:57PM
0 0
profile picture
I.B
Posts: 8137 - Karma: 24,351
This actually, would be nice.

Jan 6 2012 4:57PM
0 0
profile picture
t_rob
Posts: 6433 - Karma: 11,458
I'm pretty sure the karma exchange is a small input. Karma system is based more on the amount of videos you upload than what other people rate you on your profile... To be blunt. Check the Karma thread. It's hot right now.
Jan 6 2012 7:43PM
0 0
profile picture
Mr.Bishop
Posts: 22708 - Karma: 76,635
I mean thanks for being honest. This is the thing... with the current rating system, few people just want to see what happens and blammo... fucks the whole thing up.

The thing that is funny about this, is if we all decided to be a super respectful and helpful content community (very similar to what happens on vimeo) this issue would be irrelevant. We could help each other, be mature, and carry ski media into the future together.

We still can... but we need to eliminate a few shitheads.

Sometimes I wonder if this is what trying to build a perfect socialist society would be like. At a certain point, you just might have to shoot the people that don't help out the community as a whole.


Jan 6 2012 11:09PM
0 0
profile picture
I.B
Posts: 8137 - Karma: 24,351
I have a question

If i happen to see a couple bad edits, that IMO are pure shit, will i get banned for throwing low numbers on them? I mean that can't be right.

I know what you're saying, with the downrating and all. I've done it before. (when i first joined)

Jan 6 2012 11:12PM
0 0
profile picture
james6345789
Posts: 4763 - Karma: 467
This is a way better system.
Jan 6 2012 11:23PM
0 0
profile picture
TOAST.
Posts: 7478 - Karma: 36,012
I think it should be a 3 point rating system. Great, average, bad. Because there is a whole lot of room between something that is good and bad.
Jan 7 2012 12:11AM
0 0
profile picture
Downtofilm
Posts: 6896 - Karma: 20,395
Word. Some good ideas in here.

Threads for sure
Jan 7 2012 5:17AM
0 0
All times are Eastern (-5)