Changes: Beer Hads, Full Reviews & Ratings

You can write as much as you want. Has to be MORE than 100 words.

I agree that the lists, rankings, and beer scores are a big part of it. However, I'd be willing to wager that the percentage of traffic devoted to reading individual written reviews is tiny.

People want scores, lists, compiled information.

Those are both great ideas.

Dope

Regarding the problem of users with all 1s or 5s, couldn't you compute a uDev score to evaluate which users you need to address? I would think you could average rDev^2 for each user and identify them almost instantly.

Keep had scores as a second rating to compare, I use for personal review/tracking. I like reading others full reviews AFTER I drink and check it to compare my own tastes. I do not feel I have the "expertise" to write a full review as I am still fairly new to the full craft beer experience and I believe that those that take the time to fully review will give a beer a more fair shake than someone ticking a box on a style the hate falsely lowering the score. No matter I will adapt and continue to love this site. Forums and others from the community provide so much info along with everything from BA that I will keep coming here daily to my favorite beer site. Looking forward to the app release!

I'm going to guess that most of the users that have the long list of 'hads' with a score of 1 were merely ticking the beers and not actually wanting to bring the overall score of the beer down.

I like the idea of others that perhaps the full review scores be weighted, so as to be worth more in the overall score.

Also, perhaps they can have 3 levels of tracking: Had (no score), Had (with score), and Review (with weighted score).

p.s. I'd hate to see my name show up on one of these witch-hunts, I'm trying to do good reviews! haha

I only use the had to rate beers. I won't feel confident until I've tried 500 or more before I can give a written review.

Maybe I missed this, but did not see any mention of it from the Bros. Why are there no brewery listings on the beerfly/places pages?

Example
http://beeradvocate.com/beerfly/city/65

I have another suggestion and some feedback on the changes.

SUGGESTION: Have you guys given any thought to segmenting by additional beer traits other than simply style? The biggie that jumps into my head is barrel-aged (or wood-aged) vs. non barrel aged. I find that BA, as a whole, seem to have a preference for barrel aged beers, so it "skews" the ratings/rankings (see Russian Imperial Stout Top Beers) I'm just as much of a barrel-aged beer fan as the next guy/gal, but it'd be nice to be able to sort or filter on this trait.

Anyone else have an opinion on this?

FEEDBACK: I like that your'e including all the ratings/hads but do find scrolling through the empty review list (no text for the hads) to be a bit annoying - glad a fix is planned for that. A sort/filter for overall score including vs. not including had's would be nice also. I agree with many posters that say a review is more thoughtful - as the reviews I've done are much more purposeful and thought out than my hads.

The Iphone app is also much welcomed, as I'm an avid Untappd user to keep track of beers I drink with a quick rating (although I don't like that it's only a 5 point scale, as I find a lot of beers I want to rate 3.5 or 4.5 (I think a 10 point scale [or five point with .5 increments] is perfect, BTW), and usually am not on my computer when I'm drinking. Not sure if I'd pull away from untappd, but would definitely give it a try as I like the wealth of thoughtful ratings BA has.

Cheers!

I agree with you on the users that have all '1' Had scores. Still, whether or not there's actual abusive intent going on, these ratings should not be counted (should not affect the overall score). You, on the other hand, do a stand-up job of giving accurate and widely-varied rating scores for both your 'Hads' and 'Reviews'.

And, I like the idea of three categories. Perhaps 'Had', 'Rated', and 'Reviewed'.

Cheers!

not to be hypercritical since i love this site, but...

...they are also going to put out a mobile app, and redesign the site, and...and...and...xmas of 2013 is coming eventually too. i guess i just don't understand why you'd put something half baked out there. this change essentially makes the reviews and scores useless until the additional "going to make it where's" are incorporated.

putting out a massive change without really getting any input from your core user base is suicide. why not create beta.beeradvocate.com and put these changes there first let the users hash things out and discuss before going live. i understand that this place isn't a democracy, but essentially 3 people are making drastic changes and decisions that affect the entire user base. what will end up happening is your user base will just dry up. like i mentioned in my last post...every change here results in losing long time contributors. we've seen what that did to the trading community, why do that to the review section of the site?

why not make hads just that...something you had that you didn't rate. if people want to tick something, let them tick it...but not rate it. if people want to review and score something, let them do that.

Noticed alot of hate for the new system on the front page, wanted to provide another view point.

I'm a fan of combining the Reviews/Hads. There's a time and a place for critically reviewing a beer, and usually when I'm drinking I'm hanging out with people at home or at a bar. Even though I'm surrounded by other beer nerd at all these places, It's not the place to constantly be on my phone taking notes, holding beer up to light and smelling tasting beer like a pretentious douche.

That and the fact that there are styles that I just do not know enough about some styles to give a good review... but I still want to track these beers. So I give them an overall had. I do put thought into it, and it's really nice that I now have them in the same place and can look at Top 100 lists and visually see how many I've had.

That being said, I do not think Hads should factor that highly into the beers overall score because many people do not put that much thought into it. I'm also not a fan of Hads showing up in the reviews section, but I can live with that.

TL;DR Fan of the new system, but the Hads should impact a beer's score that much.

Todd, are you saying that you're aware of the aforementioned users already? Consider my posted links above reports, but if they were already on your radar, then I'll need report no further. Are you saying that they have an allotted amount of time to revise or their 'Had' ratings will be purged?

From the sidebar on the reviewing form page:

All "5's" should be rare (ie. cannot improve anymore). Just keep that in mind.

Obviously taste is subjective, but clearly the Bros want to constrain subjective variation within some reasonable limits, limits dictated by norms of evaluation accepted on the site. Learning those norms of evaluation takes time and experience, and some dude rating the 3rd beer he's ever tried a 5 without anything substantive to say about that rating obviously is not the goal of beer reviewing here, and really doesn't help anyone looking at beer ratings.

anyone who takes the time to start typing up a review will not abandon that review and move onto a "had" if they missed the character limit. they'll add the extra characters.

We should just delete all users from this site that don't write reviews. I mean they don't contribute in any way, right??????

This recent change of delineating between Hads and Reviews on the beer tracking page is a step in the right direction.

There are extreme examples for sure, someone linked earlier to someone who rated all 1s, clearly these anomalies should be dealt with, but if someone wants to give Stone IPA a 5 because to them it is the best IPA in the world, who are we to say they are wrong?

Sorry I don't have time to read everything, but to prevent trolling hads, especially one person signing up for multiple accounts, you could have a 50 had minimum for that user's scores to factor in to a beer's rating.

I have, in fact, done that on a couple occasions. I.e. just 'Had'ed a beer when I really don't have anything noteworthy to say about the beer (typically because it isn't a noteworthy beer).

Gotcha. Thanks, Todd!

I am not too thrilled with the changes. I did not like the numerous Hads coming up with a review and changing the beers final score. .. no text just a number which is not useful at all. I suggest a way for users to only view actual reviews with content. The why (i.e context) is important.

Next 100 characters minimum. I want more content not less. Tasting notes should be detailed. Less details make it harder for me to discern between those that are accomplished at judging beer (i.e Certified Ciccerone, Master Ciccerone, BJCP etc.) and those that are not.

Lastly, I would really like to be able to toggle between Had reviews and full reviews (the format with more characters - not less) so I can compare the difference in the final score. rDev and the other stats are more useful when I can manipulate the data to compare between people who actually know how to review a beer (people with training, brewing experience, or just have had hundreds and hundreds of beers) as opposed to some 21 year old fan-boy saying Heady Topper was a 5.

My first reaction was not positive for adding the hads to the reviews, but I always review the beer, so it did not appear to affect my use of the site. Then I decided to gather some actual data before deciding and was a bit shocked by what I saw.
The bogus "1" and "2" rating phenomenon is very clear for many beers. My percentage of over-rating shot through the roof on some of the beers, and when I looked is was all a result of adding 50% hads with ratings of "1" or "2" to overall decent review scores. As noted and documented above, these are clearly the result of lazy or overly fast use of the hads and not an actual result of wanting to rate the beer.
Check out this local Elysian beer that has less than 50 ratings: Hansel and Gretel. The actual reviews range from 3.0 to 4.08 (with one 2.38 outlier). the hads range from 1 to 4 with *6* 1's or 1.5's, *2* 2's. So 8 of 28 hads have a low rating that is completely outside the range of the scores from the reviews (INCLUDING the outlier). if you drop the big outlier 2.38 review rating which has an rdev of 22% (and undoubtedly much higher if compared only with other reviews), there are total of 16 of 28 that are under or just touching the bottom edge of reviews (3). This really shows that the rating columns between 'reviews' and 'hads' are vastly different numbers.
Even weeding out the "1"s, which are almost certainly a systematic error in use of the 'hads' form, would not be sufficient because ALL the 'hads' ratings are obviously much, much lower for every beer than the 'review' ratings.
After looking at a bunch of my reviews, my confidence in the 'hads' review numbers is near zero. There is no possible usefullness in including them in the total averages, and I personally don't see getting any useful information from even looking at them at all
My strong personal vote is to remove the 'hads' from the review section completely. They are a bad, bad, bad estimator of beer quality and make scanning the actual reviews more difficult.
Edit: I see that Todd addressed the issue of filtering the hads from the view, which is great, but I'm still not convinced that the hads review values are comparable to review scores for many beers.

I'm not worried about my own reviews, I'm worried about the average quality of reviews on this site. Interesting, well-written reviews (especially those by BuckeyeNation) were what prompted me to join BA in the first place, and the higher minimum character limit at least helped keep reviews to a certain standard*. I don't want to have to filter through tons of useless reviews to find the good ones, and with this change I could see the "top reviewers" becoming dominated by terse, uninformative reviews.

Maybe criticizing the 100-character limit is the wrong way to go, though. Perhaps a better solution (already mentioned in this thread) could be allowing "liking" of reviews or of reviewers, and allow sorting by that. Ideally, the reviewers who put more care into their reviews will percolate to the top. I would be happy with that approach, and the lower character limit wouldn't be a problem.

(*) Granted, there's a lot of variation in reviews, but just compare the reviews for a beer on BA sorted by "Top Reviewers" with the same beer on the other site sorted by "Top Raters". It's typically a world of difference.

I'm in support of combining the two reviews/hads, however I'm not in support of lowering the reviews minimum to 100 characters. If your taking the time to analyze and critique a beer, what's a few more words.

Lots of good suggestions here, so far.

I'd like to add that I think the minimum # of reviews/hads for a beer to make a Top List and for a beer to make the Top Beers of Fame list, specifically, should be bumped up. (25 for Top List? 2000 for Top Beers of Fame?)

Also, I like the idea of 'Had', 'Rated', and 'Reviewed'. Or, maybe, have the 'Had' review page drop-down box read...

'Did Not Enjoy/Would Not Drink Again',
'Mediocre/Not Something I'd Seek Out',
'Average/Would Revisit',
'Above-Average/Would Like More Of,'
'Exceptional/One Of The Better Of Its Class'

And, then you could assign reasonable scores for each rating above. Perhaps...

'Did Not Enjoy/Would Not Drink Again', = 1.5
'Mediocre/Not Something I'd Seek Out', = 2.25
'Average/Would Revisit', = 3.0
'Above-Average/Would Like More Of,' = 3.75
'Exceptional/One Of The Better Of Its Class' = 4.5

A couple posts of mine disappeared as they were responses to some of the user callouts that got deleted from this thread, so I'm going to try to reproduce the gist, which I think had some independent merit:

Some of the users who were named have all their Hads rated 1. This is arguably a misuse of Hads, but also a consequence of the UI making 1 the simplest rating to give. If the UI defaulted to 3, you'd probably see these users with all 3s. This does seem problematic, although I think the effect is minimal so long as users are consistently giving all their ticks the same rating.

However, some of the users had predominantly 1s with smaller numbers of 4s and 5s. I'd like to suggest this might be okay. Among much of the trading & reviewing community, we've normalized an attitude where it's only to give a 4 to a beer that's "okay, not great, glad I got to try it once, wouldn't seek it out again". I don't see a problem with a contrarian view where people give a 1 to a beer they wouldn't buy again and a 4 or 5 to beers they choose to drink all the time. I hardly think this will be the calamity some people expect, and might lead to greater diversity in the Top Beer lists.

I'm ok with adding "hads" to the score but think that full reviews should be weighted more heavily.

A good beer is still going to get good scores and a bad beer will not get undeserved praise. Yes, the lists will get shuffled around but it's not the first time it's happened on the site and probably won't be the last.

In the end, they're just numbers. It's all about enjoying and advocating great beer.

Personally I like the change. I have to imagine that a lot of reviewers reverse engineer the individual rankings to get the overall score where they wanted it to be in the first place.

I think the Popular list should have a 100 review minimum vs 10.

aside from the jackasses who are trolling beers they can't get by '1'ing, this is a really cool change.

Todd - thanks for all the hard work first of all. Consider me as a passive user of the site. I don't review . . I just read reviews from those I trust. Detailed tasting notes are helpful. Maybe allowing a user to pick reviewers that a user trusts would be helpful.

Since the forum has implemented "following", perhaps the reviews from people you follow are listed first on the page? Similar to how when you click the "bros" score it lists a few reviews sorted to the top?

I am all for Had's being part of the equation but why are they being displayed where the reviews are?
Why not just show a had's score that can be clicked on and show all of the people who have had it there?
I mean come on, just look at the new #1 beer, Heady Topper.
25 listings shown on the first page, 5 reviews.
How is that helping anybody?