Scoring and FeedbackPosted on October 25, 2012 by Greig McGill
Today's peek inside the workings of the NHC will discuss scoring and feedback. Part of this will overlap with judging, but I'm waiting for our head judge to return from Australia to write a dedicated piece on that!
The entire point of the NHC is to give homebrewers a chance to have their beers evaluated by some of the best palates in New Zealand. There is always a subjective element to judging, but our goal as organisers is to minimise this subjectivity in order to give you some real practical feedback on your beer.
We start by having judging panels, rather than individual judges. This year, we are running four tables, each consisting of two experienced judges, and one less experienced. Each beer is evaluated and discussed by one of the four tables, and so subjective differences in opinion are "smoothed" by debate into one authoritative opinion. This is recorded by a scribe at each table. If there is any disagreement which isn't easily resolved in the short evaluation window, the table captain (typically the most experienced judge) has final say, with any disputes being arbitrated by the head judge as necessary.
The beers are assigned scores in several categories - Aroma, Appearance, Flavour, Mouthfeel, and Overall Impression. The last of these may (and surprisingly often does) bear little relation to the first four. Why? Well, often a beer might be absolutely delicious, and the judges love it, but it's completely out of style. The first four ratings are all "as appropriate for style". Styles are another tool we use to minimise subjectivity, so even when we'd love to score a beer highly, it might just not do so well due to being out of style. Conversely, a beer might be spot on for the style, but maybe it's simply not very good - the hop flavours clash, or it simply leaves a negative impression. In this case (and these are rare, as it's VERY subjective) the beer might receive high scores in the style areas, but a low overall score.
The total score for a beer is the sum of the individual parts, and is out of a total of 50. Appearance and Mouthfeel are the least weighted components of the overall score, with Flavour, Aroma, and Overall Impression being the highest weighted. Appearance is notable for only being worth three points, though a judge might also dock more points from the overall impression score if the style notes require a beer to have brilliant clarity, and yet the otherwise good beer is very hazy.
Comments will almost always be provided where the judges can suggest things you may have done wrong, or areas you might be able to improve upon. If there are no comments, then you probably have a 49-50 point beer, or a 0!
If a beer presents as infected, the judges will score it zero, and useful feedback will probably be missing. Have mercy on the judges if this is the case with your beer! It's unfair to attempt to judge a beer which has become infected. The judges have no idea what the flavour behind the infection might have been, and the only useful advice they can realistically offer is "fix your sanitation". This is not lazy judging, merely the best way to be fair and consistent. Given every text on brewing ever written (well, since the causes of infection have been known) has included heavily emphasised advice on the importance of sanitation, there's really little excuse for infected homebrew. The judges will ALWAYS request a repour from the second bottle provided though, so if the infection is limited to one bottle, the beer may still be rated fairly. Comments will indicate the first bottle was infected where this has happened.
Finally, a word on the scoring range, and methodolgy. Gold medals (and 4 points) are awarded for scores of 45-50. These beers are considered world class examples of style. They are not easily achieved, so if you are awarded a gold, you have done very well indeed, hence the 4 point award instead of 3. Silver medals (and 2 points) are awarded for scores of 38-44. These are generally beers as good as many commercial beers available, and are good examples of style and brewing. Bronze medals (and 1 point) are awarded for scores of 30-37. These beers are generally great beers which may fall a little out of style, or have some minor brewing flaws, though nothing a consumer would mind in their session beer.
If your beer scores 21-29, it falls into what BJCP would call "Good", though we consider this a bad descriptor. It's generally out of style and/or spoiled by some evident, though not dominant flaws. No points are added or subtracted. 14-20 is called "Fair" in the BJCP guidelines, though reflects major style deficiencies and/or serious off flavours and aromas. Generally this reflects at best a very bad style entry, and at worst, a very unpleasant beer. You'll lose a point from your total for beers in this range. Finally, if you score 0-13, BJCP calls this "problematic". We consider beers in this category to be undrinkable, often to the point where providing useful feedback can be very difficult. You'll have two points subtracted for a beer in this range.
We use the combined scores of all a brewer's entries to calculate champion brewer. Combined scores may also be used to settle best in class, or best overall beer, where sensory analysis and debate becomes deadlocked!
I hope that's shed some light on how we think, and the "why" behind scoring and comments.
Best of luck!