Rating System - nuwen.net

There are many different rating systems for individual opinions of content (i.e. books, games, movies, etc.), and most of them suck because they are either too fine or too coarse. These are the common rating systems and the problems that they suffer:

Percentage rating systems are too fine; individual opinions are neither accurate nor precise enough for 94% to have any meaning or for there to be any difference between 94% and 95%. (Percentages are appropriate for objective ratings, and may be meaningful when averaging many opinions, but are not suitable for individual opinions.)
Five star rating systems are too coarse; one and two stars both mean "awful", three stars mean "inferior", four stars mean "average", and five stars mean "good".
Four star rating systems are even worse.
N/10 rating systems are also too coarse. Everything at or below 5/10 means "awful", 6/10 and 7/10 mean "inferior", and 8/10 through 10/10 mean "average" through "good".

These are all the same rating system, just viewed at different resolutions - they summarize quality with a linear rating. Consider what quality is: it is a continuum, much of which is occupied by horrible or boring stuff. Some stuff is good, and a few things are really awesome. Directly mapping this continuum to a linear rating wastes a lot of ratings on the horrible and boring stuff, and insufficiently distinguishes the merely good stuff from the really awesome stuff.

What is the real purpose of a rating system? It's not to summarize quality, but to advocate action. Actions are discrete. The simplest possible rating system would be binary, advocating for each item of content either "avoid" or "consume" (i.e. "read", "play", "watch", etc.). In practice, more ratings are necessary. "Avoid" needs no further elaboration, but there's always content that's on the frustrating edge of not being awful enough to recommend avoiding outright, but also not being good enough to actually recommend. This content should get a "neutral" rating. (No one will agree with a reviewer all of the time, so a reviewer should know when both "avoid" and "consume" are going to be wrong for a lot of people.) Also, it is useful to distinguish merely good from really awesome, so that they can be prioritized; really awesome content should be consumed as soon as possible, while merely good content can be consumed when there's nothing better to do.

(Note that the Flick Filosopher's ratings of "green for go", "yellow for maybe", "red for no" advocate action - yay!)

My Rating System

I use five ratings, each of which recommends a different level of action. For convenience, I express them with stars, but they are not a linear measure of quality. Using books as an example of content, my ratings are:

Image	Meaning	Recommended Action
	BAD	Avoid this book.
	OKAY	None.
	NOTABLE	Read this book; low priority.
	EXCELLENT	Read this book; medium priority.
	TRANSCENDENT	Read this book; highest priority.

I rate most science fiction novels highly (three and four stars) because I'm reasonably good at choosing SF novels that I'll like. One and two stars are uncommon; every once in a while, I take a chance on an author who turns out to be bad, or I'm surprised by a bad novel from a good author. Five stars are also uncommon; this rating is very special.

https://nuwen.net/stars.html (updated 9/23/2007)
Stephan T. Lavavej
Home: stl@nuwen.net
Work: stl@microsoft.com
This is my personal website. I work for Microsoft, but I don't speak for them.