Golf Digest, May, 2009

DANCING ABOUT ARCHITECTURE
By Ron Whitten

The classic put-down of a music critic is Elvis Costello's line, "Writing about music is like dancing about architecture."

It makes you wonder what he'd say about those of us who rank golf architecture using mathematical formulas. Still, this is what we do. Occasionally we have tap-danced around a course ranking we didn't agree with, but we don't choreograph the results. The numbers are what the numbers are.

Between survey periods, we do tinker with our system, always striving to reach a level of statistical bliss. In the past, we introduced bonus Tradition points to balance the buzz of hot new courses. We also added bonus points to promote walking. Both are gone now.

In 2005, we began training panelists at periodic summits. Our goal is to teach them how to think, not what to think. We remind them, for instance, that they are to analyze Shot Values hole by hole, that backdrops of oceans or mountains are just one element in our definition of Aesthetics, and that we have a new definition of Conditioning. ( How firm, fast and rolling were the fairways, and how firm yet receptive were the greens on the day you played the course? )

Two years ago, we hired statistician Dean Knuth (whose invention of the USGA's Handicap Index system earned him the name "The Pope of Slope") to examine the 2007 results. He confirmed what we suspected: Some of our 900-plus panelists score like Santa Claus and others are simply grumpy old men.

To reduce the influence of such extreme voters, he devised a version of "Winsorization," eliminating outlying scores, and applied it to every course in our 2009 survey. Instead of dropping three highs and three lows (a standard Winsorization practice), Knuth's system is more sophisticated, removing all scores beyond two standard deviations from a course's mean. So it might remove three high scores and one low outlying score from one course, but only one high but no lows from another, because its lowest scores were still within range.

It's applied equitably, and no course is targeted for promotion or punishment. (If you're wondering about Augusta National's ascent to No. 1: It was on top even in the raw scores.)

-- R.W.

Our most recent improvement to the 100 Greatest ranking system will take effect with the next survey. Golf Digest has retained the professional services firm of PricewaterhouseCoopers LLP to oversee the tabulation of the results for future rankings of America's 100 Greatest Golf Courses, America's 100 Greatest Public Courses, Best in State, and Best New Courses of the Year.