A/V Veteran

In 1997 I was on the staff of Stereophile magazine as a consulting technical editor, and also a contributor to (and later the editor of) the Stereophile Guide to Home Theater, at the time a nascent publication subsumed into Sound & Vision in the mid 20-aughts (itself a very long story!)

We were visited that year by Phil Abbate, then a reporter for the AAS (Atlanta Audio Society, today the Atlanta Audio Club). Phil interviewed me during his visit to Santa Fe, New Mexico, then the headquarters of Stereophile. Whether that interview ever made it into the pages of the AAS (or onto the bits of the then also nascent internet) I don't know. But during a recent effort to sort out my increasingly cluttered files, I ran across a copy of that interview, which Phil had sent me shortly after it happened. What follows here is a summation of the second part of the interview, which has been significantly shortened and edited. Turns out, much of what I had to say more than 25 years ago is still relevant today and, where necessary, I’ve updated my commentary with new responses, set off in [brackets], to reflect how I would respond today if asked the same question. You can read the first part of my interview with Abbate here.

Phil Abbate: It appears to me that Stereophile makes the best effort possible to assure that the subjective reviews of equipment are done objectively and thoroughly. Could you summarize the process? And how long do you think acoustic memory is, and what aids do you use to extend it?

Tom Norton: It's an individual reviewer's responsibility to do a careful job at this and I can only speak for myself. [I spend considerable time, often over several weeks, listening to the product under review in a system (and room) I'm familiar with, before coming to a conclusion. Where possible, I then try to supplement those extended listening sessions by comparing the product under review with a known alternative. It's important that those comparative sessions be conducted as close together in time as possible, and at the same volume. By the latter I don't mean about the same. About the same volume isn't good enough. In the days when I once haunted hi-fi stores (be honest — you did that too, or maybe still do!), the salesman would play a music passage, then turn down the level, swap out the loudspeakers (presuming that a loudspeaker speaker comparison was on tap), then turn the level back up, hoping that the volume was "about right!" Of all the characteristics of audio playback, we're most sensitive to volume differences than to anything else, unless the other differences are dramatic.

That leads into your other question: acoustic memory. It's very short. Most of us can't, for more than a few seconds at least, recall in precise detail the subtlest characteristics of a piece of music we just heard. Instead of actually remembering every detail, we instead recall prominent aspects of the sound. You might remember that a particular passage on a given loudspeaker was too bright and not particularly natural. But minutes later, on a different loudspeaker, the same clip sounds both compelling and accurate. (The concepts of "natural" and "accurate" belong to a very long conversation that will never be settled to everyone's satisfaction!)

Abbate: Would it be more objective if the equipment reviewed came from dealer stock rather than from the manufacturer? I guess the root of my question is, do you think you get the same equipment either way?

Norton: As to dealer loans, that might be best in theory. But there are very few dealers willing to loan you equipment for more than a day or two, much less for 90 days! On the subject of getting the same equipment from a manufacturer as is provided to the public through a dealer, I think we probably do. Stereophile's John Atkinson has mentioned that we sometimes do get defective gear in for review, not often but often enough to make him confident that the manufacturers aren't cherry picking the products they send us.

[In the home theater field, I once spoke to a manufacturer's rep about a product we were anticipating for review. He commented that they were sending us their "champion" unit. That made me wonder if he had inadvertently let the kitty out of the bag — that we were receiving a sample that was somehow different than the typical sample on a dealer's shelves. I didn't press him further on this, but it was likely he meant that this "champion" sample simply underwent a doubly careful QC to avoid unnecessary returns and delays. That particular review unit wasn't a TV, but of the hundreds of TV's we've reviewed over the years I only know of two that had to be replaced because of a defect, and even then severe enough to have been caused by shipping damage.]

Abbate: When you can't measure something a reviewer hears or vice versa, would it be more objective to have a third reviewer submit an opinion or let the first reviewer have a second chance, perhaps in another room?

Norton: [This goes to the measurements vs. listening conundrum on which we could spend all week. In general, you'll occasionally measure something that a reviewer hears or vice versa. This is probably more true of components that involve mechanical conversions (such as loudspeakers and phono cartridges) than of any purely electronic component. A third opinion might then sometimes be useful. But since reviewers at both magazines are scattered all over the country, and a third reviewer isn't working for free (with rare exceptions most reviewers are freelance and not on salary) the cost of the added shipping and evaluation time, not to mention delays in getting the review into print, would be impractical apart from in exceptional circumstances.]

Abbate: A lot of audiophiles say, "there are a thousand ways to make a speaker measure flat but only a few ways to make it sound good." What would you trust more to base a purchase decision on, a short listen in an unknown environment or exhaustive e measurements you made with familiar equipment?

Norton: That's an easy one. I'd trust the measurements. But I'd prefer to do both: a loudspeaker that measures good and also sounds good. Such loudspeakers do exist.

Abbate: Would you bet $10,000 that you could tell the difference between two CD players, preamps, amplifiers, or speakers in a double blind test?

Norton: That's easy. I would not, [though I'd have the best shot on loudspeakers, particularly if I could pick the candidates, say a Klipschorn vs. a BBC LS3/5A!]