Primary tabs

Are Blind Audio Comparisons Valuable?

In this week's Home Theater Geeks podcast, Home Theater and Stereophile contributor Steve Guttenberg argues that blind comparisons of audio products are meaningless for several reasons. First of all, he claims, most people cannot reliably discern the difference between similarly performing products, and perhaps not even between products that perform quite differently. As you can see in the graph above, listening tests conducted by Floyd Toole and Sean Olive reveal that blind comparisons of four speakers resulted in much more equal preference ratings than the same comparisons in which the listeners knew what they were listening to.

Also, Guttenberg maintains that the tester's ears are psychophysiologically biased by the sound of one product while listening to the next product. Finally, the conditions under which the test is conducted are rarely the same as those in any given consumer's room, so the results mean nothing in terms of deciding what to buy.

Do you agree? Are blind comparisons of audio products valuable? On what do you base your position?

Vote to see the results and leave a comment about your choice.

Are Blind Audio Comparisons Valuable?

EdSubtwoi's picture

I feel that blind testing is beneficial because of what I read in Michael Fremer's posts. He indicates that in a blind test he can indeed tell the difference between various implementations of certain types of components, including amplifiers and cables. In the one amplifier test he cites he was correct 5 out of 5 times and John Atkinson was correct 4 out of 5 times while all the other scores were abysmal. He also cites cable blind tests he has aced. What this tells me is that blind testing was beneficial in revealing a difference but only detectable by the very few who have sensitivity to notice. I'm not sure if that is a blessing or a curse. So perhaps the correct assertion is that there is no audible difference among cables and amps for the vast majority of listeners. Of course we are assuming we aren't throwing junk components from K-Mart into the mix. For people like me the difference is not audible, in particular for cables, whether Monster, Audioquest or Monoprice, even in a non-blind test. I sat in on a Nordost interconnect cable demonstration at the Toronto show (TAVES) last fall (great show BTW) where the host started with the cheap generic pair and moved up the cost line of Nordost cables. I tried real hard to hear a difference, but I couldn't, even though it was a sighted test. I am more sensitive when it comes to comparing speakers, especially how well they “disappear”. So for me personally, applying the law of diminishing returns, I skew my purchasing decisions away from cables and more toward other components, especially speakers, where I can tell the difference. All that said, I'd hate to go shopping and making my decisions via blind testing. That would definitely not be beneficial.

Pedal's picture

I think ABX is great if you need a null result. Like when the music industry wanted to push mp3 into the marked as sounding “indistinguishable from CD”.

Pedal's picture
ulrjoerg's picture

From my experience, I tend to agree with Steve Guttenberg's statement saying that differences diminish in blind tests. Our brain always filters, never perceives the whole content at a time. We need to study how that filtering process works - not an easy thing for an engineer! My observation is we discard subtle differences in a blind test, eliminated as non-significant information by our brain. I would suggest a "guided" test as opposed to a simple preference test, to force the brain to concentrate on particular attributes. Then it might become less relevant whether the test is sighted or blind!

The ad-hoc experiences reported in this discussion seem to confirm my observation. The first impression, while not knowing there is a test going on, may reveal the truth!

As an engineer, I don't believe in cables changing the signal. It's our perception (brain information filter) that changes and creates the illusion of hearing differences. Curiously looking forward to Michael Fremer's demo, though!

d7green's picture

I learned while working in a stereo shop with a display behind a curtain, that with identical speakers in front and behind, listeners judged the hidden speaker as significantly better. My conclusion is that the mind tells you that the sound is coming out of a box when you see the box. With a curtain trimmed to look like a stage, the mind imagines musicians behind the curtain and makes supportive instead of disputing assumptions. I did the same comparison at home by moving the curtain just to confirm that speakers position was not the controlling factor. I couldn't tell the difference with my eyes closed, but I found it more pleasurable with the curtain with eyes open. Yes, it's self deception, but isn't that the point, to make it feel like being there by any means available?

Regarding speaker position, small changes can cause audible differences. One manufacturer (sorry I don't have the name handy) built a pneumatic rotating platform to move each speaker under test into the same position to eliminate this difference and the directional cure that gives away which speaker is playing behind the curtain. Since two speakers cannot occupy the same space, are all other tests tainted?