The Price of Listening Tests

The Neumann KM184 is one of my all-time favorite microphones and, at $1,000/each, it's a microphone that I would gladly accept a cheaper-but-equal substitute. Michael Joly Engineering's Octava modifications are interesting, but if you listen carefully to his sound examples on the company's sound sample page I do not think you will agree that he's captured the sound of the KM184 with the SDC 991.

Electro-mechanical devices are insanely complicated, both to manufacture and to design. Anything that sounds good is an impressive accomplishment. Sounding spacious, detailed, and natural is miraculous. I think the Joly mics are impressive, but not enough to displace the Neuman small element condenser from it's holy place by the Grail of beautiful electromechanical instruments. At least, that would be my position based on the sound samples we have from Joly.

There are lots of problems with listening tests of all sorts. For over a century it has been widely believed that high quality classical guitars – their soundboards being the major exception – have to be constructed from tropical woods. This applies particularly to backs and sides (rosewood/mahogany) but also to necks (mahogany/Spanish cedar), fingerboards and bridges (ebony/rosewood). Indeed most builders and players alike consider such tropical hardwoods as the preferred, if not the only choice for instruments of the highest tonal quality. Whilst some non-tropical back and side woods have become established for certain applications - most notably cypress for flamenco guitars and maple for the more affordable “student” classical models - experimentation with other, alternative non-tropical woods has only ever occurred on a small scale and has not delivered the empirical proof required to indicate that such woods can be regarded as acceptable tonewoods. Until this present study, there has been no reliable, scientific investigation into the suitability of non-tropical species such as alder, ash, oak, plane, birch, false acacia, chestnut etc. for classical guitar building.

 In order for the widespread belief in the tonal superiority of tropical wood in guitar making to be validated, research would be required to prove that, under blind conditions, experienced guitarists and listeners show a clear preference for guitars made from tropical woods, and are able to reliably distinguish them from their non-tropical counterparts. The Leonardo Guitar Research Project made such a study: http://www.leonardo-guitar-research.com/research-report-lgrp. "In order for the widespread belief in the tonal superiority of tropical wood in guitar making to be validated, research would be required to prove that, under blind conditions, experienced guitarists and listeners show a clear preference for guitars made from tropical woods, and are able to reliably distinguish them from their non-tropical counterparts."

The conclusion probably irritates more people than it convinces, "This study, however, shows that first-rate guitar players and experienced listeners alike are unable either to register a clear preference between the two sets of guitars or to distinguish between both wood groups at anything better than chance levels. 

"Furthermore, under non-blind conditions, the non-tropical wood guitars saw, on average, a marked fallback in preference of some 50%. This would indicate that sound perception is strongly influenced by visually transmitted information such as the aesthetic qualities of an instrument, or the preconceptions surrounding good and bad tonewoods that their recognition allows."

You could argue that since this test as conducted on a moderately primitive guitar design, classical guitars, that it doesn't mean much. What it does for me, however, is prove that human hearing is generally as malformed as the schematic diagram and biology indicates. Our ears, unlike other senses, are a cobbled mess of mechanical, hydraulic, and electrical components that are mostly tuned to the human voice frequency range with some consideration for self-defense in the natural world. Many components are so poorly designed that they are easily damaged and age quickly and weirdly. Similar results have been found in ABX tests on electronic audio equipment, disproving some of audiophiles' and professional recording engineers' favorite myths. In the 90's, my old employer (QSC Audio Products) made a device called the "ABX Comparator" which was designed to allow the company's sales force to demonstrate products in fair comparison tests. Mostly, that proved to piss off wanna-be golden ears who were forced to admit they were nothing more than typically hearing-damaged audio professionals.