In its July issue, Consumer Reports surveyed low-priced ($100-$200 per pair) loudspeakers. Highest rating went to Zenith’s model H2000W ($125/pair), the speakers that come with the HR 587 Allegro 2000 compact stereo system. That system has been panned by almost every audio critic and expert — with the exception of Consumer Reports.
Consumer Reports‘ ratings, according to audio experts, are specious on almost every count, from perspective to methodology. First, they argue, the magazine’s definition of high fidelity is altogether too low — rather than proceeding from a state-of-the-art viewpoint, CR is frequently willing to make large sacrifices (in terms of bass response in loudspeakers, for example) on matters the experts consider crucial. Second, audiophiles claim, the selection of products CR tests is not broad enough; among the low-priced speakers, JBL and AR, which are big sellers, were not tested, although such “low-fi” units as Zenith, Realistic, Lafayette and Sylvania were included. There is also the matter of whether price alone is an equitable point of comparison — perhaps power ratings or other characteristics ought to be given priority in grouping items to be surveyed.
But beyond that, the experts allege, Consumer Reports‘ tests are often irrelevant, ignoring crucial aspects of sound in search of a system that will fit everyone’s taste and needs. In short, the audiophile argument concludes, the Consumer Reports people don’t know what they’re talking about.
These criticisms are given even more weight by the fact that Consumer Reports is the mouthpiece of the 200,000-member Consumer Union and has a readership of 2 million, giving the magazine ten times the circulation — and market influence — of any of the specialized audio magazines. A favorable rating in Consumer Reports is almost certain to boost sales. To find out if Consumer Reports‘ surveys are really made up by a gang of curmudgeons with a long-standing grudge against the stereo industry, I visited the Consumer Union’s vine-covered building in semi-industrial Mt. Vernon, New York, one of the least delightful suburbs of Manhattan’s bedroom, Westchester County.
The electronics lab — which measures the noise level of hedge trimmers and automobiles as well as hi-fi and television products — is in a building out back by itself. It could probably be any high-fidelity testing lab. The equipment was what you’d expect: anechoic chamber, listening room, closed-circuit transmitter for testing tuners with in-house FM signals, shops for tearing equipment apart and putting it back together after analysis, oscilloscopes, third-octave analyzers and other items neither you, nor certainly I, comprehend. If there was an oddity, it was the computer that receives, analyzes and prints out data on graph paper. (Sample scores from the July speaker survey: Zenith, 93; EPI 70, 91; Advent 2W, 87; KLH Model 331, 78; Sylvania AS5710, 72.)
Larry Seligson, the electronics division chief, looks like Marvel Comics’ idea of a mad scientist, with his thick glasses, beard and emaciated physique. The audio division’s senior engineer is Alan Lefkow, a studious, altogether normal, compact guy in his early 30s. And the rest of the crew is much closer in style to Lefkow than Seligson. So I ruled the curmudgeon theory out right away.
“We could be called the Great Conservatives,” Seligson said. “We’re interested in what sounds good to a reasonably well-trained, acute listener. We don’t make superfine distinctions, or at least those we do make aren’t heavily weighted.” Lefkow broke in with some thoughts on psycho-acoustics: often, he said, the precise measurements that audiophiles insist upon can’t be heard, according to Consumer Reports‘ studies. “You get impressive measurements, but it’s impossible to resolve that data into meaningfulness.” Play the same scratchy disc for a musicologist and an audiophile, he argued, and one would only hear wonderful music, while the other would respond only to the clicks and pops.
Both men are well aware of audiophiles’ opinions of the Consumer Reports surveys. “I think the search for hi-fi-perfection is like the quest for the Holy Grail,” Seligson said. “If audio buffs were to find the absolute sound, they would be dashed. It would be the worst thing that could happen.” As a result, the Consumer Reports crew doesn’t make judgments for audio buffs but for the masses. The selection process, as it happens, makes certain of this. Products to be reviewed are chosen by general type from reader surveys; the equipment in which readers express the highest interest is surveyed more frequently, although very expensive, esoteric equipment is sometimes included as a state-of-the-art reference point for other tests.
Consumer Reports buys all the equipment it tests on the open market, which immediately and drastically separates it from most audio critics, who receive test products from the manufacturer. (Consumer Reports buys everything it tests this way, including automobiles.) Indeed, the staff lives in fear of someone jimmying with the test. Once, Lefkow and Seligson told me, they bought a receiver with the unusual serial number IX; when tests showed it was brilliant, they panicked, sure that the receiver had been “seeded” by the company in hope of a favorable report. So they bought another–which tested equally well.
The tests are done principally by sophisticated instruments; some years ago (around 1964), the magazine relied more heavily on listening tests, but since then, they feel, they’ve been more successful in quantifying the subjective. Listening material varies: for a recent test, the music used was Beethoven’s Sixth Symphony and Fleetwood Mac. (Because the magazine reviews only classical music, jazz and show tunes, some readers suspect an antirock bias, but that is reflected neither in the staff’s listening taste nor in the lab’s record library, though there’s not much harder rock than Fleetwood Mac.)
The real problem, perhaps, comes in the interpretation of the data. Speakers, for instance, are tested primarily for flatness in the mid-bass to high-treble range. Deep-bass response is reported as a “freedom from bass distortion” comment in the chart accompanying the low-priced speaker review, with rankings from “above average” — the Advent 2W, Heathkit AS 1332, Jensen 21 and the Sylvania AS5710 and Lafayette Criterion #38 — to “below average,” which the Zenith speaker received. The staff contends that differences in deep-bass response are too relative to be crucial. Most people who listen to hard rock, played loudly, would of course disagree. (The listening tests are used as a check on the computer ratings, and, according to the staff, listeners tend to confirm the instruments’ findings.)
There are factors that the system pays almost no attention to – notably compatibility, either with other components or with a specific musical taste — but in general, the trouble seems to be that most audiophiles respond more sharply to the magazine’s graphic presentation of its results than to the several thousand words of commentary which accompany each survey. For the low-priced loudspeakers, for instance, the copy notes that “the five top-rated speakers are essentially equal to each other in accuracy; they would perform very well if their power needs are compatible with your music system’s receiver and the size of your listening room.” Which seems to take care of that aspect.
Consumer Reports does have some very real advantages over the audio experts who write elsewhere. No other widely distributed magazine pans audio products; inferior equipment is almost always left unreviewed. And though manufacturers are free to visit and inspect the CR facilities, there’s a distinct arm’s-length approach to the stereo industry that is noticeably absent among other audio reviewers. (Even a nonpartisan journalist like me couldn’t buy the staff lunch.)
The real question about the Consumer Reports reviews is — or ought to be — how useful are they? And that, too, is highly subjective. If you’re interested in reproducing a specific musical effect, or if you listen only to loud music or have other eccentric listening habits, the reviews probably are as useless as audiophiles say. But if you’re interested in finding out about a fairly broad range of equipment with a wide potential for use, then the CR reviews are probably a valuable supplement to the other hi-fi critics. At the very least, if you find a component rated highly by audiophiles and CR, you can be confident that it not only works well, but might even be a bargain. But in that case, you probably aren’t an audiophile. And that group will continue to curse Seligson, Lefkow and company for failing to hear what they hear.