by Michael

A Hybrid Review Approach…

… mixing the subjective and objective…

Essentially it uses a subjective analysis of objective aspects of the sound of a component.

Even more essentially, the listener gives it the old ‘college try’ and guestimates how good a component is in several pre-determined and STANDARDIZED categories.

See, if the categories are well-chosen and are standardized, and a number from 1 to 10, say, is assigned, then components can be compared.

Anyway, in the most recent Spintricity article:

Towards a Hybrid Subjective Objective Review Process

… we attempt to come up with some categories that we, at least, use here [tho in the past we have not been so disciplined as to assign a number to score a component’s performance in each category]

The idea is to create a list that we can print out and then enter or circle a number of something so that the review process can be raised up out of the muck and mire.

in High-end Audio HiFi and Stereo / Philosophy / Reviewers / Reviews / Spintricity 0 comments
 by Michael

Press as the Public Relations Arm of Industry

There are a lot of pieces and parts of the show report – kind of sprinkled in amongst almost 2000 photos – and even I forget where I said what.

But one of the themes this year was influenced by a ‘Cranky Geeks’ episode (online or special TiVo download) that crystallized, for me, the problem with most reviews in our industry. Although Cranky Geeks was talking about the software industry – it applied equally well to ours [and the Washington Press Core as well, but let’s not go there].

That is that the press is serving as the Public Relations arm of the Industry. The corporations. They take press releases [well, in high-end audio, we don’t need no stinking press releases – so our reviewers have to do more work], massage it, and spit it back out as a review.

Thinking about this off and on during the last few weeks as I do the kind of laborious task of juggling CES photos, I think that there are telltale signs of reviews that are really just PR – that really just server as pages for the industry to link to, to serve as an incentive for the industry to donate free equipment to get more of, a safe place to advertise on.

And those signs are that it reads like PR.

PR, like your prototypical salesperson, never, ever, ever say anything bad about what they are selling. Nothing that can even be *slightly* construed as negative about the piece of equipment.

Now reviewers are famous for putting in clues that seem like they maybe might be a hint at what they really think about the component.

But, think about it. All components have a sound and have issues. .

So most reviews are like describing a car crash by saying over and over again how great the car was – and about what songs were playing in the car at the time of the crash.

Anyway, I grew up respecting the press as being honorable, reporting what was really happening, very often DIFFERENT from what the official spin was about the events. Now it is all in support of making more money. Not making waves.

And the only reviewer who I have any confidence in that they actually write real reviews is Mike Fremer.

Sure, he ignores his prejudices, and ignores how badly his poor sad room affects the sound, and ignores the fact that much of his system equipment is flawed and affects what his results are going to be. So he ain’t perfect [and he has some anger management issues – or at least, he should learn to count to ten before posting. I know. I know. Many people want me to count to a billion before posting]. But I think he is the best we got. [Which is saying a lot about the state of the press industry here, huh?]. And I think he has been getting better.

HP hasn’t written a real review for years and years. Art Dudly, Srajan are runners up – and they may be just as good as Mike, but I just don’t read enough of their reviews, especially of equipment that I am familiar with that have easily agreed-upon issues.

As far as the other reviewers go, as far as I have time to read their copy – uh, well, there is really no reason for me to go farther.

I haven’t met Mr. Fremer, nor Art Dudley and have no specific desire to do so [I emailed once or twice with Fremer about 6 years ago]. I have met Srajan at shows when he used to go to shows here in the U.S. [and he is the only member of the press that acknowledges we exist – all the others try hard to ignore us because we encroach on their (abandoned, as this post testifies) domain] but he doesn’t review stuff I am familiar with – and I instead mostly end up reading his editorials.

Anyway, to wrap this up – it is not like reviewers, in general, are any more a**holes than the rest of us – in fact, of the few I see at shows, most of them are just like the rest of us. It is just that I see the responsibilities of being a reporter differently than they do.

in CES 2008 / High-end Audio HiFi and Stereo / Philosophy / Reviewers / Show Reports / Sounding Off 8 comments
 by Michael

Audiophile 101 – Reviewers

Reviews of audio equipment are compromised because reviewers are compromised. They cannot be trusted.

Both print and online magazines are compromised because one never knows if they are writing positive reviews in response to advertising dollars, or trying to solicit new sources of advertising dollars. One thing is proven, that bad reviews chase away advertising dollars.

Online magazines are compromised further by the fact that reviews are ‘linked to’ by the manufacturers of the equipment that was positively reviewed, increasing the magazines popularity with search engines, which attracts more traffic, which allows them to raise their advertising rates.

Reviewers also are compromised by:

1) Having to conform to the stated policies of the magazine they work for

2) If they do not write positive reviews, manufacturers will not want to lend them equipment for the next review

Dealers who write reviews are also comprimised because no one ever knows if they are saying something in order to try generate more sales.

Individuals, which includes reviews at the above mentioned magazines and dealers, are firther compromised because:

1) One doesn’t know if they are an idiot or not

2) One doesn’t know if they are a shill or not [for those that don’t know, and apparaently some do not, a shill is someone who pretends to be an individual but really works for a dealer or manufacturer]

3) One doesn’t know if they are just conforming to the natural human tendancy to praise the equipment they currently own [and disparage that which they no longer own].

4) One doesn’t know if they are just trying to praise, or disparage, a piece equipment because they like, or do not like, its particular manufacturer.

————–

The point is that all reviewers, and therefore all reviews, are compromised.

They can’t be trusted!

Or can they?

What we can trust is that some reviewers care about their reputation. How they see others see them, and want others to see them.

What we have is:

** REPUTATION-BASED TRUTH **

Both institutions (like magazines and dealers) and individuals (reviewers at those magazines) have reputations – good or bad, or just plain weird.

The argument here is that you CAN TRUST People, and Organizations to more or less behave and write reviews in accordance with their view of their reputation, based on how important that rep is to their personal views of themselves, to their personal self-worth.

——

Take, for another example, TV news reporters.

Edward R. Murrow – apparently [sorry, before my time] had a reputation based on his dedication to telling the Truth.

Some popular networks, and their reporters, have a reputation based on the consistant ridiculing of other’s political ideologies. They can be ‘trusted’ to report in a way that always conforms to this reputation they and their organzation have.

Some reporters whole reputation is built around their ability to get the next scoop, the next Big Story, not having anything to do with the truth, necessarily.

——

So, back to audio,

We have some magazines whose reputation is built on all the published reviews being positive (Positive Feedback [see this recent castigation of non-positive reviews], Inner Ear)

We have some whose reputation is closer to that of Murrow, but which is distorted by what they judge to be ‘truth, but in a responsible manner’ (Stereophile, 6moons). [Here we start entering the domain of serious reporting ethics, the necessity of having to report news without ever having ALL the facts – something too serious for this post, or this website].

[The Absolute Sound and HiFi+ seem to have a mix, there being so obvious, to me, reputation associated with the magazines as a whole, except that of this plurality of reviewers with different types of reputations].

Then you got the ‘Malcontents’, as Inner Ear called them this month [are we malcontents? I hope so :-)]. These peoples reputation vary, sometimes being just ways to publically express their need for anger management, or remedial logic 101, classes. Our rep, as I see it, is that we try to shed light on the very high end in a ruthlessly honest, but inclusive, manner – in a way that seeks ways to explain the what, how and why that the high-end is not just some morass of similar sounding components all rated ‘A+’.

You also got your netizens, who consistantly praise their own equipment as being the very, very best the world has ever seen, and disparage everyone else’s as either ‘been there done that’, or ‘being privy to a special network of only the best audiophiles [i.e. not you! :-)], I have heard that your gear sucks in comparision to my gear’. Their reputation, as they see it themselves, is built upon some variation of everyone thinking that they have the best equipment in the world.

—–

The point is, they are all behaving in accordance to what they want their reputation to be.

Some people care about their own reputation. Some not so much. The ones that care the most seem to be the more consistant reviewers: Mike Fremer, J.A., Srajan for examples.

But it is not a given that their reviews are ‘better’, or worse, than that of other reviewers. It is just that some reviews can be trsuted to be written to be within the context of the individual reviewer’s, and their organization’s, reputation.

—–

The final point, finally, is that everything DOES sound good and everything DOES sound bad.

Everything sounds good to reviewers who are not all that critical of each single aspect of the sound something produces, whose rep is based on welcoming nearly all components into the wonderful world of high-end audio.

Everything sounds bad to reviewers whose rep is based on being very critical of the sound a component produces, always comparing it to what it ‘could be’, if someone had just put a little more effort into its design and manufacturing.

in Audiophile 101 / High-end Audio HiFi and Stereo / Reviewers / Reviews / Sounding Off 3 comments