Jump to content

Blind Testing Report - Lightspeed vs ME24 preamps


Sentient
 Share

Recommended Posts

Gday fellas, on 20th June I held a small GTG, with the purpose of auditioning the Lightspeed passive volume control in comparison to a fully upgraded ME24 preamp.

I admit to being a little bit of a fanboy for ME pre's, as I've grown to really appreciate my ME25 preamp over time. My experience with the Lightspeed was limited (I had it for a couple of weeks before the GTG). Since bias is in play for me personally, I thought it would be worthwhile doing some blind testing for this comparison.

The purpose of this thread, is to provide a report on:

- what we tested

- how we went about blind testing

- what the results of the blind testing was

- and summarise any learning experiences as an outcome of the testing.

I've facilitated blind testing a few times, and every time I do it slightly differently for different reasons. This time is no different, I'm happy for the methodology used to be openly discussed or any short comings pointed out, no worries.

WHAT:

The equipment involved in the testing consisted of.....

* Oppo DV-980 running as a CD transport, coaxial connected to

* CA Dacmagic, 1m interconnects to

* [either of the preamps being tested], 2m interconnects to

* SGR Stereo subs, 80hz active crossover set with high pass output, 1m interconnects to

* SGR-TAG 3 way active speakers

Note: the same cables were used for both preamps, involving a manual cable swap between tests. This was a conscious decision on my part to ensure that the systems were utterly identical in every way.

WHO:

There were 4 x forum members present. GregErskine, Stevenvalve, Gior and myself. For 6 tests, I did the facilitating and the other 3 were the blinded auditioners. And for the final 2 tests, I was blinded and they were facilitating.

There wasnt enough time on the day for everyone to get intimately familiar with the sound of each preamp. We did a little bit of sighted demoing, but it was basically straight into it - cold. Since the visitors were not familiar with the sound of the equipment, I introduced a Sanity Check into the testing.

HOW:

- Setup check involved matching the volumes with an spl meter and test tones - setting the volume to 80db.

- Each auditioner chose 2 tracks to demo. Everyone took a turn in the sweet spot, when their 2 chosen tracks were being run.

- Each track was played 3 times for approx 1 minute.

- Everytime a track was played, the auditioners did not know which preamp was being played. All they knew is, that the first 2 times the track was played it was 1 or the other preamp (order was randomised). Of the first 2 runs, each auditioner was to chose which they preferred (sweep 1 or sweep 2)

- The 3rd time a track was played, the facilitator randomly selected either preamp. The auditioners were tasked with identifying which preamp being used for a second time on the same track. This is the sanity check. If the auditioner has a preference between the first 2 sweeps, can he reliably pick that difference again?

- After every 1min sweep of a track, the system was powered down, and cables were changed. This introduced a 20s-30s delay between sweeps.

- The auditioners were given masks, and kept them on during the 3 sweeps of each track.

- The facilitator covered the equipment, whenever masks were removed, to ensure no clues were given on what had just been run.

- The preamps used in each sweep was not revealed in between each test run (changing of tracks). This was done at the end. The intention was to see if there was any consistency in preferences for the sound of 1 preamp over the other, without knowing which preamp was being used.

- After 3 sweeps of the same track, the facilitator wrote down peoples preferences and their sanity check guess. Then tracks were changed (cd's replaced).

- Seating positions were changed every 2 tracks (ie: every 6 sweeps). Each auditioner had a go in each spot of a 3 seater lounge (ie: sweet spot, left side, right side)

RESULTS: [click the link for the actual results]

http://spreadsheets.google.com/pub?key=0AuN4J3eFcDE1dGJKRndHVUNrWjVIOTlWRWcyZ0RMc1E&authkey=CJjHlbIH&hl=en&output=html

- The sanity check had a sub 50% success rate. No one in the room consistently picked the sanity check correctly every time.

- Preferences were consistently given to the most recent preamp heard. ie: when asked which preamp was preferred, 1 or 2, it was usually the latter selected. Since the order was different and random, this demonstrates to me that preferences were not consistent between preamps. The only thing that was consistent, was the most recent one heard was usually given the preference.

LEARNING EXPERIENCES:

- The first set of sweeps was compromised, as the right channel started distorting on sweep 2. On the ensueing sweeps it became evident it only happened with the ME24, and only when using the 1st set of outputs. Switching to the 2nd set, and the problem didnt re-occur (thankfully, this was a stroke of luck).

- All auditioners were confident that there was definetly an audible difference between the preamps, but they couldnt reliably pick the difference.

- Over a cuppa tea, it was discussed and agreed that our auditory memories are too short, and the delay between sweeps compromised our abilities to reliably pick between the preamps. They were so close, the switchover needed to be immediate.

- Some other system short comings. The subs were not dialled in correctly, and bass was lacking for the demos (I've since resolved this). Early in the afternoon during some room sweeps Gior identified that the right channel appeared slightly softer than the left channel, I later diagnosed this as being related to the right channel fault on the ME24. I dont believe the imbalance was evident during the tests (when running off the good set of outputs on the ME24), but it may have compromised the volume matching process before the testing began??

CONCLUSION:

There is bugger all between the 2 preamps, they were so close that any difference could not be reliably picked. There were system short comings and maybe the methodology used wasnt optimal, affecting the end results. Doing the blind testing differently, on another system, may yield different results. But on this day, with the above approach, using that equipment - the conclusion was the preamps were as good as each other.

In the time since the GTG, after extended listening with various system changes, I feel the Lightspeed is a little more transparent.

---------------------------------------------

All comments and observations on the above are welcome. Dont pull any punches, if you think something was done poorly, or if there could have been a better approach?

Edited by ozmillsy
Link to comment
Share on other sites
  • Replies 40
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Over a cuppa tea, it was discussed and agreed that our auditory memories are too short, and the delay between sweeps compromised our abilities to reliably pick between the preamps. They were so close, the switchover needed to be immediate.

agree.

thanks for the report

Link to comment
Share on other sites

Excellent work, and very interesting result. Will refrain from further comment before we get shunted off to the Great Debate section.

Sounds as though anyone who is interested in either of these preamps should chose on price and age alone (provided no impedance compatability issues).!

Link to comment
Share on other sites
Thank you Ozmillsy. Either the amplifier or review subforum would be more appropriate.

Yeah, I wasnt sure where this one should go. Great Debate, Amp, Review. It doesnt really fit perfectly in any of them - it's not really a review as such. So I put it in the "When it doesn't fit specifically to a forum" section. Happy enough for it to be moved somewhere else though. No worries Keith.

Link to comment
Share on other sites
Excellent work, and very interesting result. Will refrain from further comment before we get shunted off to the Great Debate section.
Oh go on, I like healthy discussion and wont take offense. :)

Sounds as though anyone who is interested in either of these preamps should chose on price and age alone (provided no impedance compatability issues).!

I do feel that both preamps represent great value. It should be noted though, that the ME24 had been fitted with an $800 power supply upgrade. So any purchases in the second hand market need to be tempered with some caution and investigation into current condition and service/upgrade costs.

My own ME25 doesnt have the power supply upgrade, but has been serviced. It sounds pretty close to the ME24 that ZB loaned me. I dont feel compelled to sell it, as it has a very nice MC phono stage that is doing a good job with the P25.

Link to comment
Share on other sites
Oh go on, I like healthy discussion and wont take offense. :)

I do feel that both preamps represent great value. It should be noted though, that the ME24 had been fitted with an $800 power supply upgrade. So any purchases in the second hand market need to be tempered with some caution and investigation into current condition and service/upgrade costs.

My own ME25 doesnt have the power supply upgrade, but has been serviced. It sounds pretty close to the ME24 that ZB loaned me. I dont feel compelled to sell it, as it has a very nice MC phono stage that is doing a good job with the P25.

I think your test was pretty well perfect. This is the point of this type of testing. If differences are consistently detected, you know that you have a valid result. In this case, it revealed that there was not a consistently detectable difference (yes, yes, with this equipment, and these (experienced) listeners). That does not mean, of course, that they both sound the same, that can not be proved by blind testing, only disproved.

However, it also means that in the real world, experienced listeners could not reliably detect a difference.

Link to comment
Share on other sites

Thanks for your time and effort Ozmillsy.

Interesting results.

My take on this is;

The ME with all it's components and complexity sounds as transparent as a simple volume control.

Not only as transparent, but with the added benefit of dynamics and drive ability that only an active pre can have.

cheers, Earle.

Link to comment
Share on other sites

Well written Oz, but it shows a couple of flaws in your test.

Firstly there needs to be more specific targets to the assessment eg bass boom or sibilance or soundstage height or a thousand other markers. Without such focus and using "which one do you like best' type review gives unreliable results which your sanity check shows.

The second flaw is the 30 secs to change cables. Differences are far more obvious when bumped up against each other ie a toggle switch. 10 sec gaps are too long unless you have written down a set of values against each 'marker'.

Sonic memory is very brief.

There are plenty of people here who don't believe in DBTs, not because of what they can do, but because they have never run a successful dbt. It is not an easy thing. The definition of each test needs to be watertight.

I believe that to run a proper dbt that yields solid results (there is a difference or not), one should get in some professional test people who know all the pitfalls, 'preferences' and 'like better' are to be avoided like the plaque on my teeth.

Hope this dbt , hasn't driven you off trying another one....

Link to comment
Share on other sites

CONCLUSION:

There is bugger all between the 2 preamps, they were so close that any difference could not be reliably picked.

It's a great result for the Lightspeed, congratulations George.

For much less than the price of a ME24 (if you can find one), you can have an equally superb Lightspeed Attenuator .

Link to comment
Share on other sites
Well written Oz, but it shows a couple of flaws in your test.

The second flaw is the 30 secs to change cables. Differences are far more obvious when bumped up against each other ie a toggle switch. 10 sec gaps are too long unless you have written down a set of values against each 'marker'.

Sonic memory is very brief.

Yeah but when Oz later did a subjective sighted listening between the two pre-amps did he get to switch between them near instantanouesly?

I assume the anwer is NO but he reported differences between the two of them, one being more "transparent" than the other.

It seems that many hold the view that you need instantaneous switching in a non-sighted level matched listening test, but somehow with subjective listening it's not a problem and doesn't seem to get mentioned. Know what I mean?

Surely if it's problem detecting a difference with one methodolgy, it must be a problem with every other! Just felt I needed to point that out, I feel better now :rolleyes:

Overall I agree with your point but if a difference needs quick switching to be detected, surely it must be subtle at best.

Cheers.

Link to comment
Share on other sites

Instantaneous switching helps, else you need to have a marker or ten, that you can refer to.

There doesn't have to be a difference. No difference is a valid outcome, but sonic memory is short and 30 secs is too long, so often an opinion is accompanied by a lack of certainty. Certainty comes with instantaneous switching.

Not sure what to make of sighted comments, we know that sight absolutely leads people to conclusions which when repeated blind are not repeated therefore inconclusive. Proper testing is performed blind to remove prejudice.

Link to comment
Share on other sites

Not sure what to make of sighted comments, we know that sight absolutely leads people to conclusions which when repeated blind are not repeated therefore inconclusive. Proper testing is performed blind to remove prejudice.

And it's off to the Great Debate section for this thread.....

Link to comment
Share on other sites

Thanks for sharing Oz. I've added it to my list on blind test results on my blog:

http://redspade-audio.blogspot.com/2010/06/blind-test-results.html

Nigel, you make a good point about focus. It's difficult to take everything in critically, we're just not good at it. Our sensory acuity is much better when there is something to carefully focus on. Music tends to flood our sense of hearing. What if testing involved a simple recording of a pen being dropped onto a table, or a piece of paper being scrunched up? Or what if we began with a sighted test and some training to notice specific differences? The tests I've heard about in detail have all failed to show a difference. There is one exception on my blog, but it is set up in such a way that a difference is virtually guaranteed.

By the way, I don't mean any of this as a criticism. Seems to me like it was set up well. I think if you can't pick it in a test like this, then it's not worth worrying about. You can argue that audio memory is short - that always comes up when talking about blind testing. But why is it that audio memory stretches over years as soon as it's a sighted test?! Let's admit it - we all compare things we hear now to what we heard years ago. We don't invalidate what we recall hearing after a few seconds do we?!

The problem with instant switching is that it tends to give emphasis to whatever difference is most dramatic at the moment of switching. It does tend to focus your attention on certain stand out differences and I wonder if this is in fact a form of bias being introduced. I hosted an event many years ago where we had 3 sets of speakers switched instantly with a box. It was interesting and it made differences more obvious, but I didn't find it helped form an opinion. I heard an instant change in tonal balance and that was it. One of the pairs was mine, they were all fairly large TLs with Vifa, Scan Speak and Seas Excel drivers. The comparison baffled me a bit and I couldn't say I had a preference. I think I could have formed a better impression on my own, with selected tracks and a couple of hours with no pressure. With speakers it's different though - it's not about hearing a difference but choosing a preference.

If I compare something sighted, I find that I like a bit of time - at least one song without change, then switch, then switch back. I form an impression, then test it with the switch.

So I tend to think a blind test is a difficult way to evaluate. I'm not surprised about the results - when testing most things apart from speakers, the differences are usually subtle enough to get below the radar. I think some are upset by such results because it shows that what they call a "massive difference" isn't close to massive. You can argue a great deal about if there is no difference at all, or about the test not being perfect but I think what you can't argue with is the obvious conclusion most seem to reach - the differences are usually not massive or huge and one part of an audio system usually doesn't lift any veils unless something was seriously wrong. Of course there will be exceptions, just not as many as we are led to believe.

Link to comment
Share on other sites

Thanks ozmillsy for your time in writing up such a detailed report. Obviously a lot of work went to setting up the test and compiling the results. Guess everyone will make their own interpretation, to my mind with a lot of these tests its the thing with the ears that is subject to error not so much the equipment.

Memory is such a short thing.

Link to comment
Share on other sites

I think i mentioned somewhere that IME it can take several tries over a period of time before you can actually figure out what the actual difference is, and be able to use it for identifying X in an ABX scenario

Can't remember the thread that was in, though, and I'm not going hunting for the thread, or re-hashing the post.

Link to comment
Share on other sites

Just in case my posts seemed critical of Oz's efforts, I should correct that and say that it seems like a pretty good attempt. Many have tried and given up and are now see them as useless. I thing they need to be carefully designed to focus on individual elements, as Paul has also suggested above. I hope Oz tries to do further tests on audio gear.

Link to comment
Share on other sites

Interesting in the results how in 3 places all agreed that the ME was better, although in each of these cases, the ME was the second. In these 3 examples, it was XYX - so they heard Lightspeed, then ME then Lightspeed again. In the second 2 of those tests, 2 out of 3 passed the sanity test.

Link to comment
Share on other sites
Just in case my posts seemed critical of Oz's efforts, I should correct that and say that it seems like a pretty good attempt. Many have tried and given up and are now see them as useless. I thing they need to be carefully designed to focus on individual elements, as Paul has also suggested above. I hope Oz tries to do further tests on audio gear.

Nigel, dont sweat it mate. I'm happy to receive any criticisms of the approach. Your comments align with the observations on the day. Instant switching would definetly have been better.

This is the 4th time I've done blind testing, over 15 years. First time comparing pre's. On all 4 occasions the results were inconclusive. But it could just be flawed methodology at play??

Would I do it again? Sure, but only as an excuse to have a social gtg and abit of fun. I wish I took a photo of my 3 guests on the lounge, blind folded, sitting there with cheesy grins. :rolleyes:

BTW, each auditioner had a writing pad and pen, it was suggested that they could write their thoughts, observations and results down on each sweep. None wrote anything more than their preference, I didnt insist on detailed observations, as I didnt want to detract from the fun.

Link to comment
Share on other sites

From some of these posts, one would conclude that the test had been a failure.

Doesn't seem so to me. It simply led to the conclusion that there was not a big difference between the two preamps in OZmillsy's system.

That's probably true!

Link to comment
Share on other sites
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.




×
×
  • Create New...
To Top