Double Blind Tests- NOthing to see hear!!

[Guest fordgtlover]

Orchestras use blind testing to eliminate gender bias in member selection; and it works. But, many audiophiles still view that blind testing has no place in the world of high-end audio. Noting that the process used in the audition process is simplicity itself. The performers simply play from behind a screen.

 

What's the difference between selecting an orchestra member and selecting speakers or cables?

 

How can blind testing be effective in one area of audio, but not another?

 

 

Goldin, C., & Rouse, C. “Orchestrating impartiality: The impact of ‘blind’ auditions on female musicians.†American Economic Review, 90 (2000): 715-741. A study of leading symphony orchestras that implemented “blind†auditions, whereby candidates played their instruments behind a screen that masked their identity, found that blind auditions accounted for an increase in the percentage of orchestra musicians who were female. In fact, the screen increased– by 50 percent – the probability that a woman would be advanced through certain preliminary audition rounds, and increased by several-fold the likelihood that a woman would be selected for the final audition round.

 

http://www.theguardian.com/women-in-leadership/2013/oct/14/blind-auditions-orchestras-gender-bias

[Newman]

Do you remember the laughable claims how no visual screen can be acoustically transparent? And hence that they cannot be used when comparing hifi gear?

 

Any excuse will do. 

[rocky500]

I think with Audio, the changes are so minute that it is hard to blind test them.

Longer listening sessions for me give me a chance to see if I can hear a difference.

 

The above example in the first post, if you got the same person to audition twice, would you hear a difference?

Edited September 7, 2015 by rocky500

[Tony B]

It's a good example of just how silly this hobby can be.

Sent from my iPhone using Tapatalk

[rocky500]

I had 2 amps here and thought one sounded a lot better. I always seem to get swept away with the music in different listening sessions with this one amp.

I would then swap between them and could not pick a difference in switching them quickly.

 

So used the cheaper amp and my music lost its appeal when I listen to it over the days following. I did not enjoy listening to it.

Do the tests again switching between amps and I can't hear a difference. They sound the same in these tests!

 

Put the better amp back and my listening sessions are so much more relaxed and really enjoyable. Look over to check which amp I am actually listening too and those sessions that are relaxing and enjoyable is the more expensive amp.

Tried this many times over a period as I really wanted to sell the more expensive amp as they sound the same when switching quickly between them!

 

Maybe I am just bad at short listening tests as I have been to others when they can pick a difference and I can't.

[Guest myrantz]

How can blind testing be effective in one area of audio, but not another?

First of all, as your quote says, it's an audition, not a test.

 

If you want to audition your hifi gear this way, go right ahead, nobody questions it remove bias... The problem is it also also adds bias...

 

The problem is misguided people thinking blind tests are scientific... And made worse when poorly constructed ones get parroted like they mean anything... Note the most recent one conducted at Las Vegas, there's actually a person that passed the test but was dismissed as noise... 

 

I'm happy auditioning my decisions without blind testing.. As I am happy to take months and months and months to evaluate stuffs, I don't need to remove gender bias when buying hifi gear. 

[sir sanders zingmore]

First of all, as your quote says, it's an audition, not a test.

If you want to audition your hifi gear this way, go right ahead, nobody questions it remove bias... The problem is it also also adds bias...

The problem is misguided people thinking blind tests are scientific... And made worse when poorly constructed ones get parroted like they mean anything... Note the most recent one conducted at Las Vegas, there's actually a person that passed the test but was dismissed as noise...

I'm happy auditioning my decisions without blind testing.. As I am happy to take months and months and months to evaluate stuffs, I don't need to remove gender bias when buying hifi gear.

What bias does it add?

[Guest myrantz]

What bias does it add?

How about a bias that a blind test removes bias?

The article said they are using the blind testing process to select performers (some use it for the first interview, some used it all the way)... It's for selection, not to prove A is better than B..

In the way orchestras are doing it, it's actually adding in an element of randomness, as the assessors are forced to make a decision with incomplete information, and that incomplete information choice leads to better woman representation. (i.e. previously they have a tendencyto hire male performers, now it's more up to chance.

Not chance? In the article, there is no confidence test to say with 95% confidence that the selection is random(chance) or specific. It did say the woman's likelihood is now 50%, i.e. it's a random guess...

This is totally not what I want in audio, that's why I prefer to take my time instead of dabbling in pseudo science.. It's like the Op don't even understand the article at all.

Note to self: I should stop reading SNA on Monday.. Probably when my patience is at my worst.

[rocky500]

Not chance? In the article, there is no confidence test to say with 95% confidence that the selection is random(chance) or specific. It did say the woman's likelihood is now 50%, i.e. it's a random guess...

If anything the first post confirms that blind listening most probably is just a random guess.

If the same person auditioned 4 times without them knowing, one outcome could most probably be they are picked twice.

[Satanica]

It did say the woman's likelihood is now 50%, i.e. it's a random guess...

 

Huh, I must have missed that. It says:

 

"Even when the screen is only used for the preliminary round, it has a powerful impact; researchers have determined that this step alone makes it 50% more likely that a woman will advance to the finals."

 

50% more likely is not the same as likelihood is now 50% as you wrote.

Edited September 7, 2015 by Satanica

[Newman]

Unless it was originally 0% [emoji41]

[Satanica]

Unless it was originally 0% [emoji41]

 

Yeah well 0% X 150% is still less likely than 50%

Edited September 7, 2015 by Satanica

[Guest myrantz]

If anything the first post confirms that blind listening most probably is just a random guess.

If the same person auditioned 4 times without them knowing, one outcome could most probably be they are picked twice.

In a way... The key is the interview linked above is completely different to what audiophiles want - audition vs testing..

What audiophiles are interested in say is WAV better than FLAC?

The choices one made for this test needs to pass the 95% confidence test to show that it's not chance.. This is the test, audition doesn't do this.

 

Huh, I must have missed that. It says:

 

"Even when the screen is only used for the preliminary round, it has a powerful impact; researchers have determined that this step alone makes it 50% more likely that a woman will advance to the finals."

 

50% more likely is not the same as likelihood is now 50% as you wrote.

That outcome can only come about when the result is known. in this case it's male vs female... So 50% more likely to advance is the same as 50%.. Now if you increases the sexes to 3 (or more), then the 50% no longer means "remove bias", there still is a bias, because given 3 sexes, 50% more likely to advance = high odds.

Putting what you wrote, and if you think the above applies to gear evaluation - you have 50% more likely to make the right decision when evaluating power amp A over power B... Thing is, you have no idea what that right decision is...  So good luck on that.. 

Edited September 7, 2015 by myrantz

[Guest fordgtlover]

How about a bias that a blind test removes bias?

The article said they are using the blind testing process to select performers (some use it for the first interview, some used it all the way)... It's for selection, not to prove A is better than B..

In the way orchestras are doing it, it's actually adding in an element of randomness, as the assessors are forced to make a decision with incomplete information, and that incomplete information choice leads to better woman representation. (i.e. previously they have a tendencyto hire male performers, now it's more up to chance.

Not chance? In the article, there is no confidence test to say with 95% confidence that the selection is random(chance) or specific. It did say the woman's likelihood is now 50%, i.e. it's a random guess...

This is totally not what I want in audio, that's why I prefer to take my time instead of dabbling in pseudo science.. It's like the Op don't even understand the article at all.

Note to self: I should stop reading SNA on Monday.. Probably when my patience is at my worst.

 

Dude, you should really take the time to read and digest things before posting.

 

Edit: I think I misread your post about A/B testing 

Edited September 7, 2015 by fordgtlover

[Guest Sime]

Auditioning for an orchestra compared to listening to different components to hear which sounds better are not even close enough to be compared against.

Two completely different situations.

[Guest fordgtlover]

How about a bias that a blind test removes bias?

The article said they are using the blind testing process to select performers (some use it for the first interview, some used it all the way)... It's for selection, not to prove A is better than B..

In the way orchestras are doing it, it's actually adding in an element of randomness, as the assessors are forced to make a decision with incomplete information, and that incomplete information choice leads to better woman representation. (i.e. previously they have a tendencyto hire male performers, now it's more up to chance.

Not chance? In the article, there is no confidence test to say with 95% confidence that the selection is random(chance) or specific. It did say the woman's likelihood is now 50%, i.e. it's a random guess...

This is totally not what I want in audio, that's why I prefer to take my time instead of dabbling in pseudo science.. It's like the Op don't even understand the article at all.

Note to self: I should stop reading SNA on Monday.. Probably when my patience is at my worst.

 

Don't shoot the messenger. I'm not the researcher or the author. This issue is well documented. To pretend otherwise is simply ridiculous.

 

You can go and argue with the authors of all of the papers if you feel that you are actually right and they are wrong. We'll just wait here for you to get back.

 

So, 'pseudo-science' is the new catch-phrase... Ok, got it

[Guest fordgtlover]

Auditioning for an orchestra compared to listening to different components to hear which sounds better are not even close enough to be compared against.

Two completely different situations.

 

How so Sime?

 

Edit: In both cases are you not listening to determine whether something meets a specified criteria (overt or otherwise)?

 

And, it's more like listening to all available speakers than just A/B'ing between two speakers.

Edited September 7, 2015 by fordgtlover

[Guest myrantz]

Dude, you should really take the time to read and digest things before posting.

 

Edit: I think I misread your post about A/B testing 

Actually I didn't  mis-read that article.. I may have not written the responses in a clear and easy to understand manner.. I am not shooting the author, I am shooting you for mis-respresenting the article.

 

Article show there is a strong suggestion blind testing removes gender bias.. Explain how this is useful for audiophiles? 

 

Edit: Have a read of the comments made by John Atkinson here. Look at what Ars Technica did, and you'd understand JA's comments ... Esp this:  

Just because a test is performed blind does not make it "scientific," as you and Gordon appear to suggest. So the choice is between two forms of flawed testing procedures to support reviews.

Edited September 7, 2015 by myrantz

[Satanica]

That outcome can only come about when the result is known. in this case it's male vs female... So 50% more likely to advance is the same as 50%.. Now if you increases the sexes to 3 (or more), then the 50% no longer means "remove bias", there still is a bias, because given 3 sexes, 50% more likely to advance = high odds.

Putting what you wrote, and if you think the above applies to gear evaluation - you have 50% more likely to make the right decision when evaluating power amp A over power B... Thing is, you have no idea what that right decision is...  So good luck on that.. 

 

This is just rubbish and rambling, as usual, stick to the point.

 

I repeat, what you said was wrong because the article did not say for women likelihood is now 50%

 

"Even when the screen is only used for the preliminary round, it has a powerful impact; researchers have determined that this step alone makes it 50% more likely that a woman will advance to the finals."

[Guest Sime]

I'm not going to even attempt to explain why two musicians can play differently compared to blind testing two interconnects/amps/pre-amps etc, etc.

It's called common sense and commen sense will tell you why there's a difference.

Edited September 7, 2015 by Sime

[rocky500]

Quote from article
"As late as 1970, the top five orchestras in the U.S. had fewer than 5% women. It wasn't until 1980 that any of these top orchestras had 10% female musicians. But by 1997 they were up to 25% and today some of them are well into the 30s."

So 30% are female then if you have a 50% increase then that comes to around 45% are female selected now.
Close enough to 50% as there may actually be more males auditioning. Seems more like a random pick now.

Edited September 7, 2015 by rocky500

[Guest fordgtlover]

Actually I didn't  mis-read that article.. I may have not written the responses in a clear and easy to understand manner.. I am not shooting the author, I am shooting you for mis-respresenting the article.

 

Article show there is a strong suggestion blind testing removes gender bias.. Explain how this is useful for audiophiles? 

 

Edit: Have a read of the comments made by John Atkinson here. Look at what Ars Technica did, and you'd understand JA's comments ... Esp this:  

 

In what way have I misrepresented the article?

 

I posted some information showing that in one form of listening a reduction in bias has been achieved through blind testing. That is what the research found. Yes?

 

And then...

 

I asked two questions:

 

What's the difference between selecting an orchestra member and selecting speakers or cables?

 

How can blind testing be effective in one area of audio, but not another?

 

 

I am not interested in you misdirection. I asked two legitimate questions for discussion.

[Satanica]

Quote from article

"As late as 1970, the top five orchestras in the U.S. had fewer than 5% women. It wasn't until 1980 that any of these top orchestras had 10% female musicians. But by 1997 they were up to 25% and today some of them are well into the 30s."

So 30% are female then if you have a 50% increase then that comes to around 45% are female selected now.

Close enough to 50% as there may actually be more males auditioning. Seems more like a random pick now.

 

Oh my! It couldn't actually be that women can play as good as men could it and therefore get selected somewhat equally. No, that could never happen!

[Guest Sime]

All this is telling me is that the auditioning panels in these situations are heavily biased towards men, and wow, what a shock.

This cannot in any way shape or form be converted to blind testing for audio gear.

Stupid thread/article is stupid.

No offense.

[Guest fordgtlover]

Quote from article

"As late as 1970, the top five orchestras in the U.S. had fewer than 5% women. It wasn't until 1980 that any of these top orchestras had 10% female musicians. But by 1997 they were up to 25% and today some of them are well into the 30s."

So 30% are female then if you have a 50% increase then that comes to around 45% are female selected now.

Close enough to 50% as there may actually be more males auditioning. Seems more like a random pick now.

 

As noted previously in this thread, the research is well documented. The authors can be easily found. As with Myrantz, if you want to discredit the research you should take it up with the authors.