Let's design a hifi experiment NB No cable argument please!
Discussion
After numerous discussions about cables, CD players, DACs etc I thought it would be interesting to discuss how one would go about designing a double blind experiment to test the effect of changing cables, CD players, and DACs on the sound of a hifi system.
Maybe we could list the design parameters and then talk a bit about statistics (any experts, I hate this topic!) to get over the dreaded sample error problem.
It would be very interesting to have input from both the for and against parties but if we could please avoid actually letting things degenerate into a discussion about the items themselves and whether they make a difference this will avoid derailing the thread.
So, what do we need to do this and how do we go about it?
Maybe we could list the design parameters and then talk a bit about statistics (any experts, I hate this topic!) to get over the dreaded sample error problem.
It would be very interesting to have input from both the for and against parties but if we could please avoid actually letting things degenerate into a discussion about the items themselves and whether they make a difference this will avoid derailing the thread.
So, what do we need to do this and how do we go about it?
So, to start with cables. I think a good principle is KISS with this whole thing.
Two cables, a high budget cable say £20 or £30 a meter and standard 2.5mm2 cable at a few quid a meter.
For the double blind part we need a system of changeover so that the person changing the cable doesn't know which is which.
For the "for" group would it be acceptable to have some sort of chunky, low resistance, double pole rotary switch doing the duties. We could simply mark this as A and B for the two cables?
Another big question I would ask is how many times do we need to do this to avoid sample error and get a meaningfull result? As I said I'm no statitician but in all the experiments I've done before we've spoken about thousands of subjects or samples which may or may not equate to thousands of litening attempts.
I reckon we could cut this down quite a bit and still get a reliable result. Let's say we do 25 tries on each cable so 50 overall? Can any statiticians or anyone in the know comment?
Lord Flathead or Hairy Krishna?
I think a 30sec piece of music should be sufficient each time and of course the same piece should be played each time.
Of course all other pieces of equipment would remain unchanged, only the cables would be changed over-or not!
Otherwise, how would the changeover be decided? Do we need a control cable or not? Would the changeover be random or should a pattern be decided beforehand or could we even rig up some sort of device whereby a button is pushed and a random choice is made (and secretly noted for later consultation).
Any other ideas?
Two cables, a high budget cable say £20 or £30 a meter and standard 2.5mm2 cable at a few quid a meter.
For the double blind part we need a system of changeover so that the person changing the cable doesn't know which is which.
For the "for" group would it be acceptable to have some sort of chunky, low resistance, double pole rotary switch doing the duties. We could simply mark this as A and B for the two cables?
Another big question I would ask is how many times do we need to do this to avoid sample error and get a meaningfull result? As I said I'm no statitician but in all the experiments I've done before we've spoken about thousands of subjects or samples which may or may not equate to thousands of litening attempts.
I reckon we could cut this down quite a bit and still get a reliable result. Let's say we do 25 tries on each cable so 50 overall? Can any statiticians or anyone in the know comment?
Lord Flathead or Hairy Krishna?
I think a 30sec piece of music should be sufficient each time and of course the same piece should be played each time.
Of course all other pieces of equipment would remain unchanged, only the cables would be changed over-or not!
Otherwise, how would the changeover be decided? Do we need a control cable or not? Would the changeover be random or should a pattern be decided beforehand or could we even rig up some sort of device whereby a button is pushed and a random choice is made (and secretly noted for later consultation).
Any other ideas?
I think really it has already been designed for you:
http://en.wikipedia.org/wiki/ABX_test
Even a circuit for it
http://sound.westhost.com/abx-tester.htm
http://en.wikipedia.org/wiki/ABX_test
Even a circuit for it
http://sound.westhost.com/abx-tester.htmI don't get the arguement on cables.
Change the cables, it sounds different. The only people who argue against this are doing so out of blockheadedness, or are listening to s
te (if you're off a low rate mp3 it doesn't matter etc).
Is it worth it? Up to the end user. The law of diminishing returns, combined with the obviously high margins in the product making the cost - value gap bigger means that everyone can find the right level for them on everything, including cables.
What i don't get is why people are so passionate about it. Non belivers (in cables) seem to think slagging off the purchasing decision of others is right. Some of these people doing the slagging off own kef eggs - proof they kow fk all in the first place (see what i did there?).
Tl;dr - does it matter? Buy em if you want, but don't act all high and mighty about it either way.
Change the cables, it sounds different. The only people who argue against this are doing so out of blockheadedness, or are listening to s
te (if you're off a low rate mp3 it doesn't matter etc).Is it worth it? Up to the end user. The law of diminishing returns, combined with the obviously high margins in the product making the cost - value gap bigger means that everyone can find the right level for them on everything, including cables.
What i don't get is why people are so passionate about it. Non belivers (in cables) seem to think slagging off the purchasing decision of others is right. Some of these people doing the slagging off own kef eggs - proof they kow f
k all in the first place (see what i did there?).Tl;dr - does it matter? Buy em if you want, but don't act all high and mighty about it either way.
Some Gump said:
I don't get the arguement on cables.
Change the cables, it sounds different. The only people who argue against this are doing so out of blockheadedness, or are listening to ste (if you're off a low rate mp3 it doesn't matter etc).
Is it worth it? Up to the end user. The law of diminishing returns, combined with the obviously high margins in the product making the cost - value gap bigger means that everyone can find the right level for them on everything, including cables.
What i don't get is why people are so passionate about it. Non belivers (in cables) seem to think slagging off the purchasing decision of others is right. Some of these people doing the slagging off own kef eggs - proof they kow fk all in the first place (see what i did there?).
Tl;dr - does it matter? Buy em if you want, but don't act all high and mighty about it either way.
Can you not read FFS? Christ your user name is accurate.Change the cables, it sounds different. The only people who argue against this are doing so out of blockheadedness, or are listening to s
te (if you're off a low rate mp3 it doesn't matter etc).Is it worth it? Up to the end user. The law of diminishing returns, combined with the obviously high margins in the product making the cost - value gap bigger means that everyone can find the right level for them on everything, including cables.
What i don't get is why people are so passionate about it. Non belivers (in cables) seem to think slagging off the purchasing decision of others is right. Some of these people doing the slagging off own kef eggs - proof they kow f
k all in the first place (see what i did there?).Tl;dr - does it matter? Buy em if you want, but don't act all high and mighty about it either way.
clived said:
I think really it has already been designed for you:
http://en.wikipedia.org/wiki/ABX_test
Even a circuit for ithttp://sound.westhost.com/abx-tester.htm
Excellent link, thanks http://en.wikipedia.org/wiki/ABX_test
Even a circuit for it
http://sound.westhost.com/abx-tester.htm
paul.deitch said:
First you need to standardize/normalize the user ears!
Quite.Driller asserts that "he" cannot hear the difference between different DACS or cables, ergo (using his bizarre logic) there IS no difference between different DACS and cables.
The notion that this could be down to poor hearing, and/or that his system is not revealing enough to demonstrate the difference, seems to have escaped him
He then sets about setting up an experiment to prove that everyone who claims they can hear a difference is wrong - despite the fact they THEY can clearly hear a difference and don't a) need a test as it's obviously Drillers hearing that is the problem, and b) don't care what he thinks anyway - only on Pistonheads
StuH said:
Quite.
Driller asserts that "he" cannot hear the difference between different DACS or cables, ergo (using his bizarre logic) there IS no difference between different DACS and cables.
The notion that this could be down to poor hearing, and/or that his system is not revealing enough to demonstrate the difference, seems to have escaped him
So you figure that by ridiculing my proposition that automatically renders it pointless?Driller asserts that "he" cannot hear the difference between different DACS or cables, ergo (using his bizarre logic) there IS no difference between different DACS and cables.
The notion that this could be down to poor hearing, and/or that his system is not revealing enough to demonstrate the difference, seems to have escaped him
By ridiculing it you are just showing that you would be afraid to take part in it.
Forget me, I'm not the subject of the experiment, your group are as you are the one who reckons he can hear the difference.
StuH said:
He then sets about setting up an experiment to prove that everyone who claims they can hear a difference is wrong - despite the fact they THEY can clearly hear a difference and don't a) need a test as it's obviously Drillers hearing that is the problem, and b) don't care what he thinks anyway - only on Pistonheads
And it always ends this way. ALWAYS. With the pro group refusing to take part because there's no point as they couldn't possibly be wrong.Lets do the experiment Stu. If you prove to me you can hear the difference, I shall come on Pistonheads and beg forgiveness
You're going to say you don't need me to beg forgiveness right?
Meh and indeed meh.
Driller said:
It would be very interesting to have input from both the for and against parties but if we could please avoid actually letting things degenerate into a discussion about the items themselves and whether they make a difference this will avoid derailing the thread.
I'm glad this area has been brought up - back in the epic 'speaker cable' thread probably the most interesting discussion was about the testing - and as you have alluded to it is not just as simple as a switch and letting people listen.While I've got expensive gear, I've not been to proud to sell a £3000 CD transport simply because a £300 Mac sounded exactly the same to me. I spent hours trying to ascertain a difference in *something* but I could not.
As an audio engineer by profession I would say I have pretty good 'raw' hearing skills and my ears are particularly tuned to small changes in music, through recording and particularly mixing.
What I have learnt from that is that it is possible for the ear to hear quite tiny differences. However, the difference the perceptual part of the brain has on that 'raw data' of what is being heard is normally much larger.
For example, in the mixing trade, there is a bit of an 'in the know' joke about the 'Emperors New Clothes'. As the legend goes, the record label A+R guy or annoying bandmember wanting to exert their ego on the mix makes an ill informed comment like it needs a bit more 'slam'/ 'sparkle'/'jam'/'lightness'/'meat' to the mix, or a single instrument.
They go away, and the mix engineers chat amongst themselves. The next day the commenter comes back and listens in, and with the knowledge that the magical advice they have imparted has translated to changes proclaims that it now sounds perfect.
Of course, the mix is exactly the same as it was the previous time they heard it. The 'Emperor' is wearing his 'New Clothes'. If the engineer is being a bit kinder to them, and wants to definitely make sure it is liked, he'll just turn a volume up a notch because then everything subjectively sounds better.
My point relates to the philosophy of the test. Even people with great ears will hear differently from listen to listen. It could also be argued that the perceptual differences between listens imparts such a 'noise' to the data it will be difficult to see the data underneath it. Then the test is not such whether it is possible to hear the difference between X and Y, but whether ABX tests in general can be 'passed' at all. If you have a small data sample, it won't be significant and prone to error, but dozens of listens to the same music would mean you wouldn't 'see the wood for the trees' any more through fatigue.
IMHO from experience, in an ABX situation to positively pass it you'd have to change a component in the system which would impart more of a change than the 'noise' of perception - and that may be surprisingly large.
Driller, I'm with Gump - why bother when the only people who do the purchasing are those who (think they) can hear the difference?
If you set it up, prove to even yourself there's something in it, then what?
Will you go out and recable your system with 10-20% it's value of new cables?
Which ones will you buy - the ones demonstrated to work in your experiment, or something else?
If you can't hear any difference, then be satisfied at least, that you don't continually pour money into wanting the BBD (bigger, better deal).
If your system sounds good to YOUR ears, then that's all that matters, and no amount of anecdotal evidence from others to convince you it may be better will make you change your position, if you don't hear those changes for yourself.
If you set it up, prove to even yourself there's something in it, then what?
Will you go out and recable your system with 10-20% it's value of new cables?
Which ones will you buy - the ones demonstrated to work in your experiment, or something else?
If you can't hear any difference, then be satisfied at least, that you don't continually pour money into wanting the BBD (bigger, better deal).
If your system sounds good to YOUR ears, then that's all that matters, and no amount of anecdotal evidence from others to convince you it may be better will make you change your position, if you don't hear those changes for yourself.
Driller said:
StuH said:
Quite.
Driller asserts that "he" cannot hear the difference between different DACS or cables, ergo (using his bizarre logic) there IS no difference between different DACS and cables.
The notion that this could be down to poor hearing, and/or that his system is not revealing enough to demonstrate the difference, seems to have escaped him
So you figure that by ridiculing my proposition that automatically renders it pointless?Driller asserts that "he" cannot hear the difference between different DACS or cables, ergo (using his bizarre logic) there IS no difference between different DACS and cables.
The notion that this could be down to poor hearing, and/or that his system is not revealing enough to demonstrate the difference, seems to have escaped him
By ridiculing it you are just showing that you would be afraid to take part in it.
Forget me, I'm not the subject of the experiment, your group are as you are the one who reckons he can hear the difference.
StuH said:
He then sets about setting up an experiment to prove that everyone who claims they can hear a difference is wrong - despite the fact they THEY can clearly hear a difference and don't a) need a test as it's obviously Drillers hearing that is the problem, and b) don't care what he thinks anyway - only on Pistonheads
And it always ends this way. ALWAYS. With the pro group refusing to take part because there's no point as they couldn't possibly be wrong.Lets do the experiment Stu. If you prove to me you can hear the difference, I shall come on Pistonheads and beg forgiveness
You're going to say you don't need me to beg forgiveness right?
Meh and indeed meh.
- This is obviously something that bothers you as you have jumped (voluntarily) into the threads where others discuss the benefits in sound quality to be gained by a better quality DAC, as part of some personal crusade on your part to tell us how we are all wrong.To reiterate - I don't CARE that you can't hear the difference, I and others can. I also don't CARE whether you think we are all wrong - that's your prerogative. How about just not posting on threads where these discussions arise and take your soap-box somewhere else
I'm still amazed by these topics every time they arise.
A couple of years ago, I had the bright idea to make my own interconnects with some simple "hookup" wire, this being of a fairly thin nature.
My wife carefully plaited it together, I solder a plug on each end and sat down to listen.
There was a clear difference in the sound, compared to my existing cables (some NVA 'Sound Pipe' teflon cables that I bought off Ebay).
Was the sound of the plaited cables better?
No.
Was the sound worse?
No.
Was if different?
Yes.
On the other hand..
Back in the late 70's I tried some "twin and earth" mains cable in place of my regular speaker cable (2.5mm not 1mm as in the link) - it was a bugger to work with too.
Was the sound of this cable better?
No.
Was the sound worse?
Yes.
Did it sound different?
Very!!!
The mains cable made the midrange very prominent, rendering some types of music unlistenable. This, in a late 70's budget system (JVS JAS-22 amp, some Realistic speakers and a Technics turntable).
So it doesn't matter what level the system is at, differences can be perceived.
I have also tried so called blind tests with both my wife and son (my son is a budding musician and very particular about the sound) and they have been able to pinpoint differences between cd players / dacs / amps / cables etc.
Amazingly, my son appears to be capable of remembering - as he won't hear my system for a few days, and I may have changed something - and the next time he heard it commented on something not being right.
Finally (as a conclusion to this woefully long post - sorry!)..
I have a couple of CDs where the timing goes woefully wrong if the system is not quite right.
One example is the Level 42 album - Running in the family - the track 'Freedom Someday' - during the chorus when they sing "Free-eee--eeedom someday" - that part will sound like a stretched tape if something in the system is not quite right.
ETA: From the same album - "The Sleepwalkers" is another example - where they sing "by the way.." this too can take on a stretched tape type effect when things aren't right. (and yes I'm listening to the album as I write this) :P
A couple of years ago, I had the bright idea to make my own interconnects with some simple "hookup" wire, this being of a fairly thin nature.
My wife carefully plaited it together, I solder a plug on each end and sat down to listen.
There was a clear difference in the sound, compared to my existing cables (some NVA 'Sound Pipe' teflon cables that I bought off Ebay).
Was the sound of the plaited cables better?
No.
Was the sound worse?
No.
Was if different?
Yes.
On the other hand..
Back in the late 70's I tried some "twin and earth" mains cable in place of my regular speaker cable (2.5mm not 1mm as in the link) - it was a bugger to work with too.
Was the sound of this cable better?
No.
Was the sound worse?
Yes.
Did it sound different?
Very!!!
The mains cable made the midrange very prominent, rendering some types of music unlistenable. This, in a late 70's budget system (JVS JAS-22 amp, some Realistic speakers and a Technics turntable).
So it doesn't matter what level the system is at, differences can be perceived.
I have also tried so called blind tests with both my wife and son (my son is a budding musician and very particular about the sound) and they have been able to pinpoint differences between cd players / dacs / amps / cables etc.
Amazingly, my son appears to be capable of remembering - as he won't hear my system for a few days, and I may have changed something - and the next time he heard it commented on something not being right.
Finally (as a conclusion to this woefully long post - sorry!)..
I have a couple of CDs where the timing goes woefully wrong if the system is not quite right.
One example is the Level 42 album - Running in the family - the track 'Freedom Someday' - during the chorus when they sing "Free-eee--eeedom someday" - that part will sound like a stretched tape if something in the system is not quite right.
ETA: From the same album - "The Sleepwalkers" is another example - where they sing "by the way.." this too can take on a stretched tape type effect when things aren't right. (and yes I'm listening to the album as I write this) :P
Edited by TonyRPH on Friday 16th November 12:57
StuH said:
The only ridiculing is coming from you - This is obviously something that bothers you as you have jumped (voluntarily) into the threads where others discuss the benefits in sound quality to be gained by a better quality DAC, as part of some personal crusade on your part to tell us how we are all wrong.
To reiterate - I don't CARE that you can't hear the difference, I and others can. I also don't CARE whether you think we are all wrong - that's your prerogative. How about just not posting on threads where these discussions arise and take your soap-box somewhere else
Stu, - This is obviously something that bothers you as you have jumped (voluntarily) into the threads where others discuss the benefits in sound quality to be gained by a better quality DAC, as part of some personal crusade on your part to tell us how we are all wrong.To reiterate - I don't CARE that you can't hear the difference, I and others can. I also don't CARE whether you think we are all wrong - that's your prerogative. How about just not posting on threads where these discussions arise and take your soap-box somewhere else
This is not something that bothers me, it is something that interests and fascinates me.
Let's be honest, we're all passionate about both music and music technology ("gear") right? In this context, how could looking closely at and examining the effect of combinations of different gear in a scientific way be anything else but intruiging?
Why would you possibly not want to do it?
When I ask this question aloud and someone says no, they don't want to do it, theonly conclusion I can draw is that there must be another reason, another agenda which seems strange to me.
Let me put it another way, let's turn the tables:
Someone has a very nice hifi setup and claims that they can improve the sound of their system by adding a more expensive CD player. I say that I don't really believe this and that this is probably not the case.
This other person then says to me, "tell you what, let's do a test setup and I'll prove to you that it makes a difference".
And then I say "actually no, you're talking rubbish, I'm so sure of myself I don't need to do this test and moreover it doesn't interest me at all", I'm right and your wrong".
What does that answer say about that person's views on hifi? I just don't get it. If it were me, I would reply "yeah, let's do that test, I'll show you!".
I mean, what possible reason could there be to refuse?
In all honesty, no I'm not 100% sure in my mind that these things won't make an audible difference.
There you go, I said it.
My conslusions come from listening to my own systems (studio gear and Rotel, B&W stuff) from reading books and the net, listening to people and some electronics knowledge.
I still reckon there is more chance that I would be right but I would LOVE to be proved wrong and would be very happy to admit whatever truth was shown, not only because I'm passionate about music and music electronics but I also enjoy having a laugh at myself!
I've tried to be as transparent as possible because I don't understand the hostility that this proposition is met with.
From one music/gear lover to another, does this help clarify my POV any?
Edited by Driller on Saturday 17th November 19:16
not sure i get the point of this thread, as others have said, it's up to the individual.
surely the limiting factor of a test is benchmarking and standardising peoples' ears? the music they listen to comes second..cables are well down the order in how an individual perceives hifi quality.
for the record, you can tell the difference with good quaity cables. end of...
surely the limiting factor of a test is benchmarking and standardising peoples' ears? the music they listen to comes second..cables are well down the order in how an individual perceives hifi quality.
for the record, you can tell the difference with good quaity cables. end of...
Gassing Station | Home Cinema & Hi-Fi | Top of Page | What's New | My Stuff