Originally posted by: ValsalvaYourHeartOut
Originally posted by: sxr7171Could you hear a difference even when you were shown the cable being used? Maybe you simply cannot hear the difference between cables blind or not blind.
Maybe I cannot hear a difference between cables because there ARE no differences to hear.
BTW, judging a cable by how it affects a sub isn't a good way to judge a cable. I used to use the cheapest cable to run to my sub back in the days when I used a sub in my system. Cables are best judged by how they affect the mid-bass up. The main reson for this is that the human ear isn't even very good at resolving low bass.
Hey, when cable companies describe dramatic improvements in bass clarity (blah blah), I'd expect to hear a difference. I don't. There aren't any to hear. If the "human ear isn't very good at resolving low bass," then why even spend money on a decent sub? I guarantee that I can hear and FEEL down to 20Hz, and there are definitely differences among the sub's I've listened to...unfortunately, I can't say the same for the cables used to hook them up.
If you can't hear a difference between cables then more power to you, but your words won't change what I heard when I first heard the difference a good speaker made in my system or the difference that spending an extra $60 to trade up to a used pair of decent interconnects made to my system. If you can't hear it, then accpet that you can't hear it. Of course it requires a little bit of open mindedness to actually listen for yourself instead of quoting a couple of ABX studies.
Oh boy, I can easily tell when there's a difference in sound quality...and I can assure you that there wasn't any to be heard. This isn't a matter of "golden ears"...it's not like I'm can't hear the difference between a Bose and a pair of electrostatics. I've compared Monster Cable to the cheap stuff, and I can assure you it makes no difference...if the improvements were as dramatic as they claim, I should have absolutely no problem hearing them...but I don't...cuz there are no differences.
People have done measurements of the resistance, capacitance, and inductance of high-end cables vs. cheap zip-cord...no differences sufficient to explain differences in sound quality. People have done blind AB comparisons between cables and I have yet to see ONE study that shows a single person able to differentiate between expensive and cheap cables.
What you thought you heard is probably the product of the placebo effect....when people expect to hear dramatic differences when they upgrade their cables, they will actually perceive such improvements, regardless of whether or not there really is a difference. The only way to control for the placebo effect is to BLIND your listener. You can argue that the listeners chosen for the blinded AB comparisons are not capable of hearing the subtle differences that exist...this point is always brought up in the debate. The thing people forget is the people selected as listeners are not deaf - they are generally trained audiophiles with extensive experience in listening to high-end systems...and as I pointed out before, if cables truly made a HUGE difference in sound, it should be ABSOLUTELY NO CHALLENGE for these listeners to pick out the expensive cables from the cheap ones...but time and time again, they are not able to.
With no plausible electrical explanation for difference in sound quality (e.g. equivalent resistance/capacitance/inductance) and data that also fails to demonstrate differences in sound quality, any reasonable person would conclude that indeed, high-end cables do NOT improve your sound. Unfortunately, the myth of expensive cables/interconnects has been perpetuated and imprinted into everyone's head that most people think you're crazy if you tell them otherwise...even in the face of overwhelming data. People "choose" to believe that these cables will help their stereo systems...they want to believe it...it's so comforting to imagine the signal from your $2000 Krell amp is travelling through some exotic $200 cable. Too bad the cables don't do squat.
Valsalva
There are differences between subs and it depends mostly on a sub being able to control distortion - mainly upper harmonics. Other factors are the usual such as bass extension, ability to handle transients, good dynamics etc. Bass clarity is actually carried by lower midrange frequencies or even higher. The audibility of bass is actually present in the accurate harmonics of the bass signal. Localization of bass is in due to harmonics as well (which I'm sure you already know or you wouldn't have a sub). So it is the midrange driver that handles such matters as clarity and character of bass (this is all asuming that the sub is of good enough quality not to add it's own distortions and harmonics). This is why I don't believe that spending money on a subwoofer cable is very important.
We don't even know which cables you tested to come to your conclusion. I have heard for myself the difference a good cable can make and I believe that in the least cable geometry makes a difference in sound, because I heard the difference immediately. It didn't take 5 minutes to hear it. I'll admit that when I went from lamp cord to a $50 pair of Kimbers it didn't make a difference at all, but there were other factors involved.
I had an NAD 314 powering a pair of PSB Alphas and at times I'd switch to my NHT Super Ones depending on what I felt like using. The whole setup was on a shelf in a dorm room. When I think back to those days I'd have to say that my system sounded like crap in that room. Then I got an apartment and got good stands for the speakers, I didn't really compare cables at that point (lamp cord would have been my only comparison) but I still don't think those Kimbers were very good.
About a year later I ordered my new speakers with a pair of Analysis Plus cables (oval 12 biwire) that the manufacturer recommended (he gave me a good discount and he uses the same cabling for internal wiring in the speaker). The cables came two days before the speakers so I hooked them up to my NAD and the Super Ones in the meantime. I was honestly skeptical about these cables, I just bought them because they weren't ridiculously expensive and would be fitting of my new (significantly more expensive) speakers. It didn't take me even a couple of seconds to hear the (surprisingly large) difference, and I didn't even care to hear a difference either. I just assumed that my NAD/NHT stuff was too low end for cables to make a difference. It actually made my system a lot more listenable and I really enjoyed it for the next two days.
I also switched recently from the cheapest Audioquest interconnects to a $65 used cable from Australia, the difference wasn't as significant as my experience with my speaker cables but definitely tangible.
Considering the value of my system the amount I have spent on cable is a piddling amount since I still want value for my money, however I have heard the difference and I won't forget it. The whole experience has made me a lot more open minded about things we claim to fully understand but we really don't.
BTW, most ABX studies were done decades ago with a whole different class of equipment and conditions. The studies you linked to often had only one listener in the sample group. I have no doubt that if someone tried to DBT me with the Analysis Plus cable and the Kimber I had, I would easily show that I could distinguish between the two.
For more information read below (all from
www.audioasylum.com and Jon Risch):
See:
http://www.audioasylum.com/forums/prophead/messages/2190.html
and at:
http://www.audioasylum.com/forums/prophead/messages/2579.html
and
http://www.audioasylum.com/forums/prophead/messages/2580.html
Please read them all the way through, or you wll be making comments that are not relevant. I feel that instantaneuos switching, with it's attendant extra wiring and contacts, is basically one way to invalidate such a test. I outline those reasons quite thoroughly in the referenced posts.
Some folks refuse to do a proper DBT, which require a lot of work and effort ot do correctly, and a half-assed DBT is worse than no DBT at all, again for reasons outlined in the referenced posts.
For these gfolks, I would suggest long trm sighted listening, swapping in the cable that is reputed to be the better one and listening to this one for several weeks straight, and THEN swapping in the old or "inferior" cable later. This would be especially important with the two cables you propose. It would also be necessary to assure tha trhe interconnet was nto a limiting factor, so that it did not reduce everything to a lowest common denominator (in other words, become a limiting quality factor).
Jon Risch
DBTs, ABX and the Meaning of Life? Part 1
Talking about Double Blind Tests is worse than discussing politics or religion, and the infamous DBT thread death-spiral is all too familiar to most of us who have perused the various audio message boards or news groups on the Internet.
I personally have been accused of being anti-DBT, because I moderate a message board forum, the Cable Asylum, that has an anti-DBT posting rule. Nothing could be further from the truth.
I have often come out against the unwarranted conclusions that a few certain folks come to over the results of a few certain amateur listening tests, primarily because I am more familiar than most (but not more than jj) with the problems and limitations of such listening tests.
So I am going to discuss some of the various issues and aspects of DBT's, and in this, I am primarily referring to the amateur type of listening tests, not the professionally conducted types that jj and the codec folks do. I am not going to place this disclaimer at the end of every paragraph, so jj, please print out that sentence, and attach manually. If I specifically refer to a professionally conducted test, I will say so, quite clearly.
I also may refer to audio cable testing at times, but really, what I am saying applies to almost all audio component testing, and is relevant and applicable.
Now that that is out of the way, let's get down to brass tacks.
What is valid? That is, what is a valid listening test, or what constitutes a scientifically valid set of data?
The gold standard for many years has been serious studies or papers published in a peer-reviewed professional journal. There are many reasons for this, and I am not going to cover all of them. Suffice it to say, that this kind of presentation allows one to examine all the facts, the procedures, and the data. It provides for the review of the paper and it's contents by peers in the field, and is published where other professionals have access to it and can question it or raise points they feel have been overlooked. Does such publication guarantee that the conclusions reached by the author are pretty solid evidence? No, but it does provide a certain minimal level of information, screening and review that make the data and conclusions useful to a certain point.
DBT test results from certain amateur listening tests get thrown about sometimes as if they were some sort of hard, cold facts; after all, it was "scientifically determined" that such and such was the case, right?
However, when we look at these DBT listening tests more closely, we find that most have not been published in a peer-reviewed professional journal, in fact, not one DBT on audio cables has been published in such a manner. None. Very few listening tests on other audio components, with the exception of codecs, have been so published either. There have been a few landmark studies on speakers, ala Toole, and most people agree that audio loudspeaker systems do sound different, so this is not one of the more controversial components of study.
So why all the noise about DBT's? Where have they been published, and are they valid evidence? Well, for audio cables, only a handful have been published in popular press magazines. Note that this is not the same thing as being published in a professional journal, an editor may or may not have an agenda, no one else may be reviewing the article for accuracy or proper scientific procedures, etc. When I say just a handful, this is literally the case, as there are only about a half dozen (depends on your criteria) on speaker cables, and few on interconnects. For other audio components, there may be a half dozen articles or so. Not all of these came up with null results either, so it would be very hard to come to any sort of real conclusion based on the data from these articles.
What about web sites, message board posts, news group posts? These are what is known as anecdotal data, they usually have not provided all the details of the tests, nor all of the data, nor have they been reviewed by anyone for proper scientific procedures, etc.
The vast majority of listening test accounts are of an anecdotal nature, and not traditionally allowed to be considered as any sort of good scientifically based evidence.
So the very thing that is being argued about, the DBT listening tests, on audio components, are not of a nature that one can say are very useful in terms of truly valid scientific evidence.
So what about these amateur listening tests, these anecdotal web sites, the popular press magazine articles, are they any good to make any judgments from?
One of the great little catch phrases that get used by some folks extolling these amateur DBTs, is that "In 20/25/30 years of testing, no one has found XXX audio component to sound different, under controled conditions, when nothing is broken", etc.
This is meant to sound like DBT tests have put the matter to bed years ago. This sounds all very well and fine, until you realize: what was the SOTA for 25 or 30 years ago? What kind of cables would have been compared 25 or 30 years ago? I can tell you: zip cords against zip cords. Several of the articles commonly referred to by folks citing popular press DBT results, were this old. Some of the articles on CDPs are 17, 14 years old. How far have CDPs come in that length of time? I mean, we are talking about CDPs that probably did not even have 16 or 15 bits of resolution, no dither, multi-stage, multi-opamp analog output flters, etc.
So if you stop to think about how valid, how relevant some of these really old tests are to the current state of audio, including mid-fi, then it becomes clear that some of them are not really of any use for modern audio components.
What about the tests themselves, how were they conducted? Let's look at a typical scenario for one of the more popular testing paradigms of the day: an ABX style listening test. Note that this is not intended to represent ALL such tests, but merely to provide some idea of what went on in many of the amateur listening tests commonly cited.
First, an ABX switchbox was used to connect the two DUT's, or Device Under Test. In most cases, this required additional cables to be used to insert the switchbox into the signal chain, so it could control which unit was being heard at any given time. The extra cables were almost without exception, just zip cords and/or el cheapo IC's. Even when an audio cable was the subject of the test, the extra cable portion was almost always a zip cord or an el cheapo IC. The reasoning here was that both units were subjected to the same conditions, so it shouldn't matter. So much for the weakest link.
For cable tests, this would be a serious limiting factor, as whatever losses or problems the zip cords or cheap ICs had, were now superimposed on the test cables as well. Ironically, since the vast majority of testers did not believe that audio cables had any sonic impact, they created a situation that virtually guaranteed that it would be hard, at best, to hear what was going on.
Then the listener is asked to listen to the test units, and 'familiarize' themselves with the switchbox and listening protocol.
Typically, while the listener was listening, and switching back and forth, the music was allowed to play on. The first portion was the so-called sighted portion of the test, where they knew the identity of each unit (they know which one is A, and which one is B). The listener was often encouraged to switch back and forth during this portion, and to state whether or not they felt they were hearing the same kinds of sonic differences they did under sighted listening without the switchbox. More on this aspect later.
Then after what might have been hundreds of switches back and forth, under what I would call fairly casual conditions, they would enter the forced choice portion of the listening test, and be asked to identify an unknown DUT, presented as X. They still had access to hearing DUT A or B, and still knew what device the A or B unit was, but X was an unknown, and they were asked to make a choice as to whether it was unit A or unit B.
Classically, they were exposed to a total of 16 trials where they had to select what unit they thought X was, and since it was what is known as a forced choice type of situation, even if they were to readily admit, that they did not think they could identify the DUT, or that they had listening fatigue, they still were supposed to make a choice.
Note that each trial could consist of as many switches back and forth from A to B and back again, and to X and back again.
A single listener might only participate in a single run of 16 trials, and there might only be a handful of such listeners.
Once the listening tests were completed, then the test administrator would check the ABX hardware for the accuracy scores of the listener, and check this against a table of probability ratings, to see how much of a probability existed that the listener had actually been identifying the DUT beyond a certain level of sheer chance.
The benchmark for the 16 trials was to get 12 or more correct, this would then establish that the listener had less than a 5% chance of just guessing that many correct. It is what is known as a confidence level of 95%. The criteria for what was considered 'good enough' so as to not be just due to chance, is supposed to be selected before the test, and then adhered to. Other confidence levels could be used, such as 99% (very strict, and usually extremely hard to do in these kinds of tests), or 90%. It should be noted, that for a 95% confidence level, that just conducting 20 runs would typically result in one that appeared to exceeded the 95% confidence level, even if everything was just random choices. So in order to take the test results as a valid positive, one would have to do better than this on the average.
Much was made of these kinds of tests, mainly because of the fact that they were Double Blind, due to the use of the automated switchbox hardware. The test administrator did not know the identity of the X unit until after the test was completed, and therefore, was theoretically incapable of influencing the outcome of the tests.
What were the problems with these early amateur DBTs?
Unfortunately, they were legion.
It was often assumed that since these tests were double blind, that they represented the only 'true' kind of valid listening test available. However, it was often overlooked that the mere fact that any given listening test was DBT, did not guarantee ANYTHING else at all. It could have been the worlds worst listening test ever, and still could have been double blind.
The long open (sighted) initial portion was not really training, nor were they valid controls of the test sensitivity. In my opinion, they were more of a fatigue inducing situation than anything else. The listener seldom got any real training, they were not exposed to the forced choice scenario until it was time to 'perform', and they were not really trained in terms of what kinds of things to listen for, what kinds of things to hone in on, etc.
The music was typically left to play on, and this is a huge error in procedure. In essence, the listener was never comparing the same signal on both DUTs at any given time, in fact, the same signal was NEVER compared, ONLY a different signal was ever compared. This is such a big problem with the procedure, that such listening tests could be summarily dismissed as an invalid attempt based on this alone.
In terms of listening fatigue, the listener was encouraged to switch back and forth as often and as much as they desired, and this often lead inexperienced and untrained listeners to switch back and forth a huge number of times, all the while not really focusing in on the musical presentation that much. Again, with the music playing on, it would be very hard to try and draw any sort of valid choice, and just as hard to hear what the two units were doing even when you knew which one was which.
This combined with the typically open ended initial sighted portion, and the relatively large number of trials, each of which might include dozens or even hundreds of times that the listener switched back and forth betwen the various units, in my opinion was the cause f a lot of listener fatigue, and therefore was also a very significant factor in these kinds of tests coming up with null results.
Then there was the issue of the switchbox itself, and the extra cables, often of a very poor overall quality level. The relays inside the ABX boxes were of various types over the years, the early ones were mercury wetted reed-relays, the later ones were supposedly rheuthenium plated relay contacts. It has been argued that the switchbox was a source of significant degradation of the listening test resolving power, due to the extra cables and contacts involved. The signal was exposed to magnetic fields inside the relay, and had to travel through a lot of extra wiring and contacts compared to a normal direct real-world connection.
Defenders claimed that the ABX switchbox had passed two tests that assured it was transparent, aside from the usual objective measurement standards of THD, noise and the like:
One, it had been tested using yet another ABX switchbox, and the results had turned up as a null.
Two, J. Gordon Holt, the golden-ears of Stereophile fame, was said to have found it to be 'inaudible' during one of his listening sessions once long ago.
Well, I hope that I don't have to explain the fallacy involved with the first assertion, and the second one is ironic, as one of the very things that the ABX folks were against, was the acceptance of any pronouncements from golden-eared reviewers using sighted listening to review audio products. I think it incredible that they wanted to dismiss and discount all the other reviewers, and Mr. Holt as well when he was reviewing audio equipment, but it was OK to accept his pronouncement on THEIR unit as being transparent when using the same methods. Even so, it is a good idea to note that this occurred back in the 80's, so who knows what one would hear using modern high performance audio gear?
Finally, the confidence level chosen, as well as the particular number of trials, created a very high 'bar' to hurtle, the listener had to be really hearing definite things, and would not have been able to easily discern more subtle things to the requirements chosen.
Despite all of this, certain folks try to cite these old DBT tests as definitive evidence of no sonic differences for audio cables, CDPs, power amps, etc. Not only are the previous problems cited good reasons not to do this, even if none of the problems had existed, and all the items objected to been corrected, there would still be a fundamental problem with doing so.
This fundamental problem is the equating of a null result, that is, a listening test result that simply failed to reach the previously defined criteria for a statistically significant result, as a negative result.
If you have a controlled listening test, and it fails to reach the defined level of statistical confidence, then the result is often called a null result, or "accepting the null hypothesis". However, this kind of result really and truly has no other meaning. You can not legitimately equate a null result to a negative.
Some folks have tried to argue that the equating of a null with a negative is legitimate, and even cited a lone book as a reference. However, the vast majority of statistics books, professors, and accepted authorities still maintain that doing so is just not correct.
The primary reason for not doing so was touched on earlier, you can not know how sensitive the listening test set-up is, unless you have performed a control experiment to determine this. Without such a control, a test that has determined how sensitive both the listening test set-up, and the listening subjects are to very subtle sound issues, you can not have any chance of knowing that the listening test was even inherently capable of discerning what was being tested for!
In the ABX style listening tests, the comments by listeners in the sighted portion that they are indeed hearing what they expected to hear is often cited as a sufficient provision of this test sensitivity control information.
However, this is NOT a scientific way to achieve the determination of this control condition. It is another example of the answer begging the question. Just as you can not use the test to test the test, you can not use a sighted portion to verify the performance of the forced choice portion. This is yet another example of the incorrect reasoning used to justify these kinds of listening tests, and how valid they are supposed to be.
Part 2 will cover the inherent problems and flaws with DBT listening tests, even when done impeccably. Part 3 will cover alternate methods and include some comments on doing your own DBT's.
Jon Risch
DBTs, ABX and the Meaning of Life? Part 2
This is as much a continuation of Part 1 as it is a separate Part 2.
I covered some of the problems encountered in the real world amateur DBTs that have been conducted, but what about a well done DBT, how much resolving power can a good DBT reach?
There are several issues here that need to be addressed.
Obviously, the playback system needs to be capable of revealing any sonic differences that are being investigated. This does not necessarily mean that one should use the most expensive, or the most "SOTA" system available. Often, a system the listener/s are intimately familiar with can be more useful than one of theoretically greater resolving power. But even the familiar system needs to be inherently capable of revealing the subtleties of what is being searched for.
In lieu of achieving positive listening test results for the audio component being investigated, one can use other means to determine what the listening system CAN actually detect.
How would this be done? Well, we can use known thresholds of detection for such repeatable and controllable items such as THD, or amplitude changes.
The strong suite of an ABX style test is detecting amplitude changes, and as reported at the ABX website, they were able to detect down to a 0.3 dB overall level change. If I am remembering correctly, jj has posted that he believes that with a really good test, one could get down to detecting a level of about 0.1 dB overall level change.
Or, you could determine what the sensitivity of the listening test set-up and the listeners was, to say, simple harmonic distortion. For instance, it is not too difficult to detect 2% 3rd harmonic distortion of a 1 kHz tone. The threshold is around 0.5% to 0.3%, depending on whose data you believe. So if a particular test came up empty during the audio component testing, and was determined to be unable to detect 1% 3rd HD at 1 kHz, and could only detect 0.5 dB overall level change, then there is some indication of the level of sensitivity of that particular test, and maybe it just wasn't sensitive enough to detect the more subtle aspects of music reproduction. The null results for the audio components may not have had very much meaning.
Note that if a test has been well designed and executed, and has none of the potential flaws or traps one can get into, so as to generate trivially false positives, then consistently achieving statistically significant positives for the audio component test is in and of itself, a form of validation of the sensitivity of the test. False positives can result from something as innocent as a very slight residual hum present only on one of the DUTs, and so, when asked to ID the X choice, the presence or absence of the hum would be a dead giveaway. Obviously, these kinds of problems would have to be addressed fully in order to conduct a valid test.
Failure to properly level match could also create such a problem, and there are other similar kinds of problems that could crop up, and need to be addressed.
All of the above concerns can be dealt with and the problems eliminated, but there is a more fundamental problem with many listening tests. A forced choice type of listening test also has a dichotomy between a casual listening mode and an analytical listening mode.
During an ABX style test, where the listeners were exposed to sighted listening of the components, and then were switched to blind listening, this would have involved a very casual listening atmosphere initially. So when they listen initially, they are listening casually, and without any special focus, perhaps believing they are indeed hearing the nominal differences between components, whether they actually are or not. The fact that they think they hear such differences, or that they claim to hear them, is often used as if it was some sort of major proof that they were listening attentively, when the exact opposite is probably true.
As I have conducted my own such tests, I am familiar with the foibles and problems of such tests. People will NOT listen attentively or focus on the sounds within the music unless you force them to, or direct their attention specifically to do so. Especially people without any experience or training in participating in such types of testing.
Now, when the listening session went from sighted to blind, and the listeners were asked to make a forced choice (identification of X is a forced choice, unless you allow for the option of the listener declaring that he could not reliably detect any differences, or that what he heard was identical. These are two different things BTW), the listening mode switched from casual listening to an analytical
mode. In order to decide and MAKE A DECISION that X is A, or X is B, or even that A and B are the same/can not tell, one must become analytical.
Instead of listening casually and in what some would refer to as an emotional listening mode (left brain activity), the listener is now asked to listen in analytical mode (right brain activity). The situation has changed, the listening mode is no longer the same, and all the problems and pressures of forced choices rear their ugly head.
Oh, you can say that the listener can take all the time he wants, or that there is no pressure, etc. These are all empty claims if you haven't been in the hot seat.
Facebook
Twitter