|   Do you really know what DBT means? | okiemax
Nov 9, 2003 11:40 PM | | I thought I did, but am now aware that different people have different interpretations of what "double-blind" means. Even physicians have problems interpreting the meaning according to an article in the Journal of the American Medical Association (JAMA, Vol. 285 No. 15, April 18, 2001).
"Our study has demonstrated enormous ambiguity in the conventional ways of describing blinding. Our results suggest that authors and journal editors should abandon the terms single, double, and triple blind, and substitute descriptions stating which of the relevant groups were unaware of the allocation."
"As long as journal reports of RCT's (randomized clinical trials) include the term "double-blind," clinicians will risk inaccurate references about the validity of the studies on which they base their clinical practice."
I don't think a forum member will be risking anyone's health by misinterpreting what another forum member means by "double-blind" in a sound reproduction context, but the ambiguity does make discussing the subject more difficult. I wouldn't expect members to abandon use of the term "double-blind," but in some instances describing exactly what is meant would be helpful. |
| It means put up or shut up! | skeptic
Nov 10, 2003 4:30 AM | | It means that scientists have to carefully devise a method for testing only the control element they want to examine. It means painstakingly trying to eliminate any clues to those being tested which choices they are experiencing at any given time. It means using enough data and recognized statistical methods for coming to tentative conclusions. It means that the entire process is published in professional journals for others to scrutinize, challenge, and verify by repeating the process themselves. It is not unusual for a procedure to be challenged even years later and find that the original test was flawed making the conclusion invalid. It does not mean one guy swapping wires a few times and another guy saying I like that one better than the other and having guessed right 7 out of 10 times saying aha, it really is better.
Why don't manufacturers of audio cables engage in such DBTs to prove their products' worth? IMO, there are only two possible reasons. First, they know that the outcome would not put their product in a favorable light. Why risk proving that your expensive product is no better or different than someone elses cheap product? Second is that they don't have to. So many people buy it without prooof, that even if it could be proved, there is no reason to go to the trouble and expense. Besides, if it were proved, the outcome might show that only among a certain subcategory of listeners such as those with very sharp accuity or with certain types of sound systems could a difference even be discerned. No need to risk future sales with outcomes like that. |
| You say double, I say single ! | okiemax
Nov 10, 2003 11:07 AM | | I think you have described a single-blind test rather than a double-blind test, as you have not said anything about assuring that investigators will not interact with participants before or during the trial. The biases of investigators can be transferred to participants by their attitudes and behavior.
I doubt that manufacturers of any audio equipment will engage in DBT's to prove their product's worth. From a manufacturer's standpoint, even positive outcomes of such test might be better left undisclosed. For example, what would people think if Kimber announced that a DBT proved listeners could tell the difference between their $300 speaker cable and lamp cord, or Bryston said their test proved listeners could tell the difference between their $2,000 integrated amp and inexpensive recievers?
Come to think of it, do any manufacturer's other than drug companies do blind testing that we know about? I can't remember such a test being used to promote or defend a product, but there must have been some instances of this. |
| You say double, I say single ! | Rockwell
Nov 10, 2003 11:57 AM | | Some food and household product companies use them to sell..."8 out of ten preferred our product over leading brands." |
| You say double, I say single ! | mtrycrafts
Nov 10, 2003 10:25 PM | | b as you have not said anything about assuring that investigators will not interact with participants before or during the trial.
That matters if the investigator knows which product you are testing, product A or B. If he doesn't, the interaction could not give clue which product is being tested at any one time.
In audio, if a computer controls the randomizing of A and B for X, then it is not a problem. |
| You say double, I say single ! | okiemax
Nov 11, 2003 12:51 AM | | By talking to participants before or during the test, investigators can transfer biases to participants, unless steps are taken to prevent this from happening. Why would you object to taking such precautions?
Without some assurance that investigator bias will not affect the outcome of the test, I would call it single-blind, or maybe one and one-half blind since the comparator does prevent some investigator bias. You may still prefer to call it double-blind.
Blinding terminology is not well defined. Differences of opinion on what single, double, and triple blind mean is a problem in medicine, according to the JAMA article. Why wouldn't it be a problem here on the Forum? |
| You say double, I say single ! | mtrycrafts
Nov 12, 2003 12:04 AM | | b Why would you object to taking such precautions?
I wouldn't. What could he transfer if he doesn't know which is being tested at any specific time.
b Why wouldn't it be a problem here on the Forum?
It could be. But if both are blind to know which components is in test, I don't see bias transfered effectively as it will affect both components. |
| You say double, I say single ! | okiemax
Nov 12, 2003 1:37 PM | | I am assuming investigators have verbal and/or written contact with test group participants by giving instructions and answering questions before the test begins. There also might be contact during the test.
If the investigator has a naysayer bias, he may inadvertantly communicate his attitude and expectations to the participants through both verbal and non-verbal means. Consequently, a participant from the general population might feel he is not expected to be able to identify components by sound, and fear displeasing the investigator by scoring well. A participant who is an audiophile yeasayer might feel under pressure to prove himself, which could adversley affect his performance.
You could reduce the possibility of bias by using indepedent investigators who have no position on the question of audible differences between components, or at least insist that other investigators follow strict rules when communicating with participants so as not to reveal their bias. Questionaires for participants following the test should ask if investigator bias was observed or suspected. |
| You say double, I say single ! | mtrycrafts
Nov 12, 2003 11:40 PM | | b A participant who is an audiophile yeasayer might feel under pressure to prove himself, which could adversley affect his performance.
That happens regardless of the administarator. |
| You say double, I say single ! | okiemax
Nov 13, 2003 11:10 AM | | Right, but the pressure to perform is greater if the administrator is a naysayer. I believe a yeasayer would rather fail with an administrator who is impartial than one who is a naysayer. How much pressure a person feels to perform can depend on how badly he will be embarrased by failure. |
| You say double, I say single ! | mtrycrafts
Nov 15, 2003 10:58 PM | | b How much pressure a person feels to perform can depend on how badly he will be embarrased by failure.
Well, I guess any competition must have impartial judges so the participant can not blame the judges for failures. |
| You say double, I say single ! | Norm Strong
Nov 12, 2003 1:31 PM | | The subject always knows what A and B are. He can listen to either as often as he wishes. All he has to do is match up X to either A or B, since he knows it's one or the other. After he makes a tentative match, he can go back and forth between the two as often as needed to assure himself that the two are identical.
The opinions of the proctor are immaterial, since he doesn't know which of the 2 choices is correct either! |
| You missed my point. | okiemax
Nov 12, 2003 1:52 PM | | Please see my 1:37PM reply to mtrycrafts. |
| |
