Quack Busters

bufoTo start with, let me give you an interesting link to take a look at: www.quackwatch.com.

This is a fairly interesting site managed by people who want you to believe I am a quack. Their articles are quite convincing. In fact, I read several of their articles and found that I agree with most of their statements. However, when it came to their article on homeopathy I found it to be quite misleading.

The easiest way to pooh-pooh something is to tell mostly the truth about it, but to also introduce some seemingly minor wrong facts which in the end totally negate all of the truth that was said. Nonetheless, the reader minimally knowledgeable about the subject will see most of the true facts, recognize them as truth and be impressed. They will usually miss the small lies and totally fall under the power of the flawed argument. A good dose of ridicule also helps. Once the reader is on board with the flawed argument, they will laugh along with the author of the deception, thereby cementing the delusion introduced by the author. Very cunning…

This is what Dr. Barrett did with homeopathy. He gives accurate history, theory and scientific facts, then tells some small untruths about research and makes a completely incorrect conclusion.

I wrote to Dr. Barrett the following message:

I looked through your site. Most of the info you have is quite valuable and I tell my patients the same things. Unfortunately, some of it must be wrong and I would have a very hard time figuring out what exactly is wrong not being educated in most of the subjects you discuss. The reason I am saying this is because while your article on homeopathy is mostly factually correct, its conclusion is totally wrong.

arsenicI can say this because I am an expert on the subject. I am also a board certified internist. In fact, I teach conventional medicine to lay homeopaths. Therefore, I like to think of myself as one who really understands both sides of this particular issue. Check out my site at www.bookonhealing.com. Here you will find my whole book, my CV and just about anything about myself that you would ever want.

In a few days, I will hopefully have a place to put links on my site, then I intend to link to your site. The purpose of the link will be to demonstrate to general public how you and your colleagues in quack busting community operate: present mostly correct information with only seemingly minor misstatements followed by a completely wrong conclusion, thereby misinforming the public. We could have a lively debate. I am a scientist, and you will not have such an easy time proving me wrong as you would people who don’t understand logic. Are you up to it?

I will not call you names, I will not accuse you of being in collusion with the AMA or the drug companies (in fact I have very little against either). My only goal is to provide information to people and allow them to make an informed decision. If you want the same, make some space on your site and let’s have a debate. This letter and your reply will be posted on my site. —Jacob

I did not get any answer for about two years. Then I got a few emails from some people from Quackwatch’s bulletin board participants. At first there were some questions about Homeopathy, particularly how it can be proven. I explained to them that nobody knows how it works but there are studies to prove that it does. This opened a barrage of abusive, insulting emails calling me a quack and demanding proof. It appeared that many participants of their bulletin board were involved. These emails were very emotional in nature and I had a hard time introducing logic into the opconversation. Finally I said that if they want to discuss science I would be more than happy to do so and that we should discuss a particular article from a proper medical journal that seems to prove that homeopathy works. I offered to fax the article to one of the people who seemed to have some logic left in him and then discuss it on line. He agreed and I promptly faxed the article on the use of Homeopathy in influenza.

Following this, all communication from the quackwatch list ceased. I tried to contact the people I was corresponding with, but none of them responded to my messages. This was rather interesting. I can’t believe all of them suddenly decided to not talk to me. I think they are all a single person trying to make it look like there is a discussion going on. My God! The lengths these people will go to to try to discredit something they hate for some reason!

Several years after writing the above I discovered this website devoted to discrediting the “quackbusters”. It is a must see: www.quackpotwatch.org.

Update 7-11-09 We have some new communication with the quack buster community. I found The Skeptic’s Dictionary by Robert Carroll by searching Google for “homeopathy”. So I wrote to him, he replied, and we’ve been having some debate-like discourse. He placed it on his site: http://www.skepdic.com/comments/homeocom2.html.

Dr. Carroll (PhD) is a retired professor of philosophy from a community college in Sacramento, CA. He published his site in 1994. Philosophy comes in handy when one needs to argue any subject, especially one does not have a good understanding about. Philosophers are great with words. Try to argue with a philosopher… They’ll take you for a ride every time.

I’d like to reproduce this discussion here and highlight the tactics used by Dr. Carroll and this community. My later comments are inserted in pink. Here we go:

reader comments: homeopathy

24 June 2009
I am an expert on homeopathy. Your article has some correct information but the conclusions are based entirely on emotional feelings. One needs to look at real data. I offered to one of the quack busters a few years ago to debate a good study openly online, but he disappeared. This tells me his whole campaign was deceptive. I offer the same to you.

This is my experience w/ “Dr” Barrett: http://bookonhealing.com/education/quack-busters.html

If you are open to a proper scientific discourse while putting emotional conclusions aside, let me know.

Jacob Mirman, MD

reply: You make me sound like some southern governor who tried to help a woman keep her marriage together only to fall in love with her. Do my conclusions really seem to you to be “based entirely on emotional feelings”? I’m ready to engage in discourse, but I suspect anything I say will appear “emotional” to you because you may be one of those bright fellows who thinks a conclusion is based either on evidence or emotion. Those conclusions that you agree with are based on evidence and the ones you reject are therefore based on emotions.

Dr. Mirman: Wrong. My conclusions are always based on evidence. The evidence may be in the form of a published study or experience, either personal or shared in a community. I do not reject data I have observed just because it seems improbable. I first acknowledge it, then try to study it and only then pass judgement once I have enough information. This is the scientific way, as opposed to the philosophical way, when one can argue anything based on nothing, like, for example, how many demons would fit on a tip of a pin… My personal policy is not to pass judgement on anything I have not personally examined to my satisfaction and found it to be true or untrue.

Anyway, if you promise to provide specific criticisms of my arguments in our discourse and refrain from simply accusing me of being emotional, I’m willing to accept your proposal….with one condition: you can choose the scientific study we discuss, but it must be a large-scale, randomized, controlled study done within the last 10 years. I don’t want to deal with small studies, studies that aren’t randomized, or those that don’t use control groups. Nor do I want to deal with some study done 50 years ago, where the documentation might be too scanty to allow for proper evaluation of the study’s methodology and protocols.lachesis

To help you understand what kind of control group I require, I have posted a chapter from a book I am writing for you to read. I try to explain where I went wrong in my critiques of acupuncture, homeopathy, and other so-called “alternative” medical treatments.

I assume you have read enough of my work to know that I agree that homeopathy works as well as any other placebo treatment. Any meaningful controlled study will have to be able to tease out effects from the homeopathic medicine from those due to (a short list of) placebo and non-placebo effects.

I hope you have read Bausell’s book Snake Oil Science or my review of the book. I think he has done a great service in clarifying how medical treatment is much more than the pill we take for specific problems. We still have a long way to go toward understanding how inert substances or magical rituals can affect healing.

Dr. Mirman: I agree with you here. Placebo works great. In most drug studies the difference between the active drug and placebo is fairly minimal. For example, the effect of antidepressants in many studies is only 7-10% better than placebo. Read more on this here. Many of my patients tell me they feel better after they see me, even if I did nothing other than provide a compassionate ear. The effect of homeopathy goes way beyond that, including great effect on children and animals, including herd animals with the remedy put in their water trough.

I look forward to an intellectually challenging and fruitful discourse.

Dr. Mirman replies:

I like it.

Take a look at this study: http://www.flusolution.net/bjcp.pdf

Read the Lancet editorial about it: http://bookonhealing.com/appendix.html

Jacob

reply: For those who are following this discussion, the study Dr. Mirman cites was published in the British Journal of Clinical Pharmacology in 1989: “A Controlled Evaluation of Homeopathic Preparation in the Treatment of Influenza-like Syndromes” by J. P. Ferley et al. Even though I made a condition of engaging in this discourse that the study be no more than ten years old, I’ll let this requirement pass for reasons that will become apparent.

Below is Bob Carroll’s evaluation of Ferley’s paper:

In the introduction to the study, the authors note that there are two schools of thought about testing homeopathic potions. One school thinks that homeopathy can’t be tested by randomized, double-blind, control group studies because each treatment is adapted to the specific conditions of the patient and no two patients are alike. Obviously, the authors belong to the other school, which holds that some homeopathic medicines treat symptoms, and they can be tested just like any other medicine that treats symptoms.

Dr. M: This is not entirely correct. One can prescribe individually, then randomize and blind. This way only half the study subjects will get an active remedy, the other half gets placebo. The active group does get individualized remedy. Here is a study like this. These studies are very difficult to carry out. The labor of individualized prescribing is huge, and no serious homeopath wants to see their patient get placebo after so much work being put in.

I thank Dr. Mirman for selecting a short article to discuss (only 7 pages long).

I was impressed that the article was published in the British Journal of Clinical Pharmacology, even though I had never heard of that journal before now. Anything with “British Journal” in it sounds authoritative to me. I realize that my being impressed by this is rather childish. Anyway, I discovered that the BJCP is the sister journal of the British Journal of Pharmacology. Both journals are published by the British Pharmacological Society. So far, so good.

I must admit that the title of the article did not go down well, however, because it specifies that the focus of the research was on treating “influenza-like syndromes.” Such a focus could be an open door to vagueness in evaluation criteria. However, I was pleased to discover that the study would focus on at least one objectively measurable symptom: rectal temperature. A reading of 39 °C (101 °F) was required to be admitted into the study and 37.5 °C (99.5 °F) was required to be labeled “recovered.”

lyssinExcept for the first measurement of rectal temperature, which was done in a clinical setting, the researchers allowed the subjects to do their own testing and recording. They were instructed to do so each morning and evening. Likewise for the five doses of “medicine” each was given. The “active drug” and the placebo were similar in appearance and were administered sublingually. Again, the first dose was taken in the clinic, while the rest were self-administered each morning and evening. The reliability of the compliance and measurements could be called into question. I understand, however, that having a subject come in twice a day to have someone put a thermometer up his butt is likely to discourage all but the most devoted advocates of scientific testing of hypotheses from participating, unless they were paid handsomely to do so. Most researchers don’t have vast sums of money to give to volunteers, so relying on the subjects to do as instructed is somewhat of a necessity in these kinds of trials. I also understand that no study is perfect and that being overly demanding regarding measurements and the collection of data would be unfair. Even so, these problems shouldn’t be ignored, especially if the results indicate a very non-robust conclusion. In such cases, just a few participants in either group going the other way might evaporate any statistical significance measured.

I am assuming that the authors are using ‘syndromes’ synonymously with ‘symptoms.’ In the U.S., we use ‘syndrome’ to mean a set of symptoms.

Many people don’t know whether they have the flu, a cold, a sinus infection, or hay fever because some of the symptoms are the same. The researchers followed reasonable protocols to ensure that their subjects had the flu and not something else. The other evaluative criteria used to determine eligibility for the study were subjective. One list of symptoms was designated “the five cardinal symptoms”: headache, stiffness, lumbar pain [backache], articular pain [aching joints], and shivers. They also collected data on cough, fatigue, and coryza. (I have to admit, I had to look up the last symptom. According to Wikipedia, coryza refers to “the inflammation of the mucus membranes lining the nasal cavity which usually gives rise to the symptoms of nasal congestion and loss of smell, among other symptoms.” Stuffy nose is how we describe it in California.) In any case, the researchers focused on fever (as measured by rectal temperature) and the five cardinal symptoms, which, in my experience are more likely to be found with flu than with, say, a cold. To be eligible, a patient had to have two of the five cardinal symptoms at the time of visiting the clinic for the first time. To be declared “recovered,” none of the five cardinal symptoms could be present.

We know from experience, of course, that in the case of a mild flu most patients should recover on their own in about a week no matter what drugs they take or even if they don’t take any medicine. Of course, some people recover more quickly than others. Generally speaking, young healthy folks will recover more quickly than old feeble folks. Thus, for such a study where it is known both that the illness usually resolves itself in a week to ten days and its duration is likely to be significantly affected by age, general health, and severity of the illness, special care must be taken in the randomization process that determines which group each subject is assigned to.

The researchers addressed the problem of studying something that naturally resolves itself rather quickly by requiring that to be admitted a patient had to have manifested his symptoms within 24 hours of coming to the clinic for the first time and “recovery” would be measured 48 hours after admission.

The researchers addressed the problems of age, general health, and severity of illness at admission in several ways. For example, anyone who had had a flu shot or was under treatment for an immune disorder was ineligible. The researchers used 30-years-old as the dividing line between young and old. (A minimum age of 12 was used as an eligibility requirement.) An initial rectal temperature greater than 39 °C and having at least three of the cardinal symptoms meant one had a “severe” case of the flu; otherwise, one had a mild to moderate case.

Part of their data analysis included looking at how those in each group were divided up during the randomization process according to age, sex, temperature at admission, and severity of illness at admission. (Two other items were included in this analysis, but I’m not sure I understand them. One involved when the participants were included during the flu season and the other involved a delay of less than 12 hours before inclusion.)

The number of eligible cases in the “active drug” group is listed at 237, but the number of males in that group is listed at 93 and the number of females at 127 (total=220, not 237). The number of males and females in the placebo group adds up to 226, but 241 is given as the number in the group. The average temperature (in °C) at intake for the “active drug” group was 38.9, while it was 38.8 for the placebo group, so the groups were nearly identical in terms of degree of fever. The groups were also nearly identical in terms of severity of illness. Average age was 33.7 in the “active drug” group and 35.1 in the placebo group.

General practitioners in the Rhône-Alpes region in France, most of whom were not homeopaths, recruited patients who presented themselves with flu-like symptoms during the 1986-1987 flu season. The participants were assigned to the “active drug” or placebo group by a randomized, double-blind method. Each participating physician was given enough coded boxes of “medicine” to treat four to six patients. Only after the data was collected were the codes unblinded.

No mention is made of what instructions the physicians were given regarding how to go about recruiting patients to enroll in the study or what to say when they administered the dose. We don’t even know if the subjects knew they might be getting a homeopathic treatment. What they were told could have affected how they reacted. Were they all given the same information? We don’t know.

As noted above, no study is perfect, but if I were to do a follow-up study I would instruct the physicians who do the recruiting to try to recruit an equal number of men and women. I would have coded one of the substances with even numbers and one with odd numbers, and asked the physicians to try to give half of each type to those under 30 and half to those over thirty. Before analyzing the data and before unblinding all the data, I would determine that there was not an overload of old or young people in either group. If there were, I would disqualify some of them from the study to make sure that I wasn’t measuring the effect of age on the outcome. After I was confident that age was not going to be an artifact I would be measuring, I would unblind all the oscillopackdata. (I would make every effort to make sure I was not biased in my selection of those to be disqualified because of age.)

In the study, 237 received Oscillococcinum (oscillo), a homeopathic product from Boiron Laboratories in France. (Yes, this is the same company that supported Jacques Benveniste.)

Dr. M. Now we are beginning to introduce doubt and ridicule. Dr Benveniste ran some in-vitro studies a while back, which were later “debunked” by “amazing Randy”, a magitian with inflated ego and a delusion he knows truth from untruth. I don’t really know what happened there, after several labs confirmed Dr. Benveniste’s results. Maybe he did lie, which I find unlikely. Maybe some magic trick was involved in the debunking process. Who knows… But the mention of this study in this context is used specifically to sow the seeds of doubt. Cunning…

The product is made from a combination of wild duck heart and duck liver. It sells for about the price of a flu shot and is a $15 million-a-year business in the U.S., but is much more popular in Europe.* The active ingredients listed on the packaging of oscillo are written in Latin and mean extract of the liver and heart of a Barbary duck: Anas Barbariae Hepatis et Cordis Extractum. It is diluted one part in a hundred 200 times, which means there are no duck molecules left in the final product. Bob Park calculates that the last duck molecule is vanquished after about 12 dilutions. If the entire universe consisted of nothing but oscillo, there would be no molecules of the stuff left after about 40C.* Or something like that. You get the picture. One duck could produce enough oscillo to supply everyone on earth and all their offspring for millions of years with a lifetime supply. Please, no quack jokes, but now you understand why I put “active drug” in scare quotes.

Dr. M: More ridicule… Actually, we don’t need a Bob Park to calculate this for us. This remedy is a submolecular preparation culexbecause it was diluted beyond Avogadro’s number. Chemistry 101. This means there are no remaining traces of any duck tissues in it. All homeopaths would agree.

I know that most skeptics stop right here and say that homeopathic potions can’t work because there’s nothing in them. (Bob Park, for example, simply asserts that no homeopathic remedy has ever passed the test of a randomized, placebo-controlled, double-blind trial. Park doesn’t examine any studies, though he did try for five years to get Jacques Benveniste to collaborate with him on a simple test.*) I’m not going to stop here, however. I’m going to look at the study and examine the methods, protocols, size of samples, use of controls, randomization, clear and specific starting points and ending points, justification of conclusions, and the like. I’m not going to worry about the fact that there’s no active ingredient in the homeopathic potion we are testing.

But first, a little history.

The word Oscillococcinum was coined in 1925 by the French physician Joseph Roy (1891-1978) who said he had observed an oscillating bacterium in the blood of flu victims. He searched for the bacterium in several animals until he found it on the liver of a duckling.* (Dr. Mirman supplies more information below as to why the innards of ducks might be a good place to develop an anti-flu potion: migrating ducks are natural reservoirs of influenza viruses.) Okay. That’s enough history.

Bob Park has a chapter in Superstition: Belief in the Age of Science called “The Barbary Duck,” in which he discusses this very popular potion sold worldwide as something that will relieve influenza-like symptoms. (For those who are into diluted details, consider the following: “In some regions the name Barbary Duck is used for domesticated and “Muscovy Duck” for wild birds; in other places “Barbary Duck” refers specifically to the dressed carcass, while “Muscovy Duck” applies to living C. moschata, regardless of whether they are wild or domesticated. In general, “Barbary Duck” is the usual term for C. moschata in a culinary context.) You can buy this product in America off the shelf. It is not regulated by the FDA. In fact, no homeopathic remedy is regulated by the FDA because the 1938 Food, Drug, and Cosmetics Act specifically excludes most homeopathic potions from FDA oversight. note

In the Ferley study, there were 478 participants, this study is certainly large enough. Nine participants were excluded because they did not meet the eligibility criteria: 5 from the homeopathic group and 4 from the placebo group. Why they were admitted in the first place isn’t discussed.

As noted above, the study used a rectal temperature of 39 °C (101 °F) to be eligible and 37.5 °C (99.5 °F) as an objective measurement of recovery. According to Wikipedia:

Although the value 37.0 °C (98.6 °F) is the commonly accepted average core body temperature, the value of 36.8 °C ±0.7 °C, or 98.2 °F ±1.3 °F is an average oral (mouth under the tongue) measurement. Rectal measurements … are typically slightly higher….

Body temperature normally fluctuates over the day, with the lowest levels around 4 a.m. and the highest in the late afternoon, between 4:00 and 6:00 p.m. (assuming the person sleeps at night and stays awake during the day). Therefore, an oral temperature of 37.2 °C (99.0 °F) would, strictly speaking, be normal in the afternoon but not in the morning. An individual’s body temperature typically changes by about 0.5 °C (0.9 °F) between its highest and lowest points each day.

Thus, some might quibble about the use of temperature to determine eligibility and recovery. The participants would have visited the clinic for the first time at various times during the day, but ideally they should all have had their readings at the same time of day. Likewise for the readings they did at home in the morning and evening. Depending on what time of the morning or evening, temperatures could fluctuate and skew the outcome. It seems that if temperature typically changes by about 0.5 °C between its high and low points each day, some participants might be designated as recovered and some as not recovered because of the time of day they took their temperatures. It might be more appropriate to use a range of temperatures as indicating recovered: for example, say, 37.25 °C to 37.75 °C (99.05 °F to 99.95 °F). In my view, however, I think that with the size of these samples, it is likely that any bias in the groups due to variations in temperatures during the day would cancel each other out.

So, what did the research show?

The data show that the vast majority of participants (86.4%) did not recover in 48 hours. In the oscillo group, there were 39 (17.1%) recoveries and in the placebo group there were 24 (10.3%).

Not surprisingly, the researchers found that younger people and those with less severe symptoms recovered more quickly than older people or those with severe symptoms. Two unusual stats stand out, however. The number of recoveries among those under 30 was three times higher in the oscillo group (21 to 6) and the number of recoveries in the mild-to-moderate severity group was twice as high in the oscillo group (31 to 16). Such differences are not likely due to chance. There was no significant difference in the two over-30 groups.

Dr. M: this point is very important and follows perfectly from our understanding of why this remedy would be useful in flues. I am explaining it below. Note how Dr. Carroll treats my explanation.

There was almost no difference in the numbers in each of the 12-29 age groups that had not recovered within 48 hours. Sixty-three of the oscillo group were still sick after 48 hours, while 68 of the placebo group hadn’t recovered yet. For some reason, the numbers don’t add up in the chart provided by the authors for recovery according to age and treatment group. The number in the oscillo group has dropped to 207 and the number in the placebo group has dropped to 205. The authors note that there was a mild flu that season and don’t indicate that anyone in the study died, but the data for 21 in the oscillo group and 29 in the placebo group are missing.

The researchers established to their satisfaction that even though the participants weren’t supervised during the study, neither group was taking more antibiotics or cold medicines. They did find that the control group took more drugs for pain and fever (50.2% to 40.7%).

Finally, 61.1% of the oscillo group thought the treatment worked, while 49.3% of those taking a placebo thought it was effective. It is not specified by the authors, but it seems reasonable to assume that the participants gave this assessment after the study had concluded. The patients were asked to keep records for one week. Most flu symptoms during a mild flu will last about a week, so the felt recovery may have had nothing to do with actual effectiveness. Anyway, finding out that more than 60% of a group felt a medicine was effective sounds impressive until you find out that 50% who are given a placebo agree with them. Subjective measurements are interesting, of course, but they don’t tell us anything about the effectiveness of the medicine. At least in my book, patient satisfaction is not equivalent to drug efficacy.

anhaloniumA statistical formula shows that the odds of the 6.8% difference in recovery being due to chance are 3 in 100. However, if about 1.7 percent of those given oscillo went the other way and didn’t recover, and about 1.7 percent more of those given the placebo had recovered, we wouldn’t be evaluating this study. All it would take would be a different outcome for 3 or 4 in each group to make it obvious that nothing unusual is going on here. For example, if the number of recoveries in the oscillo group had been 35 instead of 39, and the number of recoveries in the placebo group had been 28 instead of 24, the percentage difference between the two groups would drop to 3.4% (15.4% and 12%, respectively), a difference you might expect between any two groups of several hundred people with flu symptoms, both of which were given placebos.

DR. M: Let’s stick to science, rather than “what ifs”. Statistics take into account all variables and the result of any statistical analysis of a medical study is expressed in “p-value”. The smaller the p-value, the more likely that the differences between the study groups are not due to chance. P-value of 0.05 or below is considered to be the cut off for statistical significance. This means that if p-value is 0.05 or less, the observed differences between groups are probably not due to chance. This is industry standard. Yes, if 3 or 4 people responded differently, we might not have seen what we have seen. This is accounted for in the p-value. The p-value in this study is 0.03, which is statistically significant. If the results are corrected for age and only younger people are considered, p-value is less than 0.01, which is even more significant. Any talk about what if is quite immaterial. This is why the statistical analysis is done.

The authors conclude their summary of the study by commenting: “The result cannot be explained given our present state of knowledge, but it calls for further rigorously designed clinical studies.” I’m surprised that a peer-reviewed journal would allow such a statement to preface a published study. (The statement is repeated near the end of the paper.) Since this is a study on a homeopathic potion, it is likely that this claim will be understood to mean that we found the potion works significantly better than a placebo, but we have no way of explaining how any homeopathic remedy works. The claim is ambiguous enough, however, to mean (if one is generous in interpretation) that the study found that the homeopathic potion got better results than a placebo and the authors can’t come up with an explanation for the data in terms of our present state of knowledge regarding causal mechanisms and studies. But they don’t have to. All they have to say is that this difference could be a statistical fluke and more studies should resolve the issue.

Making sure everybody takes their rectal readings at the same times each day and night might yield slightly different results. Controlling for age before final analysis of the data might yield slightly different results.

So, the next question is: have all the studies done in the twenty years since this one was published helped to resolve the issue?

Not really.

Why?

Because there haven’t been any follow-up studies. Twenty years have gone by and not a single attempt at replication of this study has been published. The authors admit that the difference they found between the two groups was “modest.” They certainly did not find evidence of a robust effect from the homeopathic brew. What they found, they said, was “interesting.” Maybe so, but why has nobody done a follow-up? One would think that at least one study would have been done on young people with mild flu symptoms, since the differences in those groups were the greatest of those measured in the study.

The authors recognize that the differences measured, while statistically significant, are not sufficient to establish a causal relationship between oscillo and recovering from the flu. They note that “it would be unwise to claim that the study has demonstrated a cause and effect relationship between the drug and the recoveries.” Yet, in the very next sentence they write: “The positive effect of the homeopathic preparation cannot be explained in our present state of knowledge….” As they themselves note, they didn’t establish a causal relationship between oscillo and recovery from the flu. They didn’t prove effectiveness. They found a statistically significant difference between two groups. More studies might provide strong scientific evidence that oscillo causes some reduction of some flu-like symptoms in some people. Even so, if more studies replicate this one, the effect of the oscillo is so small as to be of little medical value. Would you buy a remedy that brags about being 10% more effective than a placebo or helps some people some of the time or helps with flu symptoms in 2 out of 10 cases?

Dr. M: Let me quote from Lancet, the most prestigious and respected medical journal: “A 7% difference in efficacy as defined would be a respectable proportion in most drug trials.” If one were to accept Dr. Carroll’s skepticism, most of conventional medicine is based on results “so small as to be of little medical value”. Actually, I feel this is true, because I am used to much better results in my practice. This study is not representative of results of homeopathy overall, just this preparation, administered in a non-homeopathic way. However, all of this is immaterial. The point is that the study is statistically significant. A submolecular preparation produced an effect significantly (p=0.03) different from that of placebo. Period. It can’t be explained by current physics or chemistry. But this does not mean it does not exist. Do we need to confirm it with more studies? Sure. Here are the studies. Do we want an explanation how it works? Sure! Are we likely to get it during our lifetime? Doubtful. Too bad. But the effect is evident nonetheless!

It has been over 20 years since this study was published and nobody replicated it.

Why?
____________

Note: This claim needs some clarification. As Dr. Steven Novella writes: “Homeopathic remedies are classified as “drugs” by the FDA, which means they fall under FDA oversight, but at the same time they are granted automatic approval as long as they appear in the Homeopathic Pharmacopeia. What this means is that the Homeopathic Pharmacopoeia Convention of the United States, the non-governmental agency that writes the homeopathic pharmacopoeia, only has to add an agent to their list and it is granted automatic FDA approval….But because homeopathic remedies do fall under FDA regulation the FDA has the power to decide that a product poses a safety risk to the public and therefore can demand evidence for safety.”

If you followed the link from the picture of the oscillo box above, you will find that the ad on Amazon states that the product is “Regulated as a drug by the FDA.” This is very misleading because the only way a homeopathic drug can be banned by the FDA is if it shown to be harmful in itself, which is impossible since there are no active ingredients in the products. (This is also misleading and confusing because of complex homeopathy and the fact that there are products on the market with active ingredients that are labeled homeopathic.)

If you gave five unlabeled homeopathic drugs in look-alike vials of liquid to a team of homeopaths and asked them to identify what each vial contains, they couldn’t do it because they have nothing to look for to distinguish one drug from another. I venture that you could mislabel every drug in a homeopath’s pharmacy and get the same level of satisfaction from both homeopath and patient.

Dr. M: More ridicule. We already discussed the point that these remedies are submolecular. The content can’t be determined by any known method in physics or chemistry. So?…

One of the most hilarious press releases I’ve ever read came from the French equivalent of the FDA announcing a recall of two homeopathic drugs because they had been mislabeled. The agency said: “…this mix-up does not pose any particular risk . . .” No kidding.

Dr. M: And even more ridicule… I guess, when one has no argument of substance, one is left to using this method in an attempt to drive their point across. However, consider the above mishap. If the pharmacy was a fraudulent outfit, they would just let the mislabeled product be sold. Obviously, they are honest in their beliefs and business dealings. This is gratifying!

*****

Dr. Mirman’s reply:

I just reviewed your review of the Ferley study. I definitely agree that the study should be replicated. In fact, I am aware of two similar studies. Both are published in homeopathic press, therefore by your standard they are not admissible here. One of them is actually in French, so I can’t even read it. It is very hard to get a conventional journal to publish a homeopathic study.

BC’s reply: Two attempts at replication in twenty years? It may be hard to get a homeopathic study published, but it shouldn’t be that hard to do the studies.

Nonetheless, we are discussing a specific study, so let’s stick to that. As you mentioned, some aspects of its design could be improved, as the authors themselves mentioned in the study. However, the study overall is good quality, and was even praised by an editorial in Lancet (http://bookonhealing.com/appendix.html).

BC’s reply: As you know, I didn’t reject the study because of design flaws. (And you know that it doesn’t matter who else has praised it or criticized it. This is our review.)

First we need to set the goal for this discussion.

BC’s reply: I took your challenge to discuss “a good [homeopathic] study” to be the goal.

In my understanding, we are working with a hypothesis that submolecular (dilution beyond Avogadro’s number) homeopathic preparations are not different from placebo. This is your assertion. Mine is the opposite: I believe they are different from placebo in biological systems (humans and animals) when properly applied. By the latter I mean the said system must in theory be susceptible to the effect of the particular preparation we are studying (i.e. if we are studying flu, the preparation must have something to do with the flu). We are in absolute agreement that these preparations contain no active molecules. There is no point in belaboring this point any further. My assertion is that there is something else in the preparation, something other than molecules, that exerts the effect, and we have no understanding of what it is. You, of course, believe that there is nothing there and we can’t expect to see any effect. Therefore, there is only one question: is there effect or not? Is the effect statistically significant as accepted by the research community, that is p<0.05. It is entirely immaterial how we feel about it. I agree that expecting any effect when looking at the issue from the confines of the science of today is completely ridiculous. Yet, first we need to look and see what is, whether it makes sense or not, then and only then attempt to explain what we see.

BC’s reply: I think I stated it very clearly in my response that I would evaluate the study on the basis of the data, i.e., any statistically significant differences between the experimental and control groups.

Now back to the study at hand. The remedy being studied is Oscilococcinum 200C, a submolecular preparation of gizzards of migrating ducks.

It is well known that migrating ducks are natural reservoirs of influenza viruses. They are the main cause of spread of new strains of influenza throughout the world every year. Oscilococcinum is a homeopathic remedy made from duck tissues, presumably containing a multitude of influenza viruses. Therefore it could in theory be used as a treatment for disease these viruses are able to produce. This remedy should have the capacity to trigger in the Vital Force some form of recognition of the energetic essence of the disease causing agent and help it deal with it more effectively. (I know you laugh at this “energy talk”, but, unfortunately, I don’t know a better language to express these concepts). This remedy should in theory be most effective for those people, who are suffering from some virus they have not previously been exposed to, because for them this “energetic” information is most useful. People who have been already exposed to all the viruses, whose signatures are contained in the remedy, would not derive much benefit from the remedy, if we follow this theory.

BC’s reply: The above paragraph has nothing to do with the study. As you say, let’s stick to the study.

Dr. M: Look who is talking! Just review the writing above to see who is not sticking to the study!. In any case, the above explanation is very important for understanding of the effect.

In my opinion, this is the reason why Oscilococcinum worked so much better for the young people than for those older. The older people have already been exposed to the viral signatures in the remedy sometime during their longer life, so the remedy did not provide their Vital Force with any useful information. As you noticed in the study, the difference of response of younger people to the active preparation vs. placebo was 25% vs. 8.1%, p<0.01. There was no statistical significance in the response of older people. This accounts for the overall smaller difference, even though still statistically significant. This pattern of response fits with the explanation given above, although, of course, does not prove it. Regardless of whether you accept the explanation, the fact of this statistical significant response is evident. The bottom line is, this puts the hypothesis of equality of Oscilococcinum 200c (a submolecular preparation) and placebo in doubt.

BC’s reply: Your opinion is not in line with the data. The data show that the number of recoveries among those under 30 was three times higher in the oscillo group (21 to 6) and the number of recoveries in the mild-to-moderate severity group was twice as high in the oscillo group (31 to 16). Such differences are not likely due to chance. But this is insufficient data from which to conclude that Oscilococcinum was the cause of these differences. It is because there could be a variety of factors that might account for the data that further studies are absolutely necessary before any talk of having established causality is considered. An interesting correlation was found, but you are jumping the gun by offering an explanation for a causal event that might not even be happening.

Dr. M: Well, the study was good quality, as you agreed. It did a good job to correct for any confounders. Now we have an unexpected result and what do we hear? It must be due to something else, rather than the active drug! This is exactly the attitude in people who have poor understanding of scientific process and want to push their agenda. When they see an inconvenient result, they discount it. Above statement by Dr. Carroll makes no scientific sense.

So now the ball is back in your court. Do you agree that this particular study, even given its minimal shortcomings,

1. is good enough 2. puts your point of view in doubt?

If yes, we can go further. I would agree that one study does not prove anything. I would be happy to discuss other studies in this manner. If not, please explain point by point.

BC’s reply: The authors of the study write: “it would be unwise to claim that the study has demonstrated a cause and effect relationship between the drug and the recoveries.” The study is certainly not good enough to assert that Oscilococcinum is a significant causal factor in the reduction of flu symptoms. The study has not changed my view about there not being compelling evidence of the causal efficacy of any homeopathic remedy.

I assume you selected this study because it was the best one you could come up with or you think it is typical of very good studies. This surprises me because the study is so old and insignificant. I have tried to provide you with a thorough evaluation of the study, indicating why the statistical significance found does not prove even a small effect from Oscilococcinum.

Dr. M: I have not seen this proof Dr. Carroll is talking about. He is introducing a positive statement here, designed to influence people sucseptible to suggestion. There is no beef behind this statement.

I read on your website how two experiences convinced you of homeopathy’s powers despite the obvious absurdity of its fundamental claims. You write in “My Conversion“: “I was certain that anyone who practices something so ridiculous must be a complete idiot or a crook…. The whole concept sounded so improbable, it fired up my curiosity. I knew I’d have to investigate it further, if only to disprove it.” Then you provide two anecdotes involving your grandmother and your father. Rather than seek alternative explanations for these experiences, you sought further confirmation of homeopathy.

Dr. M: This is the concept known in philosophy as Occam’s rasor: “Of several acceptable explanations for a phenomenon, the simplest is preferable, provided that it takes all circumstances into account”. (Wikipedia). You see something and acknowledge it. You try to reproduce it. If it is reproducible, you acknowledge the connection. You don’t theorise. It is simple. If > then. Logic 101.

You have done this for many years, you say. I doubt there is anything I could say that will change your mind at this point. I don’t expect to change your mind and I hope you don’t expect to change mine. I engaged in this discussion because I want to keep an open mind that maybe there is compelling evidence for a belief I reject. I must say that the evidence you provided is very weak and indicative of your low standards of evidence. I think you are deluded and are deceiving yourself about homeopathy. Belief in vitalism is magical thinking and a throwback to nineteenth century superstitions.

By the way, as an aside. Your use of the word POTION is intriguing. While the quack buster community dislikes homeopaths because they feel that we are basically using placebo and deceiving people, there is a different community that dislikes us for completely opposite reasons. Super-fundamentalist Christians have seen how much we can accomplish with our remedies, they understand that this can’t be explained by science of today, so they are afraid that we are practicing some kind of witchcraft and are in league with the devil. So to them our remedies are real magical POTIONS! I’d love to have a three-way discussion between you, me and them!

BC’s reply: Not gonna happen.

Another aside and a correction: Homeopathic remedies are in fact regulated by the FDA and are classified as FDA-approved drugs. This allows me, as an MD, to prescribe them and expect the nurses in the hospital to administer them. I have done it! Funny, isn’t it?

BC’s reply: I don’t think it’s funny at all. I think it’s pathetic.

As I note above, the FDA must approve any drug listed in the Homeopathic Pharmacopeia. Nobody needs to prescribe such things as Oscilococcinum. You can buy it off the shelf in most drug stores where it is probably placed next to a dozen other ineffective potions (as it is in my local Rite Aid).

***

Jacob Mirman, M.D. replies:

It is true, we won’t change each other’s minds.

However, it is interesting, how you are presented with a study with high p value and just discount it as unimportant, just because you can’t explain the mechanism of action. Of course, there are other studies we could discuss, but they will have the same problem: high p value but no explanation for mechanism of action, so you are likely to discount those as well, without any explanation. P value defines significance, by definition, yet you say it is insignificant. There is nothing else that would prove significance to you, if you don’t accept the definitions universally accepted in scientific community. Anyway, I make my attempts at showing people what is, but they see what they choose, each according to their own preexisting convictions and/or “party affiliations”. To some, my work is so effective it must be from the devil. To others, it is not effective at all. Both are quite religious in their convictions.

“…a scientist must …be absolutely like a child. If he sees a thing, he must say that he sees it, whether it was what he thought he was going to see or not. See first, think later, then test. But always see first, otherwise you will only see what you are expecting. Most scientists forget that.” (Douglas Adams So Long and Thanks for All the Fish)

Are you a scientist?

Jacob

reply: I’m not a scientist, but I know how to read a scientific study. I don’t discount high p values, but I recognize that statistical significance does not mean “important” and “unimportant” does not mean “statistically insignificant.” You are willing to assign a causal factor that needs an explanation for a mechanism of action on the basis of the p value found in one study. You even speculate as to what that mechanism is, even though it involves introducing the concept of “vital energy,” something the scientific community rejected long ago.

You are both misleading and wrong when you say “P value defines significance, by definition, yet you say it is insignificant. There is nothing else that would prove significance to you, if you don’t accept the definitions universally accepted in scientific community.”

I don’t say that P value is insignificant. You seem to think that statistical significance implies importance and causality. Statistical significance means that the statistic is not likely due to chance. It doesn’t mean that a causal event has been demonstrated, much less that anything significant or important has been discovered. Shall I repeat that? Statistical significance does not mean causal efficacy. If you don’t know that, then you should not be trying to pass yourself off as a scientist. Your ignorance is appalling. You have the gall to claim that I don’t accept the definitions universally accepted in scientific community,” when you don’t even understand what statistical significance means in science. If you understood what statistical significance means you would have understood the significance of my noting that “All it would take would be a different outcome for 3 or 4 in each group to make it obvious that nothing unusual is going on here.” Statistical significance in conjunction with a very large study and a very large effect size backed by several replications can usually be taken to indicate that a causal connection has been established. But statistical significance by itself should never be taken to prove causality. Correlation does not prove causation. Even if a causal connection is probable, statistics can never tell you what mechanism is at work. If you took the time to read the book chapter I recommended, you would know that in addition to your speculative cause of some mystical effect involving a vital force, there are many other possible explanations for the effect (if it is an effect). I’ve reproduced the list of possible causes in a recent newsletter. You can look at by clicking here.

Dr. M: I must say I don’t understand what Dr. Carroll is talking about here. A lot of words. Not that much sense in my opinion. If what I think he is trying to convey is true, then no scientific studies mean anything. Say, you test a drug for cholesterol. The active treatment group shows more cholesterol reduction than placebo group. But according to above, it means nothing, and the reduction could come from something else, rather than the drug. Does it make sense? Not to me.

Finally, you’re wrong when you claim that nothing would prove significance, in the sense of causal efficacy, of a homeopathic drug. All I require are some follow-up studies that meet the conditions I’ve already stated.

OK, which one would you like to discuss? Click here

To summarize. People like Dr. Carroll have an agenda to sow doubt that homeopathy works. They will use all possible methods to influence the minds of people susceptible to suggestion: unsupported statements, ridicule, sounding real scientific while making nonsensical statements… My question is why. What is driving them to such lengths to discredit something they don’t understand? Obviously, we can’t get into somebody’s head. But I have some ideas.

Some of them are on payroll of pharmaceutical companies. You can find some of this info here: www.quackpotwatch.org.

However, I don’t believe all of them are so dishonest. I think some of them actually believe what they preach. The only explanation in my mind is what we call in the trade “religious affections”. They believe. They believe there is no Vital Force. They believe there is no god. They believe homeopathy does not work. There is no substance behind the belief. It is exactly the same kind of belief (ultra-fundamentalist) that dictates that homeopathy is from the devil. Both the atheist and the ultra-fundamentalist suffer from the same “religious affections”. Even their expression is similar: they get quite emotional in their preaching. And they bash opposition playing on simple-mindedness and emotions of their audience. Religion… Don’t start me on this… It is the cause of all trouble in this world…