DHS still a mess, five years on

One of the main points of the creation of the Department of Homeland Security in 2004 was to centralize oversight over a wide array of agencies with responsibility for the safety and security of the United States and its territories. The 9/11 Commission made 41 specific recommendations to Congress, and one of those was "create a single, principal point of oversight and review for homeland security." But that's one that hasn't been accomplished--DHS oversight by Congress is through 86 separate committees and subcommittees (see chart below, click on it for the full-sized image).

The Center for Public Integrity and the Center for Investigative Reporting have joined forces to investigate the effectiveness of the Department of Homeland Security's efforts since its creation, and will be publishing a series of reports over the next several months which should prove quite interesting.

Science-based medicine conference posts yield new visitors

Looks like the chiropractic post has been referenced at a chiropractic forum, which is generating a fair amount of traffic:

http://www.chiroweb.com/cgi-bin/ubb/dcs_only/forumdisplay.cgi?action=displayprivate&number=4&topic=014324

And the chronic Lyme disease post has been referenced at a Lyme disease forum:

http://www.lymeneteurope.org/forum/viewtopic.php?f=7&t=2623&p=19265

It remains to be seen if this will produce any critical comments, though I noticed that an advocate for chiropractic, "nobs," showed up in the comments at the Science-Based Medicine blog and made a mistaken inference about the conference content--that it was 25% about chiropractic--because he failed to realize that my conference summary had only covered the first four of the six speakers at the time.

Nothing yet from homeopaths that I've noticed.

Science-based medicine conference, part 6: online health and social media, and Q&A

This is the sixth and final part of my summary of the Science-Based Medicine conference at TAM7, which will be followed by a summary of TAM7 itself. Part one, Dr. Steven Novella's introduction, is here. Part two, Dr. David Gorski on cancer quackery, is here. Part three, Dr. Harriet Hall on chiropractic, is here. Part four, Dr. Kimball Atwood on evidence-based medicine and homeopathy, is here. Part five, Dr. Mark Crislip on chronic Lyme disease, is here.

The sixth session speaker was Dr. Val Jones, CEO of BetterHealth, on "Online Health and Social Media: The Good, The Bad, and The Ugly." In this last post of my SBM conference summary, I'll cover her talk as well as the Q&A panel that concluded that day's events.

Personal Story
Dr. Jones began her talk with her personal history--she was raised in Nova Scotia by hippie parents from New York City and grew up on a farm with cows. She said her parents were "moderately weird"--they would ascribe magical powers to yogurt, but they vaccinated their children. She called herself a "shruggie" with respect to complementary and alternative medicine (CAM)--ambivalent about whether there could be anything to it.

She worked with the Ontario March of Dimes, became a doctor, did biophysics and vaccination research at the Mayo Clinic, and earned her M.D. in physical medicine at Columbia. She then worked at Medscape with George Lundberg, editor of JAMA, and created the Clinical Nutrition & Obesity journal there (at Medscape). She was then recruited by RevolutionHealth, an online provider of health information started by Steve Case of America Online. She described it as an "OnStar system for navigating the health care system." She moved to Washington D.C. to take the job, and, as she put it, "entered the Twilight Zone."

Revolution Health
She served as an editorial director and medical reviewer with a staff of 100 doctors across the country, and "couldn't believe the crap that came across [her] desk. Who are these people and why are they so into their colons?"

At one point, an article was submitted from a writer for Alternative Medicine magazine that claimed olive oil cures breast cancer. The study behind the story showed that breast cancer cells in a petri dish, exposed to a chemical found in olive oil, made some kind of genetic change--that was transformed into an alleged cure for breast cancer.

The company developed a health tracker tool, and developers kept adding trackers based on what they thought would be cool, such as a "hot flash tracker." She asked, "Why?" The developers answered, "so they can tell the doctor if a hot flash occurred at 2 or at 3 o'clock!" There was no clinical review of the tracking tools.

Medicine Chest
Another product was developed called Medicine Chest, which allowed people to vote their medicines up or down for how much they like them. "It's not going to be misinformation, it's the wisdom of crowds," the developers said. Not only could users vote on their medicines that were listed, they could add suggestions of their own in free-format text fields. The display of the results on the site didn't distinguish FDA-approved treatments from what people entered in on their own.

The result was that the best treatment for headaches, back pain, strains, etc. was narcotics, followed by marijuana. The best treatment for diabetes (without distinguishing type 1 from type 2) was dog walking.

Dr. Jones compared this to the Citizens' Briefing Book on Obama's change.gov website, where the general public could vote on what they considered to be the most important issues, with the resulting winner being the legalization of marijuana.

Other recommended treatments from the Medicine Chest feature included yogurt for colon cancer, acupuncture for ovarian cysts, herbal treatments for hip fracture, and steroids for cellulitis (which she observed is "very bad and dangerous"). Other similar sites took things to a further extreme, such as Patientslikeme.com, which allowed patients to conduct and report their own clinical trials online. This led to promotion of fish oil to slow the progression of multiple sclerosis and ALS. And beer and dogs as a treatment for lack of motivation.

She cited a quote from Poincare: "Science is facts; just as houses are made of stones, so is science made of facts; but a pile of stones is not a house and a collection of facts is not necessarily science."

She searched the Internet for help understanding the craziness, and came across Orac's Respectful Insolence blog, which she followed for several months. The last straw for her at Revolution Health was when it promoted chelation as a treatment for autism, which could kill a child, and she felt violated her Hippocratic Oath.

Dr. Jones listed a set of psychological factors which lead people to wrong conclusions, of the sort you might find in Kahnemann & Tversky's Judgment Under Uncertainty. On the list was the Hawthorne Effect, which purportedly showed that any change made in a business environment temporarily improves productivity. This effect was named after a study of worker productivity based on data collected after changes in lighting and other conditions at the Hawthorne Works between 1924 and 1932, but subsequent studies have failed to replicate the effect. The original data was recently rediscovered and reanalyzed by Steven Levitt (author of Freakonomics) and John List, with the result that "we find that existing descriptions of supposedly remarkable data patterns prove to be entirely fictional. There are, however, hints of more subtle manifestations of a Hawthorne effect in the original data."

Miscellaneous Slides
She concluded her talk with a few slides with various observations, such as The Onion's "NSF: Science Hard" article and a quote from Surgeon General Rich Carmona that the average American understands medicine at the 4th to 6th grade level. She pointed out that there's a cottage industry of quack cancer treatment providers around the M.D. Anderson cancer center, taking advantage of cancer patients. She criticized the 1994 passage of DSHEA and its signing into law by Bill Clinton, which exempted dietary supplements from FDA approval requirements unless they're found to be harmful. She quoted lots of examples of harm from whatstheharm.net. She recommended Memorial Sloan Kettering's herbal guide, noting that "doesn't work" is the conclusion for most descriptions, and recommended ClinicalTrials.gov for accurate information. And she closed with a quote from Hippocrates: "There are two things, science and opinion; the former begets knowledge, the latter ignorance."

Q&A
There was then a Q&A panel with all speakers. The first questioner came up to note that the CDC of Maine recently sent pediatricians copies of Paul Offit's book, Autism's False Prophets. He also noted that humor of the sort in the "That Mitchell and Webb Look" clip about the homeopathic emergency room was effective, and we need more things like that. Steven Novella responded that we need lots of different things, not any one thing, because AltMed has its marketing down pat.

Another questioner asked if there was a way for social media to work effectively in medicine, to which Dr. Jones responded that MedHealth "has lifeguards in the pool--physicians to moderate." In a later comment, she pointed out that MedHealth has 200-300 doctors who answer questions for free, because the referrals they get as a result more than make up for the [opportunity cost].

Someone else said that the book Snake Oil Science needs to be turned into an easily usable website, and complained that Quackwatch is hard to use and too polemical. Dr. Novella agreed that SBM needs to provide more, better, and more usable information. It would be good to have a place where you can find overviews on topics, allow you to dig as deep into technical detail as you want, and provides a list of sentinel references. (This is essentially what the TalkOrigins Archive provides for the creation/evolution debate, in particular with Mark Isaak's Index of Creationist Claims; the power of providing these kinds of broad and deep archives of reliable material was one of the key points of the talk I gave in June to the American Humanist Association.)

Another questioner asked whether there is anything we can do to get rid of the National Center for Complementary and Alternative Medicine. Dr. Novella said that it is becoming more widely known that NCCAM's Trial to Assess Chelation Therapy, its largest and most expensive trial to date, is loaded with "quacks and criminals" (guilty of insurance fraud and worse) and "totally corrupt," as has been reported by the Associated Press in several good stories. Bioethicist Art Caplan has pointed out that these are unethical experiments on human subjects that would (should?) never be tolerated by NIH. (NCCAM is part of NIH.) NCCAM has spent $1.2 billion $2.5 billion of taxpayer money to date, and produced zero new treatments.

Someone raised the question of what kinds of questions to ask your own doctors to make sure they're giving good advice. Dr. Jones suggested asking, "Do you use UpToDate?", which is a service that searches the world medical literature regularly and provides current data reviewed by 300 full-time reviewers. Dr. Gorski suggested asking whether a doctor follows the NCCN guidelines, which are evidence-based cancer treatment recommendations. Dr. Novella observed that just using Google, a "pull approach," how most people look for medical information online, is highly unreliable because "bad sites are good at looking like good sites." (I'd suggest that a more specialized search engine is a better way--Tim Farley suggested some ways of creating such capabilities at TAM6 last year.) Dr. Hall said that Stephen Barrett's rule of thumb for distinguishing good from bad sites is that "if it's selling something, it's a bad site." I'm not sure how effective that rule is, since even good sites are typically selling something.

Someone raised a problem for use of prior probability, noting that it could have made us miss out on the discovery of lithium as a treatment for bipolar disorder, since it was originally postulated on rather shaky grounds. He gave a second example as SSRIs, which are effective in treating depression, but the original MAOI hypothesis of their operation has been refuted. Dr. Novella responded by saying that first of all, no known mechanism should imply a neutral prior probability (i.e., 0.5). Second, in deciding what to research, it's better to err on the side of the implausible--but not for treatment. He further suggested that lack of mechanism should not be equated with implausibility. Dr. Atwood seconded that there is a difference between lack of mechanism and contravention of a physical law, and made reference to the discussion that he and I had during the break. He gave aspirin as another example of a substance where the mechanism was discovered later than its effectiveness, and expressed doubt about the questioner's story of the discovery of lithium's usefulness.

David Whitlock raised the question of framing, asking why we don't draw the distinction as science-based vs. faith-based medicine. Dr. Novella responded that this would cause more problems than it would solve, at least in the United States, because of the immunities granted to the free exercise of religion. A questioner wondered whether it might at least stop the government from paying for "faith-based" medicine under single payer. (I don't think we're likely to get single payer, and I note that we still have an Office of Faith-Based Programs, so I think this is not a good suggestion.) Dr. Gorski noted that very little CAM is actually religion-based.

A questioner asked how corrupt the Cochrane data is, to which Dr. Atwood replied that the contributions on CAM subjects are unreliable, but reviews of substances are good and Cochrane in general is good. Dr. Novella said that he uses Cochrane to get studies and results, but ignores their conclusions, and pays close attention to authorship. Dr. Hall said that when it comes to meta-analysis, if the result is negative, you should believe it, but if the result is positive, you should look further. Dr. Novella noted that the systematic reviews in Cochrane aren't actually meta-analyses.

And that pretty much wrapped up the day for the Science-based Medicine conference.

If you'd like to continue on to my summary of The Amazing Meeting 7, it begins here.

UPDATE (July 19, 2009): I've been reminded that I neglected to mention one of the more interesting questioners, a massage therapist who stood up and said that he was probably the only "woo practitioner" in the room (though the doctors disagreed that massage therapy really counts as "woo"--and see Dr. Atwood's talk, where he classified massage therapy as having high prior plausibility), who regularly attends CAM conferences. He complimented the speakers and the audience for having a level of displayed intelligence, sophistication, and scientific knowledge that is not seen at those CAM conferences.

Science-based medicine conference, part 5: chronic lyme disease

This is part five of my summary of the Science-Based Medicine conference at TAM7, which will be followed by a summary of TAM7 itself. Part one, Dr. Steven Novella's introduction, is here. Part two, Dr. David Gorski on cancer quackery, is here. Part three, Dr. Harriet Hall on chiropractic, is here. Part four, Dr. Kimball Atwood on evidence-based medicine and homeopathy, is here.

The fifth session speaker was Dr. Mark Crislip, infectious disease specialist and host of the Quackcast podcast, on "Lyme from the IDSA to the ILADS to the ABA."

Like several of the other speakers, Dr. Crislip began with a disclosure of potential conflicts of interest, saying that he had "barely any" and "[hasn't] spoken to a drug rep in 25 years."

He started his talk with a description of Lyme disease. It's caused by a spirochete related to syphilis, that comes in three varieties, European, Asian, and North American. The latter is Borellia burgdorferi, a nasty little organism that lives in ticks, primarily deer ticks. It's transmitted via a tick bite, requiring 36 hours of attachment for transmission, and has grown in prevalence in the northeastern United States with the growth of the deer population. In the northeastern U.S., most ticks have Lyme, while in the northwestern U.S., only 1.3% of ticks have Lyme, because those ticks feed on the blood of a fence-sitting lizard that contains something that kills the spirochetes. (Here, Dr. Crislip joked that despite the presence of numerous fence-sitting lizards in Washington, D.C., the effect doesn't work there.)

North American Lyme disease goes through three stages:

1. skin rash, arthritis
2. spreads to whole body, causes meningitis
3. results in encephalomyelitis and neurological symptoms

There are drugs that work well to treat the disease at all three stages.

However, there are "also people who think they have Lyme but don't," or "post-Lyme disease."

Two Camps
Dr. Crislip identified two groups that have radically different views about Lyme disease:

1. The Infectious Diseases Society of America (IDSA).
2. The International Lme and Associated Diseases Society (ILADS)

The latter says that Lyme is common, hard to diagnose, and "requires infinite antibiotic treatment." The former says nearly the opposite.

The New England Journal of Medicine published a critical appraisal of "chronic Lyme disease" which put the sufferers into four categories:

1. Symptoms of unknown cause, no evidence of B. bergdorferi.
2. Well-defined illness unrelated to B. Bergdorferi (e.g., ringworm).
3. No history of Lyme disease, but blood contains B. Bergdorferi antibodies.
4. Post-Lyme disease syndrome, chronic illness.

The study performed controlled trials of those in category four, and concluded that there is no evidence of B. Bergdorferi persisting beyond treatment, the name "chronic Lyme disease" is a misnomer, and there is no justification for continued antibiotic treatment of such persons.

Dr. Crislip then stated that the two camps present a false dichotomy, but that the truth is closer to the IDSA position. He asked, "is there asymptomatic Lyme?", and answered "yes." 7% of test subjects have asymptomatic seroconversion (show B. Bergdorferi antibodies) in vaccine trial placebo groups. He asked, "can [Lyme be] persistent due to antibiotic resistance?", and answered that there is no good data of that.

He pointed out that Borellia can exist in three forms, the spirochete, a cyst, and an L-shaped form with no cell wall. The cysts appear when the organism is stressed, but isn't found in humans (and is supported in fewer than 25 references in PubMed) and the L-shaped form can be made in the lab but doesn't survive in humans.

Diagnostic Testing
Dr. Crislip said that the standard test for Lyme disease is a two-step process, an ELISA test confirmed with a Western Blot (the same process used for testing for HIV). With classic Lyme disease, this is a very reliable method. It can also be tested with PCR and with antigen assays, and "there is genotypic variation in Lyme that could potentially make the two-step test less sensitive."

There are also labs which perform their own unvalidated tests, such as a lab in Texas that he says "almost always yield[s] positive results." These labs with unvalidated diagnostic tests have caused the CDC and FDA to issue warnings about non-valid Lyme tests.

Dr. Crislip posted a list of alleged symptoms of chronic Lyme disease, which was a very long list including "unexplained hair loss" and "feeling as if you are losing your mind," along with another list of alleged symptoms of chronic candida, and noted that they were quite similar. Using such lists, virtually any symptom is an indicator of these alleged chronic conditions.

The ILADS guidelines go even further, and say such things as:

• The labs are all unreliable, so treat for Lyme even if the test is negative.
• The primary symptom is that the patient thinks they have the condition.
• Physical findings are nonspecific and often normal.
• If the Western blot result is ambiguous, treat it as positive (the opposite of what you do with HIV).
• A comparison to tuberculosis and leprosy provides justification for long-term antibiotic treatment (even though those diseases are biologically dissimilar to Lyme).

In short, the ILADS guidelines provide a nonfalsifiable definition of Lyme disease.

The best trials in the NEJM treated Lyme disease patients with a month of cipro (and?) doxycycline. The "chronic Lyme disease" advocates say that the immune system is damaged with antibiotic use, and then Lyme disease increases as the immune response declines--based on no data.

If you don't have the data, sue
The state of Connecticut passed a bill "giving doctors immunity for giving infinite supplies of antibiotics" to patients purportedly suffering from "chronic Lyme disease." Since the IDSA guidelines are against long-term antibiotic use, the Connecticut Attorney General sued the IDSA. They couldn't afford $250,000 in legal expenses, so they settled the case.

Dr. Crislip concluded by pointing out that the cause of this unsubstantiated syndrome will be promoted by a new film coming out, called "Under Our Skin," which has the tag line "There's no medicine for someone like you." Crislip noted that of the two doctors in the film promoting this illness, one lost his license for diagnosing Lyme disease over the telephone.

(Part six of my conference summary, on online health and social media, and the final Q&A panel session, is here.)

Science-based medicine conference, part 4: evidence-based medicine and homeopathy

This is part four of my summary of the Science-Based Medicine conference at TAM7, which will be followed by a summary of TAM7 itself. Part one, Dr. Steven Novella's introduction, is here. Part two, Dr. David Gorski on cancer quackery, is here. Part three, Dr. Harriet Hall on chiropractic, is here.

The next session was Dr. Kimball Atwood, an anesthesiologist who is also board-certified in internal medicine and associate editor of the Scientific Review of Alternative Medicine. He spoke on "Why Evidence-Based Medicine is not yet Science-Based Medicine," or "Do Clinical Studies of Highly Implausible Methods Help or Hinder?"

Dr. Atwood's talk made some points right at the start similar to the critique of evidence-based medicine (EBM) given in Dr. Novella's introduction--that EBM makes the mistake of devaluing what he called "pre-clinical knowledge." Unlike Dr. Novella, however, he also stated that random controlled trials (RCTs) "cannot, by themselves, overturn well-established medical principles." I'm not sure that his talk actually demonstrated that point--as stated, it sounds like "well-established medicalscientific principles" are part of a "hard core" of medical theory (in Imre Lakatos' sense) that cannot be refuted even if found to be in conflict with empirical results, because they are shielded by the addition of auxiliary hypotheses to salvage it. I don't think that was his intent, as surely even "well-established medical principles" should be eligible for revision in light of contrary empirical evidence. Instead, what I think he meant by "pre-clinical knowledge" and "well-established medical principles" is really more like a demand for consilience with the rest of scientific knowledge, adherence to logic and mathematical principles, and having a plausible mechanism (or at least not having a purported mechanism in conflict with other known facts). More on that in the "Q&A" section, below. [UPDATE (July 21, 2009): As Dr. Atwood notes in the comments, I incorrectly transcribed what his slide said. Also note his further discussion on what he means by a "plausible mechanism.]

He began his talk with an overview of EBM--EBM advocates, to quote EBM pioneer David Sackett, "use of the best available evience, especially from patient-centered clinical research." It relies on randomized controlled trials and systematic reviews. Dr. Atwood gave two examples of successes of EBM. First was the standard practice of giving heart attack patients anti-arrhythmia drugs post-myocardial infarction, which EBM trials showed causes excess deaths and was an incorrect practice. Second was the Women's Health Initiative study on hormone replacement theory, which showed that risks exceed benefits for taking estrogen. (I'm no authority, but I am skeptical of this claim based on my understanding of the flaws in that study from conversations with the late Chris Heward, who is co-author on a paper in Fertility and Sterility that challenged the WHI study for methodological flaws which made it unable to detect cardioprotective effects.)

The four steps of "pull" EBM are:

1. formulate an answerable question
2. track down the best evidence
3. critically appraise the evidence
4. individualize, based on clinical expertise and patient concerns

Atwood stated that those in practice the longest perform the worst, by not keeping up to date with their fields.

Cochrane Collaboration
Atwood next turned to the Cochrane Collaboration, the organization that maintains a library of EBT results, classified by type of evidence and evaluated with reviews in the form of evidence statements and recommendations. He gave a few examples, such as a statement about the effect of physical activity on dementia: "[There is] insufficient evidence to determine the effectiveness of ... physical activity programs in managing or improving ... dementia." And a seemingly equivalent statement about the use of homeopathy for the same purpose: "In view of the absence of evience, it is not possible to comment on the use of ... homeopathy in treating dementia." This, according to Dr. Atwood, is B.S. -- Bogus Science. Why don't they just say that homeopathy doesn't work? Because there are no sound clinical trials in the database.

EBT categorizes evidence into three classes. Class I evidence involves randomized controlled trials (RCTs). Class II involves controlled trials without randomization (or several other forms of case-control studies). Class III involves one or more case studies of a treatment without a control group and is considered insufficient for a treatment to be called "evidence-based medicine." Within each class there are further divisions, for example, Class I is broken down further, with systematic reviews of RCTs at the top (1a), followed by individual RCTs (1b), and so forth.

Atwood objected that this devalues pre-clinical knowledge by making "evidence" synonymous with clinical trials, and that therefore EBM "is not based on all of the evidence." He provided a few more examples of complementary and alternative medicine (CAM) treatments from Cochrane, described with terms like "Not enough evidence to ...," "little evidence," and, in the case of laetrile as a cancer treatment, "No studies found that met inclusion criteria." There was, however, a 1982 New England Journal of Medicine clinical trial on 180 patients which yielded a negative result and the conclusion that the treatment is dangerous, but this constituted a case study at the lowest level of evidence in EBM.

Atwood quoted a statement from Edzard Ernst, co-author with Simon Singh on the chiropractic-critical book, Trick or Treatment, to the effect that "a priori plausibility has become less and less important" as a result of EBM (which Ernst advocates).

He then continued with more CAM treatments in Cochrane, such as craniosacral therapy, reflexology, Kirlian photography ("may be more reliable than chance"), and therapeutic touch ("remains controversial").

Homeopathy
Atwood then described the case of homeopathy in order to make his central argument critical of EBM. Samuel Hahnemann invented homeopathy in 1796, on the basis of two principles. The first principle is "similia similibus curantur," or "like cures like," or the "law of similars." This claims that if you find a substance that gives you symptoms similar to an illness, that's the substance you use to cure that illness. Hahnemann read in William Cullen's "A treatise of the materia medica" that cinchona bark could be used to treat malaria (now known to be true because of quinine in the bark). He gave himself a sample of that bark while healthy and observed that he developed symptoms that were similar to malaria. From this single example, he concluded that all medicines produce symptoms in healthy people similar to the symptoms of diseases they effectively treat.

The second principle of homeopathy is the "law of infinitesimals." He reasoned that dosages sufficient to produce overt symptoms were too high, so the substances should be diluted in order to provide an effective treatment, and in fact the more diluted, the more powerful the cure. Homeopathic remedies of 24X (or 12C) are the equivalent of diluting 0.36mL of salt into a volume of water the size of the Atlantic Ocean. Hahnemann most commonly recommended an even greater dilution of 60X (30C). For all intents and purposes, homeopathic remedies of standard dilutions are indistinguishable from the water used to dilute them.

Atwood went on to note that homeopaths do not agree on prescriptions for various maladies.

On top of that, the outcome of all trials to date have been failures. This is a long list of powerful reasons for rejecting homeopathy, but the last one is the only one EBM considers relevant. There is clearly a very low degree of plausibility for homeopathy independently of such trials, and homeopathy is a clear case in point that "some hypotheses are too implausible to spend time on (or spend more time on)."

Prior Probabilities
Atwood offered the following set of broad categories of prior probability and types of treatments that fall into them:

Prior probability of about zero: homeopathy, neurocranial restructuring (putting balloons up your nose and inflating them).

Prior probability significantly lower than (<<) 1: metabolic therapies for cancer, detoxification, chiropractic for any purpose other than back pain Prior probability very low: acupuncture for pain most popular herbal claims (St. John's wort, echinacea) Prior probability moderate to high: massage, relaxation techniques for anxiety reduction and chronic pain Prior probability depends on:

• basic science
• cogency of theory
• previous studies
• source

He then discussed Bayes' Theorem, which says that the odds of a hypothesis being true given certain evidence is equal to the odds of the hypotheses (prior probability) multiplied by the Bayes factor, which is the probability of the evidence given the hypothesis divided by the probability of the evidence given the null hypothesis. If the prior probability is near zero, then much greater evidence is needed in the Bayes factor to result in a probability of the hypothesis being true. (Note that Atwood gave the theorem partly in terms of odds rather than probability for the purposes of his talk. Odds = probability / (1 - probability).)

When Bayes' Theorem is taken into account, the p-value of a statistical result can become much less impressive. For example, with a p-value of .05, which means that a result would be expected to occur by chance 5 times in 100 with a Bayes factor of 2.7, if the prior probability is only 1%, that result only raises the posterior probability to 3%. If the prior probability is 20%, it raises it to only 40%. With a p-value of .01, a result expected to occur by chance only 1 time in 100, and a Bayes factor of 15, a prior probability of 1% is raised to 13%; a prior probability of 20% raised to 78%. Dr. Atwood provided a table with more detail that went up to p=.001 (result expected by chance 1 in 1000 times).

Dr. Atwood advocated that "prior probability ought to be formally considered in EBT," and gave the further example of a "positive" trial for intercessory prayer in the critical care unit (CCU) with an 11% reduction in some harm with a p-value of .04, and noted that if the prior probability was 1%, this still produces less than 6% odds of a genuine effect. A few other similar examples were given involving acupuncture, homeopathy, and distant healing, the final example of which had Edzard Ernst as a study co-author and concluded that it "warrants further study," but which he subsequently backed away from after "some positive trials [were] found to be fraudulent."

Finally, he noted that pre-trial knowledge is not sufficient, but is necessary for a treatment claim.

Q&A
I asked Dr. Atwood if, in his final statement, he was saying that you have to have a plausible mechanism for a treatment in order for a treatment to be justified (a positive requirement), or if he only wanted to impose a negative requirement that the proposed mechanism or method did not contradict known facts from other realms. His initial answer was that he thought those would be equivalent, but I observed that we can discover cause-effect relationships without having any knowledge of the underlying mechanism, such as Mendel's discovery of genetics. At that point, he agreed that he just wanted to require the negative condition. Another audience member then suggested that this might be accomplished by creating a categorization scheme for levels of plausibility that in some way parallels the levels of evidence scheme.

Another questioner asked how to standardize assignment of prior probabilities and address bias, to which Dr. Atwood said that you could just pick neutral prior probabilities, since if you do enough studies the posterior probability of each study becomes the prior probability for the next.

EXTRA: As appropriate for a talk that touched on homeopathy, prior to Dr. Atwood's presentation this excerpt from the fourth episode of season three of "That Mitchell and Webb Look" was shown to the audience:



(Part five of my conference summary, on chronic Lyme disease, is here. Part six, on online health and social media, and the closing Q&A panel, is here.)

Science-based medicine conference, part 3: chiropractic

This is part three of my summary of the Science-Based Medicine conference at TAM7, which will be followed by a summary of TAM7 itself. Part one, Dr. Steven Novella's introduction, is here. Part two, Dr. David Gorski on cancer quackery, is here.

The next session was Dr. Harriet A. Hall, the "SkepDoc," a retired family physician and former Air Force flight surgeon, on "A Scientific Critique of Chiropractic."

Chiropractic
Dr. Hall began her talk by observing that 10% of Americans see a chiropractor some time each year, and said that her talk would address the questions of when you should see a chiropractor, what they really do, is it based on science, and why isn't it part of regular medicine.

Chiropractic, she said, is a pre-scientific system that was discovered on a single day (September 18, 1895) by D. D. Palmer, a grocer and magnetic healer. On that day, he performed a spinal manipulation on a deaf janitor and allegedly cured his hearing, and based on that single case, claimed that he had found the cause of all disease. Hall noted that in 1895, Pasteur had just died, X-rays had just been discovered, and the germ theory of disease was just catching on. For perspective, she noted that 1900 was the first year in which you would have a 50% chance of having a beneficial outcome from a visit to a doctor.

Chiropractic theory says that all disease is caused by subluxations, or dislocated joints, causing nerve impairment, thereby impacting the flow of "Innate," a mystical force that flows through us. There are real subluxations, which are visible on X-rays, but chiropractic subluxations have never been demonstrated to exist. The chiropractic finding of "Boop"--bone out-of-place--can't be documented on X-rays, and chiropractors have claimed that they are as small as 1/10,000,000 mm.

The current chiropractic definition of subluxation (as defined by the American Association of Chiropractic Colleges) has been adjusted to be more vague: "a complex of functional and/or structural and/or pathological articular changes that compromise neural integrity and may influence organ system and general health."

Palmer said that 95% of ill health and disease is caused by spinal subluxations, and 5% from misalignments of other bones. Hall suggested that if this were true, invertebrates wouldn't get sick, though perhaps chiropractors would say that Palmer was only speaking of the causes in humans. She went on to point out that a ruptured disc is an example of a spinal injury which causes pain without subluxation, directly refuting Palmer's claim.

The chiropractic theory of nerve impingement often uses a garden hose analogy, that a kink in the hose causes water to back up on one side of the kink. But this analogy is not correct, because nerve conduction speed is only affected at the point of compression, as seen in carpal tunnel syndrome. While such nerve dysfunctions do exist, they don't work the way chiropractic says, and they are not the cause of most disease.

A further problem for chiropractic is that 12 cranial nerves and 5 sacral nerves are out of reach for chiropractic manipulation.

Hall showed a diagram about the flow of "Innate" which showed the doctor's "Innate" influencing the patient's "Innate," but for which there was no described mechanism. There is, of course, no evidence that "Innate" exists, and it's now rejected by many chiropractors.

The audible crack from chiropractic manipulation, Hall said, is just like knuckle cracking, and has no therapeutic effect except perhaps for its psychological impact. She noted that at least one chiropractor cracks her own wrists and doesn't actually touch her patients!

Osteopathy and Chiropractic
Hall compared osteopathy to chiropractic with this chart:

Osteopathy	Chiropractic
restoring blood flow minimize spinal manipulation therapy (SMT) adopted science	restoring nerve flow maximize SMT nope

Spinal manipulation therapy, said Hall, works as well as other treatments in treating back pain, and there is no evidence that it assists in treating anything other than back pain. A problem in testing SMT is that good placebo controls are impossible, because the patient can tell when SMT is occurring or not. Single blinding is impossible, let alone double blinding. There are also many variations of technique possible.

Tooth Fairy Science
Dr. Hall argued that chiropractic study is something like what she called "tooth fairy science." You could perform tests of the tooth fairy, by putting lost teeth into baggies vs. facial tissue before putting it under the pillow, comparing the amount of money received for the first lost tooth to the last lost tooth, looking for correlations with parental income, and so forth, but none of this would tell you anything about an entity called the Tooth Fairy.

She gave an anecdote of a man suffering from back pain who made an appointment with a chiropractor for the following Monday. Over the weekend, his pain went away--if he had made his appointment for Friday, he would have attributed the pain going away to the chiropractic treatment and perhaps ended up as a believer and regular patient.

She compared this to the "blue dot cure," a treatment by painting a blue dot on your nose (any nonsensical treatment suffices for the illustration). If the patient gets better, that's evidence that it worked. If the patient stays the same, that's evidence that the treatment kept him from getting worse. And if the patient gets worse, that's evidence that the disease was too far advanced, and if only he had come sooner it would have been treatable. (Or, alternatively, it's evidence that more treatment is required--say, upping the dosage of the remedy or painting a darker blue spot on the patient's nose.)

What chiropractors do well, Hall said, is help with back pain and act as good psychologists. But they've gotten thumbs down the New England Journal of Medicine, Consumer Reports, The Medical Newsletter, Canadian neurologists, and many other sources, not just because of claims to be able to treat things that it can't treat, but because of safety issues.

The Big Downside of Chiropractic
Chiropractic manipulation of the head and neck turns out to be dangerous. It can compress or tear vertebral arteries, resulting in strokes, perhaps as frequently as in one in a million manipulations, and perhaps 20% of basilar strokes are caused by spinal manipulations. Hall pointed out that mobilization is as effective as manipulation, yet is safe--so there's no reason to use the riskier method.

The Canadian neurologists mentioned above have recommended that signs warning signs be posted in chiropractic offices and that neck manipulation of children be prohibited.

Insurance data on payouts for chiropractic malpractice show that about 9% of payouts in 2002 were for "CVA," or cerebrovascular accident.

The numbers are likely under-reported, since there may be some time between a manipulation that causes damage and a stroke.

Chiropractors as Quack Magnets
A further problem with chiropractic is that practitioners are "quack magnets," promoting all sorts of bogus diagnostic methods and treatments such as moire contour analysis, use of a "neuroscope" that measures temperature differences and pressure, biofeedback as a form of electrodermal testing, applied kinesiology (pseudoscientific muscle strength testing), unnecessary dietary supplements, homeopathy, and reflexology. Hall also mentioned B.J. Palmer's (D.D. Palmer's son) "Atlas Adjustment" technique, or "hole in one" technique, which suggests that manipulation of the Atlas vertebra is sufficient to cause the rest of the spine to fall into alignment.

Chiropractors also tend to overuse X-rays, such as taking full-spine X-rays which expose the reproductive organs and inevitably produce overexposed or underexposed areas. Proper X-raying focuses on smaller areas to get the right exposure.

Chiropractors offer bad advice, frequently discourage immunizations and other medical treatments, and they frequently miss diagnoses of real illnesses. In one test of chiropractors, patients were sent in to describe classic heart attack symptoms, but none of the chiropractors in the test recognized it or its significance, and none suggested that the patient visit an emergency room.

Types of Chiropractor
There are three major chiropractic groups. The International Chiropractors Association (ICA), or "straights," who practice only chiropractic. The American Chiropractic Association (ACA), or "mixers," who mix chiropractic with other diagnostic and treatment techniques. And the National Association for Chiropractic Medicine (NACM), who are attempting to reform chiropractic by disassociating it from Palmer's pseudoscience and using only evidence-based scientific medicine.

Chiropractic Thinking and Hallmarks of Pseudoscience
Hall gave some examples of chiropractic thinking that is blatantly wrong or harmful:

• If spine is straight, we can't die.
• Germs don't cause disease, or we'd all be dead.
• Muscle testing to find allergies. (In one case, a patient was tested for job-related stress with applied kinesiology, by pushing down on his arm while he thought of work.)
• Spinal adjustments as the only treatment for meningitis, resulting in the child's death--and it was the chiropractor's own child.
• A "no-touch" chiropractor (cracking own wrists, mentioned above).

And examples of how chiropractic exhibits the hallmarks of pseudoscience:

• "If science disproved it, I'd still use it."
• It doesn't give up ineffective treatments.
• It's made no progress over the last century.
• It doesn't matter whether it's true as long as it makes you feel better.

Dr. Hall concluded her talk by giving several three examples of concrete harm from chiropractic taken from Tim Farley's "What's the Harm" website, which lists 312 specific cases of chiropractic harm:

Kristi Bedenbaugh: Suffered a stroke and died.
Sandra Nette: Suffered multiple strokes and was left paralyzed with locked-in syndrome.
Laurie Jean Mathiason: Fell into a coma and died after receiving 186 neck manipulations in a six-month period.

Dr. Hall recommended the Quackwatch site and Chirobase.org as online resources on chiropractic, and the book Inside Chiropractic by chiropractor Samuel Homola and Quackwatch founder Stephen Barrett.

Q&A
In the Q&A session, Dr. Hall was asked whether chiropractors are required to follow a standard of care and whether informed consent is required. She said that informed consent is not required, it is voluntary, and the informed consent forms that she's seen are very bad. She was also asked how many chiropractors meet the good and safe criteria, and she made a guess of under 10%.

(Part four of my conference summary, on evidence-based medicine and homeopathy, is here. Part five, on chronic Lyme disease, is here. Part six, on online health and social media, and the closing Q&A panel, is here.)

Science-based medicine conference, part 2: cancer quackery

This is part two of my summary of the Science-Based Medicine conference at TAM7, which will be followed by a summary of TAM7 itself. Part one is here.

David Gorski, surgical oncologist and associate professor of surgery at Wayne State University, perhaps better known as Orac of the Respectful Insolence blog, spoke next on "Case studies in cancer quackery: Testimonials, anecdotes, and pseudoscience." He began with a disclaimer (he doesn't speak for his employer) and some disclosures (he receives no pharma funding and isn't paid to blog).

His talk was about misrepresentations by cancer quacks, who use exaggeration and misrepresentation and make false promises. To illustrate exaggeration, he showed a cartoon that described "three approved paths to the graveyard," "cut" (surgery), "burn" (radiation), and "poison" (chemotherapy) (the exaggeration is in the title rather than the descriptions, which are he admitted were accurate). To illustrate the latter, he showed a series of book covers by Hulda Clark--"The Cure for All Advanced Cancers," then "The Cure for All Cancers," and then "The Cure for All Diseases." She thinks that all cancers and diseases are caused by liver flukes, to be diagnosed with a "Syncrometer," a device similar to a Scientology E-meter, a galvanometer that measures electrical resistance of the skin, and cured with the "Zapper," a low voltage electrical device.

Some of the frequent claims of cancer quacks are that they are "wholistic" and treat the whole patient rather than a part or a symptom, that "we treat the real cause of cancer," that their treatment is "natural," and that "cancer is not the disease, it's a manifestation of something else" such as psychological conflict. And, of course, the ever-popular generic "toxins." They also claim that natural cures are being suppressed because Big Pharma can't make a profit from them.

Testimonials
Gorski next turned to the role of testimonials in cancer quackery, which he said are being used for several reasons, the first N of which are "to sell a product." Two other reasons are "to persuade others" and "to attack standard evidence."

He gave Gorski's Laws of Testimonials:

1st Law: When a believer in Alternative-Based Medicine (ABM) uses a combination of both science-based medicine (SBM) and ABM and gets better, it's always the ABM that gets the credit.

2nd Law: When a believer in ABM uses a combination of both SBM and ABM and dies or gets worse, it's always SBM that gets the blame.

He next described two cases of testimonials, the first of which was an example of a "not cancer" testimonial. This was a testimony of a man who felt a lump on his chest which he claimed to be breast cancer, which was successfully treated by some quack remedy. But this was never diagnosed as cancer, and Gorski noted that from the description it actually sounded like a case of gynecomastia rather than cancer. His second case was that of Daniel Hauser, a 13-year-old boy with Hodgkin's lymphoma, who went through one round of chemotherapy with good results, but then stopped taking it because he and his mother wanted to use an alternative treatment from "Chief Cloudpiler." The judge ordered chemotherapy to be continued, and he and his mom took off, though ultimately returned and re-started chemotherapy. During the time chemotherapy stopped, the tumor grew to larger than it was originally, and when it was restarted, it again responded to treatment--but of course his mother gave credit to the alternative treatment.

The problems with testimonials are that there may not have been a diagnosis of cancer, there may have been a misunderstanding of the diagnosis (e.g., "I was sent home to die"), there may be important information withheld, the diagnosis may have been done by quack tests with no validity (e.g., the Syncrometer or live blood cell analysis), and there may be a selection bias. As an example of the latter, he noted that dead people don't give testimonials.

Questions for Evaluating Testimonials
He provided a modified version of Dr. Moran's questions to ask in order to evaluate testimonial evidence. These questions include:

• Was cancer definitely present?
• Did it go away?
• Was the advocated treatment the only one used?
• Was the alternative therapy a replacement for primary or for adjuvant therapy?

At this point, he distinguished primary, adjuvant, and neo-adjuvant therapies. The primary therapy for most cancer treatments is surgery, to remove as much of it as possible. Adjuvant therapy is designed to reduce the risk of recurrence, where radiation is used to reduce the risk of local recurrence (cancer in the same place, to make sure you get rid of it all) and chemotherapy is used to reduce the risk of a systemic recurrence (cancer that may have spread to other parts of the body). Neo-adjuvant therapy is designed to shrink a tumor prior to surgery, and may reduce complications and produce better results from surgery.

Suzanne Somers
To illustrate the importance of these questions and distinctions, he used the case of Suzanne Somers, who was diagnosed with breast cancer at the age of 54, probably at stage I. She had no positive lymph nodes and underwent a lumpectomy, radiation, and a lymph node biopsy, but refused chemotherapy with tamoxifen in favor of mistletoe extract and other supplements.

In her case, the answers to the questions were:

• Was cancer present? Yes.
• Did it go away? Yes--it was removed by surgery.
• Was the alternative medicine the only or primary treatment? No.

He then examined her probable survival rates with and without chemotherapy, and noted that if the tumor was small, the benefit of chemotherapy for her 10-year survival rate could be as low as 1%. With a larger tumor, her 10-year survival rate improvement could still be as low as 4% (and would already be at 90% prior to chemo). But, Gorski noted, most women say that they would go with chemotherapy even for as little as a 1% increase in survival rate.

Surgery cures most cancers that can be cured, up to stage III, and the corresponding benefits of chemo and hormonal therapy increase with more advanced stages of cancer.

Gorski then observed that there may be cases where a person is diagnosed with cancer by a biopsy, declines further treatment, and has a good survival rate, where they fail to realize that the biopsy itself has been a surgical primary treatment that has excised all of the cancer--an excisional biopsy may be equivalent to a lumpectomy. He also noted that many people say to go ahead and take out the tumor but don't touch my lymph nodes, and he agreed that lymphedema, which can be caused by surgical or radiation treatment of the lymph nodes, is "not a fun thing." But the new standard of care is to use blue dye and a radiotracer procedure to find lymph nodes likely to be positive for cancer ("sentinel lymph nodes"), and treat accordingly.

Kim Tinkham
Kim Tinkham is a woman who saw The Secret, had stage III breast cancer, and declined all treatment. She now claims the cancer is gone, based on a quack blood test, even though the lump is still present, and has written a book about it. She is a follower of Mormon naturopath Robert O. Young, who claims that acid is the cause of all disease and alkalinization is the cure for everything. He says there is no such thing as a cancer cell, just a healthy cell spoiled by acid. Two years after her initial diagnosis, Tinkham is still alive.

Gorski pointed out that for a case like hers, expected survival for five years with treatment could be over 50%, but at ten years it goes way down. Data about untreated cancer comes from 250 cases of "large palpable tumors" from 1805-1933 at Middlesex Hospital in Connecticut. At 10 years, 3% were still alive, and at 15 years, 0.8% were still alive; the median survival rate was 2.87 years.

He noted that breast cancer biology is "highly variable in clinical behavior" and in some cases may be "indolent, slow-growing, and slow to metastasize."

The answers to the testimonial questions for Tinkham are:

• Was cancer definitely present? Yes.
• Did it go away? No.
• Was the alternative treatment the only one? Yes.

Time will no doubt soon tell how (in)effective this alternative treatment has been, unfortunately.

Testimonials as Conversion Stories
Gorski suggested that these testimonies are really part of "cult medicine" and seem to follow a pattern like religious conversion stories. The specter of death comes like a "bolt out of the blue," the person repents and says "I brought this upon myself," they face temptation in the form of standard medical care, they search for enlightenment, and then they find enlightenment in the form of some alternate description of their ailment which they then want to evangelize.

Michaela Jakubczyk-Eckert
Dr. Gorski concluded his talk with the story of Michaela Jakubczyk-Eckert (warning, graphic images), who was born on November 14, 1964 and died on November 12, 2005, just two days short of her 41st birthday. She had a T4 lesion eating through the skin of her breast, a case of "classic delayed diagnosis." She received neo-adjuvant chemotherapy treatment which shrank the tumor considerably, but then discovered Ryke Geerd Hamer, the inventor of German New Medicine, who argued that cancer is caused by psychological conflict rather than anything biological. She stopped her chemotherapy, and suffered a horrible relapse. As Gorski put it, she "died a horrible, horrible death" with her final days being subjected to the pain of a rotting-away body of skin and bones--a death far worse than chemotherapy. It was a vivid depiction of the alternative that cancer quacks can cause for their victims. Her husband has put up a website to try to dissuade others from being fooled by Hamer's theories (see link above to her story).

Dr. Gorski has written a blog post at the Science-Based Medicine blog on alternative medicine testimonials that covers some of the above subjects in more detail.

(Part three of my conference summary, on chiropractic, is here. Part four, on evidence-based medicine and homeopathy, is here. Part five, on chronic Lyme disease, is here. Part six, on online health and social media, and the closing Q&A panel, is here.)

Science-based medicine conference, part 1

This year's "The Amazing Meeting" (TAM7) was preceded by a Science-Based Medicine conference, organized by Steven Novella and the Science-Based Medicine blog; the speakers were all contributors to that blog. This summary is from my hand-written notes--I've tried to quote and summarize accurately, but keep in mind that some of the quotations and bulleted items may not be verbatim. The conference was videotaped and may become available via DVD or online video; keep an eye on the SBM blog for that. Steven Novella has posted a short summary of the SBM conference at the SBM blog.

I am in the process of posting a summary of TAM7 itself, which begins here. (I summarized 2008's TAM6 here.)

Novella's Introduction
The SBM conference began with Novella's presentation, titled "Science-Based Medicine: Science and Pseudoscience in Clinical Decision Making," which distinguished science-based medicine (SBM) from the recently popular "evidence-based medicine" (EBM) and explained the motivation for promoting the SBM concept.

Novella, assistant professor of clinical neurology at Yale University School of Medicine, began with a slide titled "Foundations of Medicine" which described some historical varieties of medicine:

• Superstition/philosophy-based medicine.
• Scientific medicine.
• Evidence-based medicine (EBM).
• Eastern vs. Western medicine.
• Complementary and alternative medicine (CAM).

He made the point that modern scientific medicine, which arose in the western world, is relatively young, though attempts to put it on a more scientific footing go back much farther.

Next, he discussed "What is science," first observing that "scientifically formulated" is a meaningless marketing label, then noting that there is a battle of memes about science between its defenders, who use it as a label connoting "objective truth, quality, and professionalism," and its critics, who use it as a label connoting "arrogance, oppression, and elitism." Marketers also use it "to imply product safety, effectiveness, and overall value."

He argued that we should adopt "common sense standards" which require that treatments "reasonably account for all available evidence," use "valid and internally consistent logic," have been rigorously and methodically investigated and judged with fair and unbiased criteria, and are conducted by practitioners who adhere to "standards of ethics and professionalism."

Evidence-Based Medicine
Next, he looked at evidence-based medicine, a term of art in use for the past couple of decades. EBM begins with the premise that "products and practices that work and are safe are better than those that don't work or are unsafe," an uncontroversial premise. It promotes scientific investigation as the measure of what works and is safe. But, according to Novella (and later, other speakers), EBM has "too much focus on evidence, and not enough on logic and prior probability, and good science must consider both." He argued that EBM made sense at the time it was introduced, because practices were being used largely "because they made sense, not because of supporting evidence." The introduction of EBM effectively "leveled the playing field, but also opened it up to implausible treatments," with bad timing due to the rise of complementary and alternative medicine (CAM).

The standard reference for evidence in the EBM framework is the Cochrane Reviews. The data reported in the Cochrane Reviews includes not only tests of legitimate medical treatments, but of completely implausible research such as chiropractic treatment for migraine. The problem with the framework is that it assumes that everyone is "playing fair," it does not account sufficiently for fraud or publication bias (such as the "file drawer effect"), it ignores prior probability, and it "doesn't adequately consider the big picture of the entire literature." According to Novella, with EBM it is typical to see the quality of studies decline over time, in order to continue to yield positive results for implausible treatments.

He then discussed "The Work of John Ioannidis," who argues that "most published research findings are false." The reasons are that

• The majority of highly-cited initial medical research is later refuted.
• There's a bias towards publishing positive studies.
• There's a bias towards researchers publishing provocative research.
  

And a "low prior probability worsens the effect," i.e., studies of treatments with low prior probability are more likely to be refuted.

Science-Based Medicine
By contrast with EBM, Novella identified the following features to distinguish Science-Based Medicine, SBM:

• It affirms high-quality science as a basis for standard of care in medicine.
• It acknowledges the consilience of science.
• It considered scientific medical plausibility of an intervention when weighing evidence.
• It considers the overall pattern in the literature.

In other words, SBM considers prior probability in a Bayesian sense as part of the evaluation, it looks at whether there is other scientific evidence that casts doubt on the plausibility of a suggested treatment (like violating the laws of physics or including unknown entities and mechanisms), in addition to merely looking at the specific results of controlled trials of the particular treatment. CAM, in particular, is loaded with claims that have extremely low and near-zero prior plausibility, as evidenced by the fact that $1.2 billion of U.S. taxpayer funding to the National Center for Complementary and Alternative Medicine since its founding in 1991 (originally as the Office of Alternative Medicine) has so far yielded zero effective treatments for anything.

He stated that finding anomalies argues for deeper research, and we can't have two inconsistent views that both work. Here, I think he overstated his case, since we have had cases in science where there are mutually inconsistent theories that both work, though we also take the inconsistency as grounds that something is wrong and ultimately needs to be reconciled (e.g., light wave/particle duality, quantum field theory vs. general relativity). Even theories that are wrong at some level can still work for solving certain kinds of problems (e.g., Newtonian physics)--and I'd agree with Novella's claim that scientific medicine is still in its infancy. A point Novella didn't make that I would like to insert here is that when you have two inconsistent views that doesn't mean that only one of them must be wrong--they could both be wrong.

Novella did go on to mention two cases where things that seemed initially implausible or lacking in mechanism have turned out to be correct, the postulation of dark energy in physics, and, more directly relevant to the topic at hand, the use of botulinum as a treatment for migraine. This treatment seemed to him completely implausible even though the evidence of trials suggested its effectiveness, and now a mechanism has been discovered and is understood. (My Google searching on this subject, however, yielded some recent evidence that it is not a good treatment for migraines and is no better than placebo, so this appears to me to still be somewhat controversial.)

Clinical Decision-Making
Novella ended his talk by talking about the process of clinical decision-making and pitfalls that arise as a result of human psychology and limitations. While clinical decision-making "individualizes the best available evidence to a specific patient" and "considers risk vs. benefit in both therapeutic and diagnostic intervention," this evaluation needs to include not acting as an alternative. In some cases, screening for certain diseases causes more harm than not performing the screening test, because conducting the test will yield far more false positives than true positives. (This is an effect discussed in some detail in John Allen Poulos' book, Innumeracy, and is a reason not to do things like mandatory HIV screening as a condition of a marriage license, drug testing of grade school students, and certain kinds of security screening for terrorists--if your baseline prevalence of what you're testing for is very low, your false positive results will swamp your true positive results.)

He briefly discussed the claim that "surgery kills more people than car accidents," noting that it doesn't really compare against the outcomes that would occur without surgery--far more deaths.

He then recounted some examples of pitfalls in the clinical context, such as the human capacity for pattern recognition even when the pattern isn't really there (pareidolia), the tendency to be "unduly influenced by quirky experience" or to "value experience over evidence," "failure to consider alternatives," "over-reliance on non-specific signs and symptoms," and confirmation bias (e.g., the sorts of heuristics and biases discussed in Kahneman and Tversky's classic Judgment Under Uncertainty). He then listed a few logical fallacies, pointed out the confounding factor of the placebo effect, and a couple of statistical effects--regression to the mean and the fact that most diseases are self-limiting.

Q&A
In the Q&A session, someone asked what Novella thought of legislation supporting evidence-based medicine, apparently referring to $1.1 billion in the stimulus package for evidence-based medicine research. Novella said that he thought conceptually it was a good idea but wasn't familiar with the specifics of the legislation. Another question was whether, given the current state of health care and the desire for reform, SBM would be challenged or supported. Novella said that the delivery of healthcare is a separate issue from how we decide what to research or what treatments are appropriate, and that things will either get much better or much worse. If he had also added that things might also stay about the same in overall quality, I'd say he's certainly correct; without it, merely probably correct.

(Part two of my conference summary, on cancer quackery, is here. Part three, on chiropractic, is here. Part four, on evidence-based medicine and homeopathy, is here. Part five, on chronic Lyme disease, is here. Part six, on online health and social media, and the closing Q&A panel, is here.)

NPR ombudsman on torture

About a week and a half ago, I heard NPR's ombudsman, Alicia Shepherd, defending NPR's policy on refusing to identify waterboarding as torture. Her argument was that NPR had a journalistic responsibility not to take sides on any issue, and that to identify waterboarding as torture was to take a side. She actually wrote that "I believe that it is not the role of journalists to take sides or to characterize things."

I think this is not only ridiculous, but an abdication of journalistic responsibility in favor of a bogus view of reporting "objectivity" by using only "he said, she said" descriptions, to an extreme.

Here's what I posted to the NPR blog on July 2:

There is no reasonable debate about whether waterboarding is torture. Waterboarding has been legally determined to be criminal torture by U.S. courts in 1947, when Yukio Asano was sentenced to fifteen years hard labor for it (among other war crimes). Other Japanese war criminals, such as Kenji Dohihara, Seishiro Itagaki, Heitaro Kimura, Akira Muto, and Hideki Tojo, were tried by the International Military Tribunal for the Far East for engaging in torture during WWII, including waterboarding, and several were executed for it.

U.S. soldiers who undergo waterboarding as part of SERE training receive that training in order to understand what torture is.

It is bad journalism to defend "there are two sides to every issue" as a form of phony objectivity. Sometimes there are more than two sides of merit, and sometimes there is only one (and there is *always* some nut who will take issue with any well-established claim). In this case, there is no reasonable argument by which waterboarding is not torture. It makes no more sense to call it "what some people refer to as torture" than it does to insert similar qualifications on the front of every noun used in a sentence on NPR.

Another commenter replied to point out that waterboarding has been legally torture for longer than that in the U.S.

I was glad to hear Adam Savage of Mythbusters, at TAM7, answer the question "what has been the biggest media failure of skepticism lately" by saying that the biggest failure has been the NPR ombudsman's statement that calling waterboarding torture is taking sides and they have to be "balanced."

Prestons win in court

The judge has ruled in Preston v. Hallman, and again it goes in favor of the Prestons. As expected, he ruled that the city's decision to revoke their permit was "arbitrary and capricious" since it did not meet any existing standard for denial.

The judge ruled that Tempe must re-issue their permit and allow their business to open. The city has 30 days to file an appeal.

I haven't heard how the judge ruled on the other issue, which I predicted might go the city's way, but it doesn't matter for the overall outcome--it was enough for the Prestons to prevail on either of the two issues, and they won on the one that they had a very strong case for.

The Goldwater Institute has issued a press release about the victory, and the Arizona Republic has run a story on it.

(Previously.)