Wednesday, August 19, 2009

Brian Dunning on debate

In Skeptoid #167, Brian Dunning argues that scientists should never engage in debate on pseudoscientific topics. His arguments include:
  • It's a waste of time.
  • It gives pseudoscience undeserved credibility by putting it on an equal footing with science.
  • There are few people in the audience who haven't already made up their minds.
  • Most of the people in the audience can't distinguish good from bad arguments.
His position is similar to that of Eugenie Scott of the National Center for Science Education, who recommends that scientists not engage in formal debates with creationists.

While Dunning correctly points out some major flaws in how formal debates have frequently gone, and I agree that such debates should be discouraged, I think there are cases where they are worthwhile--it depends on the formulation of the resolution to be debated, the setting of the debate, and, perhaps most importantly, the quality of the debater. Too many creation/evolution debates have involved scientists who believe themselves to be good debaters, but who don't understand how debate works and aren't sufficiently familiar with creationist arguments to an appropriate breadth and depth. Unfortunately, many of those scientists think they won the debate or did a passable job when in fact they performed very poorly.

The resolution to be debated should be formulated so that there is a clear burden of evidence on the promoter of the pseudoscience, where it belongs. It's a mistake to formulate a debate resolution as a false dilemma, where if the scientist can't refute scattershot attacks, the pseudoscientist wins. Duane Gish of the Institute for Creation Research won most of his debates by not only engaging in such a shotgun approach (the "Gish Gallop"), but also by refusing to talk about the age of the earth or flood geology, thus freeing himself from having to present any positive evidence in favor of his view. (I spoke a bit more about Gish's debate success and how to successfully counter his debate strategies in my workshop session at this year's American Humanist Association conference.)

The setting of the debate is also important, and is relevant to Dunning's concern about audience. An academic debate at a university is more likely to have audience members who are actually interested in the evidence than, say, a debate at a church. It's also significant whether the debate is being recorded and will be distributed further--a well-done debate that is recorded and transcribed, and distributed in the form of a book, DVD, or online is going to have a much larger audience and may have much more significant consequences than the potential persuasion of five people in Dunning's example. There are also debates conducted in written form, which provide the possibility of much more comprehensive argument and references to other material than an oral debate on a stage or on television, which I think generally makes them preferable.

The concern about giving a pseudoscience proponent undeserved credibility is a real one, and for that reason it's probably a good idea for the debater to be someone of similar or lesser public stature, as well as someone well-versed in both debate and the details of the pseudoscience's claims. Proponents of pseudoscience often issue challenges to prominent individuals for the primary purpose of getting publicity from it, which they may get to some degree either from denial or acceptance--but much more from acceptance if they so much as appear to hold their own.

Dunning dismisses the concern that failure to debate leaves pseudoscience unchallenged, but I think there is a real potential concern here, as a refusal to debate can give proponents of pseudoscience a rhetorical weapon when there's the appearance that no one is willing to challenge their arguments. This can, to a large extent, be defused if you can point to resources that refute the proponent's claims in detail, and make the counter-argument that the proponents views aren't deserving of a public forum. But in cases where the proponent's views have received a large public following and there aren't comprehensive resources that refute them, or such resources are little-known, I think that builds a case for debate.

I think Dunning is right that it's generally better to produce direct responses to pseudoscientific claims in a one-way format, but even that can be a form of debate to the extent it actually engages the proponents and they respond. What's distinctive about a debate--at least a good one--is that it does involve engagement by both sides with the arguments and evidence of the other, and produces a record of that engagement for others to examine. That has advantages over siloed separate arguments that never directly respond to each other. I think that such engagement should be beneficial for scientists by identifying forms of misunderstanding that need corrections in the form of better communication, as well as locating possible weaknesses in their own evidence and arguments that need further work. It's also beneficial for the proponent of pseudoscience in that it puts them into a situation where they must, at least momentarily, think about the arguments and evidence against their positions.

16 comments:

Reed said...

Regarding the concern of leaving pseudoscience unchallenged, that looks to be a ripe opportunity for science-based skeptics to build-out online resources similar to how the talk.origins FAQ addresses creationist arguments.

Can we create new sites (or improve existing ones) to improve their quality, increase their visibility, improve their structure and navigability, and maintain them in a sustainable way to address the specific arguments of the pseudoscience proponents?

Might Wikipedia with its NPOV provisions to combat pseudoscience meet this need, or do we need something different that can be heavily cited by Wikipedia?

Schtacky said...

Jim said: “I think Dunning is right that it's generally better to produce direct responses to pseudoscientific claims in a one-way format, but even that can be a form of debate to the extent it actually engages the proponents and they respond.”

I’d agree as well, but even this technique has limitations beyond what you’ve pointed out here. With any form of argument with a proponent of pseudoscience, creationism, etc., my observation has been that no matter the format, presented evidence is often ignored and the responder will resort to the usual rhetoric. Your Zeitgeist posts are perfect example of this, i.e., you’ve presented evidence for your stance that in many cases is almost completely ignored and this being YOUR blog, it qualifies as a ‘one-way’ argument.

Finding someone with whom to debate that is *willing* to consider evidence remains the biggest gap.

Reed said...

(copied from my comment on FB on this article)

In his Where Do We Go From Here? essay, Daniel Loxton argues that skeptical activism shouldn't focus on scientific topics where better, more qualified experts exist (global warming, e.g.) Instead we should concentrate on reducing harm in our traditional topics (psychics, etc.)

So with regard to pseudoscience, if Dunning says that scientists shouldn't debate the cranks, and Loxton says that skeptics shouldn't do so either, then the cranks' arguments will go unanswered.

Jim Lippard said...

Reed:

To copy the rest of our Facebook discussion:

Jim:
Dunning says not to debate, but he may have had strictly formal debates in mind--though he wasn't entirely explicit. He advocated "one-way" communication, but didn't say that scientists shouldn't *criticize* pseudoscience.

Reed:
I finally got a chance to read his piece and agree that he appears to be talking about formal debates, or at least any forum where science is placed on equal footing with pseudoscience, esp where the audience is easily confused.

It's a tough problem where I see much gray area that Dunning has vastly oversimplified. You do want those big guns like ... Read MoreHitchens, Dawkins, & Shermer who can engage in debate (formal, television, etc.) and persuade an audience that would otherwise only hear the side of pseudoscience. Those skills aren't born naturally, but are built up from years in the trenches.

On a lesser scale, for gain familiarity and expertise on a topic, it's helpful to be challenged on it. Where will that come from if not some form of debate (online forums, etc.)?

Stan:
I wish I could debate like Steve Novella -- that guy is one cool cat.

Reed:
I saw Novella last year on the Believers vs Skeptics panel at Dragon*Con last year. Where the believers where mushy and non-specific in defending their views, Novella was clear and specific, pointing out the logical fallacies committed during the argument. If there were any fence-sitters in the audience, I doubt they were persuaded by the believer panel.

DanielLoxton said...

I actually don't mean to imply that skeptics can't tackle pseudoscience — I think "pseudoscience and the paranormal" describes our traditional domain. And, I also wouldn't wish to say that skeptics can't talk about straight science topics. We can, and I do: my first books are on evolution.

What I do think is that skeptics have limited qualifications and usefulness when commenting on straight science topics, and that this limits our appropriate roles.

Where both domain expertise and expert consensus exist, we can only report the consensus as straight science journalists. We can only make so much of a contribution here, simply because many other qualified science reporters already exist. And, while we're spending time being science reporters in a crowd of other science reporters, we're not spending time on our core mandate (pseudoscience and the paranormal) — a mandate no one else has.

Where domain expertise exists, but not consensus, we can report that a controversy exists — but we cannot resolve it. Skeptics sometimes stumble badly here: we cannot, as skeptics, responsibly wade into an area in which we are not expert and expect to settle expert controversies. If we're not qualified, we should not promote our opinions; if we are qualified, we should publish our contributions in the relevant peer-reviewed literature, not popular sources like skeptical magazines.

Where domain expertise and consensus exist, but also a denier movement or pseudoscientific fringe, skeptics can roll up their sleeves and get to work. This is traditional ground for us, our bread and butter, as when we combat creationism or vaccine paranoia or HIV denial. This is also, unfortunately, the area of greatest failing for skepticism. When skeptics position themselves in opposition to a expert consensus, as when we side with climate change deniers, we become the pseudoscientists. I mean that literally: when we reject established science from a position of little or no relevant domain expertise, we are indistinguishable from any other group of cranks.

Finally, there are many, many paranormal and pseudoscientific topics in which no experts exist — only enthusiasts. These topics, from Loch Ness to astrology to alien abduction, are the primary area where we should focus our energy. There are hundreds of topics under this umbrella, so it's not like we don't have enough to do!

Jim Lippard said...

Daniel: Thanks for the thoughtful and detailed comment. I mostly agree, except for a small difference with you on this sentence: "If we're not qualified, we should not promote our opinions; if we are qualified, we should publish our contributions in the relevant peer-reviewed literature, not popular sources like skeptical magazines."

If we are qualified, there's no reason we should be precluded from publishing contributions in both the relevant peer-reviewed literature and in popular sources like skeptical magazines.

I don't see any reason why popular skeptical magazines cannot both report on the existence of controversies and allow discussion on those controversies by proponents of those who hold a particular viewpoint. To the extent that there is a legitimate controversy, the skeptical magazine probably shouldn't take sides and should allow representation from advocates for multiple viewpoints, but why exclude everything but journalistic reporting on the controversy?

DanielLoxton said...

Jim: You're right, it can be very helpful for skeptics to get a ring-side seat to watch experts argue about this stuff. (For example, the scraps between Gould and Dawkins were very illuminating.)

But we need to be very careful when we do that in the skeptical literature. It isn't enough to run a single heterodox article in a skeptics mag, even by a domain expert. It isn't enough to run two or three point-counterpoint articles, especially if one of those is from a fringe source. That debate format can easily mislead readers into giving the fringe position undue weight. And, under all circumstances there must be a robust, long-term forum or feedback system so that readers exposed to the debate are also exposed to expert rebuttal.

The worst case scenario, though, is for a non-expert skeptic to take potshots at professional science from the comfort and influence of the popular sidelines — or argue that his or her personal incredulity trumps the actual literature on the topic.

At the end of the day, the skeptical literature cannot make contributions to expert science — it can only describe it. That description can, if we're careful, take the form of a debate…but we must be very careful indeed.

Jim Lippard said...

Daniel wrote: "The worst case scenario, though, is for a non-expert skeptic to take potshots at professional science from the comfort and influence of the popular sidelines — or argue that his or her personal incredulity trumps the actual literature on the topic."

I agree. And I think it's even a bit more complex than you describe when you say that "When skeptics position themselves in opposition to a expert consensus, as when we side with climate change deniers, we become the pseudoscientists." We can also become the pseudoscientists when we *attack* the fringe, which is a point that Ray Hyman frequently makes about skeptical treatment of parapsychology.

I recently inherited a copy of K. Ramakrishna Rao's anthology of articles, _Charles Honorton and the Impoverished State of Skepticism_ (1994, McFarland), which appears to argue that even the professional skeptics (including Ray Hyman) engage in non-scientific rhetoric and methods to argue for their positions. (The term "denier," BTW, is clearly a rhetorical strategy. Although I think it is often apt, it should also be recognized for what it is.)

Philip Kitcher, in his book _The Advancement of Science_, argues that what makes a pseudoscience is not the subject matter of a field, but the behavior of its practitioners.

Technically speaking, skeptical organizations are not really scientific organizations, they are public education and advocacy groups that promote science, as well as particular philosophical views about reasonable belief. So skeptical organizations should also be wary of purporting to be gatekeepers of what's science and what isn't, and individual skeptics who aren't professional scientists (or who are professional scientists addressing topics outside of their fields of expertise) need to take care not to misrepresent themselves.

Reed said...

"even the professional skeptics [...] engage in non-scientific rhetoric and methods to argue for their positions"

If ever this was an endemic problem, does it continue to be, at least among our professional skeptics?

One of the likely paths to the growth of the skeptical movement is through grass-roots efforts. It necessarily introduces a large influx of amateurs like myself. If our professionals had struggled with proper representation, imagine how difficult it will be for us amateurs!

DanielLoxton said...

Reed writes, "grass-roots…necessarily introduces a large influx of amateurs like myself. If our professionals had struggled with proper representation, imagine how difficult it will be for us amateurs!"

That's why I'm always pressing my radical getting-our-ducks-in-a-row agenda. Suddenly people are watching and taking cues from established skeptics. That places a Spider-man-type "great responsibility" burden on the "pros" to set a good example.

I argue in the upcoming SI that it's a sort of paradox: even as the grassroots leaves the old groups behind in many respects, the rise of the amateur movement makes professionalism more important than ever.

Jim Lippard said...

On further reflection, I think have broader disagreement. I think that it's possible to productively engage in subjects where one is not an expert, but an educated layman, at least when it comes to arguments in public forums where scientific information is being more broadly disseminated to the general public.

Sure, amateur skeptics who aren't experts in the field are unlikely to resolve or contribute to those fields, but they can contribute to the discussions in a broader public context. The risk of siding with crackpots and getting things wrong is certainly there and I don't mean to dismiss it, but I think skepticism has a role to play in critical thinking in general, how the layman can identify who the relevant experts are in these disputes, and so forth.

Doctor Atlantis said...

Lippard wrote: "Sure, amateur skeptics who aren't experts in the field are unlikely to resolve or contribute to those fields, but they can contribute to the discussions in a broader public context."

I guess one big issue is how does one define a non-amateur skeptic? And without engaging in debate how can one hone one's arguments? A public, oral debate might be a bad way to "learn" but engaging in forum discussions with believers can help one measure ones own understanding of the arguments and the positions.

I don't know how often it even comes up. I find it hard to resist the TBs who say, "skeptics never respond to X-study or Y-claim" and if I can dig up good replies I like to.

But I do wonder how one becomes anything but an amateur skeptic. (If we use the Olympics old measure of being paid-for-it, that's a small number of pro-skeptics.)

DanielLoxton said...

Jim Lippard wrote, "I think that it's possible to productively engage in subjects where one is not an expert, but an educated layman, at least when it comes to arguments in public forums where scientific information is being more broadly disseminated to the general public."

Whether we agree on this depends on what you mean. If you think skeptics can speak up as citizens and speak on behalf of their own personal opinions in public debate, we agree.

Or, if you mean skeptics should defend current science on the public stage while deferring to domain experts (essentially the journalists role) we agree there as well.

My position is that no skeptic should ever use the credibility loaned to them by their skeptical work to publicly promote authoritative-sounding views on topics which are either a) subjective and value-based, or b) outside their area of expertise.

There's no wider road to pseudoscience that the conceit that expertise in one area entitles one to speak as an expert in other areas.

Neural Gourmet said...

Jim,

I guess my position on this is that the debunking role of skeptics is actually rather secondary. It's important for us, in that it allows us to exercise our skills and interests and it does sway some people but overall it's not terribly effective. We know this courtesy of research by people like Brendan Nyhan and Jason Reifler. Counter-intuitively, debunking actually seems to reinforce misperceptions [PDF]. At least that's so in politics, but personal observation tells me it applies to other fields as well. What's more important I think is giving people the tools to do critical thinking, to ask questions that give meaningful answers and sieve the actual experts from the hucksters, conmen and bullshitters.

But it does involve educating ourselves in areas where, traditionally, skepticism has feared to tread and where Dan thinks we should still avoid. Don't get me wrong. Dan is right in saying that the same old pseudoscience and paranormal BS is never going away and we still need to concern ourselves with that. Dan's also right, I think, that we need to be more professional in the way we tackle subjects, of which picking our battles is a valuable skill.

I just don't think we need to constrain ourselves to fringe topics. Non-experts can, at least, ask the right questions, educate and advocate for science and critical thinking. If it were otherwise, science, indeed all, journalism would be in an even more dismal state than it is now.

Jim Lippard said...

Daniel: I think I agree with what you say, except that there is probably no way to prevent people being given undue credibility or authority if they are well-known in some field and speaking out about another, except to keep repeating that point and get people to recognize it and be aware of it.

Neural Gourmet: That phenomenon occurs outside of politics, e.g., a study of patients given a document on "vaccine myths" found that in later recall they remembered the myths as true.

And older people are more likely to be persuaded to believe in misinformation on the basis of misleading cues, as might be given by a fraudster.

DanielLoxton said...

Jim Lippard wrote, "there is probably no way to prevent people being given undue credibility or authority if they are well-known in some field and speaking out about another, except to keep repeating that point and get people to recognize it and be aware of it."

Yes, I think you're right there.

In my view, we need to work toward a cultural norms for skepticism in which the community finds it inappropriate for skeptics to do certain things. Intruding inappropriately in other domains should bring voices of protest.


Neural Gourmet wrote, "the debunking role of skeptics is actually rather secondary."

I see we have a great deal of overlap, but I think this distinction is a point of genuine disagreement.

In my view, the critical investigation of pseudoscience and the paranormal is the only unique thing skepticism has to offer.

After all, many other fields (science communication, journalism, etc) prize science literacy or critical thinking. If those were primary goals, why bother having something called "skepticism"?