Voodoo Science

Jerry Pournelle wrote a piece many years ago called "The Voodoo Sciences". He was first and foremost concerned with defaming economics, a task that is now completed. However, he also included many of the social sciences in his definition. I thought the essay was interesting, but I think it probably could have used a more developed epistemology. I was just reading the chapter of "The Philosophy of Tolkien" by Peter Kreeft on epistemology, so perhaps that got me thinking about this.

The more proximate reason for thinking about this is a paper that has been much discussed of late. I came across this little gem on John Reilly's website.

I try to be charitable, but I can't think of a single widely publicized neuroscience study that did not strike me as simple-minded or an outright scam. Now I learn that I have not been entirely alone in these suspicions:

Late last year, Ed Vul, a graduate student at MIT working with neuroscientist Nancy Kanwisher and UCSD psychologist Hal Pashler, prereleased "Voodoo Correlations in Social Neuroscience" on his website. The journal Perspectives in Psychological Science accepted the paper but will not formally publish it until May.

The paper argues that the way many social neuroimaging researchers are analyzing their data is so deeply flawed that it calls into question much of their methodology. Specifically, Vul and his coauthors claim that many, if not most, social neuroscientists commit a nonindependence error in their research in which the final measure (say, a correlation between behavior and brain activity in a certain region) is not independent of the selection criteria (how the researchers chose which brain region to study), thus allowing noise to inflate their correlation estimates. Further, the researchers found that the methods sections that were clearing peer review boards were woefully inadequate, often lacking basic information about how data was analyzed so that others could evaluate their methods.

This study caused something of a stir. It has been rebutted, and the rebuttals rebutted, and doubtless the surrebuttals rebutted. I have not studied the arguments in detail, but I suspect that neuroscience will be deflated too soon to become the Freudianism of the 21st century.

I have written a paper myself on this subject, but from a philosophy of nature perspective rather than a statistical perspective. I also find fMRI studies a little suspicious, especially when some insight into the mind is claimed rather than simply a correlation of blood flow in the brain with some observable activity. Philosophy of mind is a field currently characterized by its variety and rancor. Most neuroscientists probably don't even imagine that viable controversies remain. At the very least, the one who was the cause of my term paper did not. I read a good introductory work on philosophy of mind by one of my favorite philosophers, Edward Feser. Classical and Medieval philosophers were a great deal more sophisticated on philosophy of mind than one might imagine. Modern philosophers of mind exhibit a great deal of creativity in their disputes with one another. However, pretty much everyone these days assumes that the viable alternatives are either materialism, or Cartesian dualism, which is not even remotely true, even in contemporary philosophy entirely innocent of anything before the 16th century.