Enter Your Email Address to Receive a
Copy of MedicExhange Member Demograhpics

Studies in a fairly new field have been greatly criticized recently within the Social Neuroscience field.  A lot of attention and interest was made by a paper written by Vul et. al. that sparked quite a controversy in the community. Ample amount of credit for writing such a paper was given as blogs, tabloid headlines, and even Newsweek featured the news.

MedicExchange had a chance to conduct an interview recently with Edward Vul, from the Department of Brain and Cognitive Sciences, a PhD student at the Massachusetts Institute of Technology, where he and his co-authors gained widespread coverage by simply questioning, “implausibly high correlations” between brain activity and emotions in data measured by functional magnetic resonance imaging (fMRI) within these studies. The widespread coverage in Social Neuroscience, in this recent study, is also privileged with ample amounts of attention mostly because of many high profile articles published in the press as well.  Edward has this to say regarding all the attention.  “Oh, it's obviously somewhat controversial: anybody reading the title will smell conflict and I suppose people in science are somewhat starved for conflict and drama,” he said.

But, why all this attention?  Why are scientists so concerned that the news of the accusations have spread so quickly.  “I think it has a lot to do with the catchy title and the understandable prose, which has resulted in what is commonly called the 'viral effect': any given person who sees the article passes it on to more than one other person, on average. Therefore, the few people to whom we sent it originally, sent it to more people, some of whom run blogs, and once it was on the blogs everyone seemed to see it,” Edward replied.

His and his co-authors paper, cleverly titled Voodoo Correlations in Social Neuroscience was quickly rebutted by certain scientists, some of whom currently have red-listed papers themselves. The paper used a questionnaire, to elaborate the methodological details, given to the authors of 54 papers in social neuroscience, as Vul et al. had concerns about the unusually high correlations within the studies.  Regarding Edward’s expectations relating to the use of this questionnaire,  “We were expecting to clarify the method sections of these papers,” Edward said succinctly.  The scientists, who comment with rebuttal suggest the  questionnaire, which was used to gather a substantial amount of information for the basis of rationale of certain studies, including their method sections, claimed no more than a few minutes to complete, and could hardly capture the rationale for the analyses used. According to Edward, the questionnaire was used to gather details of the statistical procedure that was used to produce the reported results, not the rationale for the statistical procedure.

A notable response to the paper, appropriately titled Response to “Voodoo Correlations in Social Neuroscience” by Vul et al. – summary information for the press also describes the questionnaire as being “ambiguous and incomplete.” The questionnaire “neither asked which correction method was employed nor whether a secondary statistical test was actually really applied to the data of the significant voxels.”  The response particularly criticizes Vul et al. for highlighting certain studies as "problematic" without discrimination in reference to non-independence errors.  In response, Edward commented,  “The problem is that fMRI data gives you many, many different measurements of different parts of the brain, called 'voxels'.
Somehow you have to choose which voxels you consider relevant for your analysis.  The non-independence error that we describe is the following two-step procedure: First, choose voxels that show a high correlation with some behavioral measure. Second, use those same voxels, along with the same data, to estimate the magnitude of the correlation.  Because fMRI provides so many noisy measurements, some will be more correlated by chance, and if you then look only at the high correlations, the aggregate measure of the correlation magnitude will be very high, but only because you initially selected the correlations that were very high, by chance. Thus, this procedure will end up artificially inflating correlations by preferentially
picking out favorable noise -- chance fluctuations that go in the direction of the hypothesis.”

The authors in the Response to “Voodoo Correlations in Social Neuroscience” and another version called Rebuttal of “Voodoo Correlations in Social Neuroscience” ultimately conclude that brain-behavior correlations  have been replicated through many analyses and findings in social neuroscience, not only by independent studies but by separate laboratories.  The authors further point out and strongly defend their position that these correlations are valid under the presence of good statistical practice.  The key words here, however, “good statistical practice”, draws our attention.  Is this not what led Vul et al to question the findings in the first place?

MedicExchange recently contacted many of the authors that wrote the rebuttal.  An invitation was given, but all declined the offer to join in a round table discussion amongst several of their peers to discuss my brief interview with Edward Vul.  They all appreciated my interest in this matter, however, none had any interest in further discussion of this issue outside the peer-reviewed scientific literature.  Some of my invitees, who critique the paper by Vul et al. neither feel the topic to be important enough nor are willing to devote more time to it than already has been done.  Some feel, the style and the way the paper was distributed was unprofessional, citing an attempt to maximize media attention and do not think this style of interface is appropriate.  Some of my invitees assured me that the critique will be dealt with in peer-reviewed literature in forthcoming papers by the scientific community.  The recommendation that I postpone a round table discussion was suggested only after a proper scientific dialogue occurs; not a dialogue by press and anonymous blogs who cannot evaluate the statistical claims made by Vul et al.  Popular opinion asserts that the way in which the paper is discussed does not support fair and suitable scientific manner.

The authors of Vul et. al. issued a statement acknowledging the authors critiqued and are pleased for the authors’ responses to the press.  The authors are looking forward to the opportunity to make their case in peer reviewed journals.

Audio Interview

MedicExchange had a chance to conduct an interview recently with Edward Vul, from the Department of Brain and Cognitive Sciences, a PhD student at the Massachusetts Institute of Technology ...

Download Embed Embed this video on your site