Thursday, March 1, 2012

Sensitivity and Salmon

This week we talked briefly about sensitivity and predictive value, and reviewed a fantastic poster by Bennett et al. Here is methods section, in toto:



Let's just translate this briefly.  What Bennett is saying is that he bought a dead salmon to calibrate his fMRI machine, and that, like a good scientist, he performed the entire experiment he intended to run on humans on the salmon in order to control for all potential unmeasured variables.

Reminding us once again of the striking graphic efficacy of fMRI results, here's a slice from his scan:



As you can see, the salmon appears to be...ah...."mentalizing."

Bennett is to be commended in publishing this cautionary tale, whcih reminds us that high sensitivity is not always a good thing.  In addition, this is about the best illustration I can think of to explain why predictive value depends on prevalence.  If you're testing whether a dead fish has the capacity to perform a "mentalizing task," then all results are false positive results, because, unless fMRI counts among its widely trumpeted virtues the power of resurrection, dead salmon don't mentalize.

2 comments:

  1. I don't get it... what is "mentalizing" and how does this image show the fish doing it. So to re-translate what you said bennett et al said. The fMRI machine is suppose to show what parts of the brain light up (are activated) with different stimuli (ie. the amygdala will light up in response to a fearful picture). He calibrated him fMRI machine with a dead fish. A dead animal should have no blood flow therefore should not reveal anything on the fmri machine however in this study the fish brain lit up. Therefore you are saying the point of this is that fMRI are useless or what exactly are you saying...

    - Dr. Peoples

    ReplyDelete
    Replies
    1. I'm saying two things:

      1) The fact that he detects blood flow in a dead salmon means his machine's thresholds for detection are set too high; i.e., the machine is too sensitive. Obviously, it is not the case that all fMRI machines are NECESSARILY too sensitive, since they can be calibrated, but it should give us pause before unquestioningly accepting some Rorschach-esque picture of an amygdala as evidence that, say, PTSD is a purely neurologic event.

      2) I'm using this to point out why predictive value (positive or negative) depends on prevalence. The point is that when the prevalence of a condition (in this case, consciousness) in a given population (in this case, dead salmon) is zero, any positive test result will be a false positive, regardless of the sensitivity and specificity of the test. Likewise, in a population (say, people between the ages of 19 and 23 on the Las Vegas Strip at 23:45 on December 31st) where the prevalence of a condition (say, drunkeness) is very high, negative results are more likely to be false negatives. These are extreme examples, but you can see how the probability of a positive or negative result being accurate will vary continuously according to the background prevalence of the condition.

      Delete