The trade-off between Type I (false positive) and Type II (false negative) errors is one of the fundamental laws of nature. High sensitivity comes at the steep price of a low sensitivity and vice versa. Where along the trade-off curve you position yourself depends on your utility function. While it might seem as only an arcane statistical/machine learning topic, it is in fact ever-present in our lives. A typical example is the security vs. freedom or security vs. privacy debates.
In this discussion, Cheney is defending the use of
enhanced interrogation tactics torture in reaction to the recent Senate’s report. At a critical point of the interview, he states that he believes that national safety required a high-sensitivity (and hence tragically low specificity) policies:
I mean, let me go to Gul Rahman. He was chained to the wall of his cell, doused with water, froze to death in C.I.A. custody. And it turned out it was a case of mistaken identity.
–right. But the problem I had is with the folks that we did release that end up back on the battlefield. Of the 600 and some people who were released out of Guantanamo, 30% roughly ended up back on the battlefield. Today we’re very concerned about ISIS. Terrible new terrorist organization.
Not only is the trade-off here demonstrated in a crystal-clear form, but it is also a situation of highest stakes – the choice of the trade-off point is set by the potentially most important utility function a society has – its national, democratic and civic values as interpreted by its policy-makers.
Update 12/01/2015: Thirtyfiveeight have an excellent article on the costs of Type I errors in cancer treatment.
 The interview is very interesting (if I am allowed to say this given the tragic nature of it) from many aspects. Cheney seems to see himself and a hard decision-maker, willing to do unpopular decision for the greater good of his nation. The terrifying part is, that it seems that doubt never seems to play any part in his epistemology.