NSA Is Building a New Datacenter In San Antonio 119
An anonymous reader writes in with an article from a Texas paper on the NSA's new facility in San Antonio. "America's top spy agency has taken over the former Sony microchip plant and is transforming it into a new data-mining headquarters... where billions of electronic communications will be sifted in the agency's mission to identify terrorist threats. ... [Author James] Bamford writes about how NSA and Microsoft had both been eyeing San Antonio for years because it has the cheapest electricity in Texas, and the state has its own power grid, making it less vulnerable to power outages on the national grid. He notes that it seemed the NSA wanted assurance Microsoft would be here, too, before making a final commitment, due to the advantages of 'having their miners virtually next door to the mother lode of data centers.' The new NSA facility is just a few miles from Microsoft's data center of the same size. Bamford says that under current law, NSA could gain access to Microsoft's stored data without even a warrant, but merely a fiber-optic cable." The article mentions the NRC report concluding that data mining is ineffective as a tactic against terrorism, which we discussed a couple of months back.
Type II error (Score:1, Informative)
Of COURSE data mining is ineffective.
Consider the following analogy. A company creates a test that is 99.9% accurate to detect a rare genetic disease. 1 person in 10,000 has the disease.
Let's say your test comes back positive. You should be worried, right? I mean, 99.9% accuracy, and you came back positive.
Actually, no. Let's say you test 1,000,000 people. Of those, 100 will actually have the disease, and 999,900 will not. With 99.9% accuracy, you'll see:
* Of the 999,900 people who do NOT have the disease, about 1000 people will incorrectly test positive.
* Essentially all of the 100 people with the disease will also test positive.
1,100 people tested positive. Only 100 of those have the disease. This means that, even of the people who test positive, 91% do NOT have the disease. Statisticians call this kind of problem is called "Type II Error", which is a major problem for detecting rare conditions in a large population, even with a very accurate test.
Why does this relate to NSA data mining? Even if you're paranoid, the number of terrorists operating in the US is very small. Even if we concede that NSA data mining/profiling is very accurate (something I personally don't), it will STILL be the case that the vast number of identified individuals will be "false positives."