Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Privacy United States

NSA Utah Data Center Blueprints Reveal It Holds Less Than Thought 197

cold fjord writes "Break out the tin foil hats, and make them double thick. Forbes reports, 'The NSA will soon cut the ribbon on a facility in Utah ... the center will be up and running by the "end of the fiscal year," ....Brewster Kahle is the engineering genius behind the Internet Archive,... Kahle estimates that a space of that size could hold 10,000 racks of servers .... "So we are talking $1 billion in machines." Kahle estimates each rack would be capable of storing 1.2 petabytes of data. ... all the phone calls made in the U.S. in a year would take up about 272 petabytes, ... If Kahle's estimations and assumptions are correct, the facility could hold up to 12,000 petabytes, or 12 exabytes – ... but is not of the scale previously reported. Previous estimates would allow the data center to easily hold hypothetical 24-hour video and audio recordings of every person in the United States for a full year. The data center's capacity as calculated by Kahle would only allow the NSA to create archives for the 13 million people living in the Los Angeles metro area. Even that reduced number struck Internet infrastructure expert Paul Vixie as high given the space allocated for data in the facility. ... he came up with an estimate of less than 3 exabytes of data capacity for the facility. That would only allow for 24-hour recordings of what every one of Philadelphia's 1.5 million residents was up to for a year. Still, he says that's a lot of data pointing to a 2009 article about Google planning multiple data centers for a single exabyte of info. '" Update: 07/25 16:33 GMT by T : For even more, see this story.
This discussion has been archived. No new comments can be posted.

NSA Utah Data Center Blueprints Reveal It Holds Less Than Thought

Comments Filter:
  • Saving face (Score:5, Insightful)

    by Anonymous Coward on Wednesday July 24, 2013 @11:39PM (#44377807)

    Expect more articles like this that downplay the scale of the NSA.

    • Re: (Score:1, Troll)

      by Capsaicin ( 412918 ) *

      Expect more articles like this that downplay the scale of the NSA.

      Yeah it's a mere 12 exabytes (and of course Moore's law won't apply ... ahem), on us. Nothing to see here kids, just move along.

      • Re: (Score:1, Insightful)

        by cold fjord ( 826450 )

        To be fair, Paul Vixie thought it was more like 3 exabytes. The NSA has world wide responsibilities for all sorts of signal intelligence. I would guess that purely domestic data would be a minor part of it. No sense being narcissists about it, not everything is about "us."

        • No sense being narcissists about it, not everything is about "us."

          I'm not in the US. I'm a narcissist who plans overthrowing the old word order from my island bunker somewhere in the South Pacific ;) (.au). By 'us' I meant humanity.

          • I'm a narcissist who plans overthrowing the old word order from my island bunker somewhere in the South Pacific ;) (.au).

            LOL. Personally, I like the sound of that. I really must look into getting one of those island bunkers for myself. ;) Of course, maybe a cabin in the outback will do until I can afford it.

            I'm already practicing my evil laugh [youtube.com].

        • by AHuxley ( 892839 )
          "world wide responsibilities" where mapped out and taken care of during the cold war. The USA has bases, camps, forts, agreements as needed around all the nations it wishes to 'contain'.
          Any global gaps in "signal intelligence" where fixed long ago.
          The "domestic data" aspect is new and seems to need a new location, a new vetted domestic workforce, lots of cooling water and power supply.
          The USA dislikes huge raw encrypted movements around the world.
          The USA likes to get the data in bulk, work on it nea
        • Re: (Score:1, Insightful)

          by Anonymous Coward

          To be fair, anyone looking at the declassified floor plan ought to ask about the basement that was left out of their version of the blueprint.

          None of the people involved in this article have seen the actual facility, nor the actual plans for the facility.

      • and they haven't even taken into consideration compression tellaphone has really low audio quality so it should take that much space when compressed

        • by Anonymous Coward
          If we assume 32 kb/s for telephone, then 272 PB they give in the summary would only be 5 minutes per person in the US per day. Even if you drop that down to 8 kb/s, that is only 20 minutes per person, per day. Doesn't seem that far off.
        • Re: (Score:1, Interesting)

          by Anonymous Coward

          and they haven't even taken into consideration compression tellaphone has really low audio quality so it should take that much space when compressed

          Or if its processed to transcript and stored as text .. then deduped .. then compressed .. and who said they were using magnetic media? Or only had above-ground capacity?

          Pretty unimaginative to assume that this is a giant storage node ..

        • and they haven't even taken into consideration compression tellaphone has really low audio quality so it should take that much space when compressed

          Even without storing actual conversations the knowledge of whom you call, how often, when from where etc. for most people connected to telephone systems in the civilised world is an amazingly powerful tool for profiling individuals. People worried about 24 hour video of them miss the point. In fact, such info might be preferable as it would lower the chances

      • by Anonymous Coward on Thursday July 25, 2013 @12:00AM (#44377911)
        Never mind that the annual production of hard drives is about 100 million drives. If they were all on the order of 5 TB, 10 EB would represent 2% of the global hard drive market for a year. Annual tape production is actually very similar order of magnitude to the annual hard drive production, so it is not like tapes gain you much. At least this is more reasonable than the estimates that previously were in the zetabyte range that would have to assume they had ten years worth of hard drive and/or tape production at current storage density.
    • Re:Saving face (Score:5, Interesting)

      by Anonymous Coward on Wednesday July 24, 2013 @11:47PM (#44377831)

      This was submitted by cold fjord, Slashdot's resident neo-con who supports waterboarding, said the Iraq war was "worth it", and said Bradley Manning deserved to be tortured for 'faking' feeling suicidal. What do you expect?

      Oh, and this facility will "only allow for 24-hour recordings of what every one of Philadelphia's 1.5 million residents was up to for a year". It is convenient that the article fails to mention that this is only one facility out of a dozen or so.

      • This was submitted by cold fjord, Slashdot's resident neo-con who supports waterboarding, said the Iraq war was "worth it", and said Bradley Manning deserved to be tortured for 'faking' feeling suicidal.

        Oh please! It's nothing that elaborate. All indications are that he has been hired to write this stuff, which really smells of professional marketing. He's damn near a 'bot.

        • by Anonymous Coward

          Be careful; cold fjord might reply to you and spam a million links that don't demonstrate that security is more important than freedom and probably aren't even relevant.

          • Hell, I already friended him. I think he's funny.

          • Re: (Score:2, Troll)

            Be careful; cold fjord might reply to you and spam a million links that don't demonstrate that security is more important than freedom and probably aren't even relevant.

            Worse, As all good propaganda mouthpieces should have - Cold Fjord appears to have an army of accounts/mod points to blast your karma to kingdom come, fairly easy to spot when you check the details/timestamps. I wonder what his exact relationship is with slashdot editors is, given his record getting obvious propaganda placements posted by them. Perhaps there is a gag order covering that too...

        • Nah, if he was in disinfo he'd be posting as cold fnord, so no one would notice him.

        • He's damn near a 'bot.

          Not a bot, just "cold." ;)

      • ... said Bradley Manning deserved to be tortured for 'faking' feeling suicidal. What do you expect?

        At the moment I'm coming to expect that you are pathologically unable to accurately relay factual information.

      • Re: (Score:1, Offtopic)

        "This was submitted by cold fjord..."

        Lets take a close look at what "Cold Fjord" has been up to for the last week.

        July 24, 2013 Posting from 11:06am --- 11:25pm Total Posts:13 Submissions:1 Longest Break from Slashdot:8hrs

        July 23, 2013 Posting from 12:09am --- 11:46pm Total Posts:30 Submissions:2 Longest Break from Slashdot: 4hrs

        July 22, 2013 3 Posts Total, 1 in the AM, 2 in the PM DAY OFF (no links in posts-posting from a phone?)

        July 21, 2013 Posting from 8:54am --- 10:29pm Total Posts:18 Submissi

        • I challenge you to delve into my comment history, go on do it. You will find significant periods in my comment history where I am *substantially* more active than the examples you give for Cold Fjord.

          Am I a paid forum-manipulator? I wish. It's trivially easy to post a lot of comments even when doing a full days work for an employer.

    • Expect more articles like this that downplay the scale of the NSA.

      I can think of arguments for the NSA wanting to overplay its capability and also downplay it.

      • Expect more articles like this that downplay the scale of the NSA.

        I can think of arguments for the NSA wanting to overplay its capability and also downplay it.

        It's a conspiracy either way!

        (Only Goldilocks can be trusted.)

    • Re:Saving face (Score:5, Interesting)

      by Anachragnome ( 1008495 ) on Thursday July 25, 2013 @01:41AM (#44378325)

      "Expect more articles like this that downplay the scale..."

      Downplay the scale? We haven't even seen the drawings for the below-ground facilities.

      But, seriously. From the article...

      "...and that the sheer size of the data centers in Utah and elsewhere suggests that the agency wants to vacuum up everything it can..."

      That's my emphasis--plural. There are more then one of these centers. Take a look at the layout of the Utah Data Center article at Wikipedia.
      http://en.wikipedia.org/wiki/Utah_Data_Center [wikipedia.org]

      Does that building layout look anything like the one at the top of the linked Forbes article? The picture of the buildings and the layout right above are a match in the Wikipedia article, yet they don't match the plans in the Forbes article.

      So where is this data center that Forbes has the plans to? They're obviously not the same.

    • Perhaps this one is just to monitor everyone in Utah?...

      or.... A datacenter for each man woman and child in the US... that's exactly the way a beurocrat in charge of datacenter construction would think.

    • by CAIMLAS ( 41445 )

      Yep.

      And why do we keep assuming that the NSA is wasting the time recording hours upon hours of audio, and that's how much storage is required?

      Google has records of every phone call made through their system, undoubtedly. They have transcripts of all voicemails. They don't (necessarily) record all calls. Why wouldn't the NSA bother doing the same thing? This is relatively trivial today.

  • by maynard ( 3337 ) on Wednesday July 24, 2013 @11:48PM (#44377839) Journal

    This is a vast amount of storage. Obviously, the puzzle they've bought a data palace of a storage facility to assemble doesn't require indefinite storage for everyone. They're looking to cache everything they can get and then filter what's interesting. Maybe they have a range of target levels from indefinite storage of everything collected for one group, a year for another group, a month for a third group, a week for another, all the way down to a day or hours for the entire slush.

    They don't need it all. They just need to run whatever algorithms they care about so they can toss whatever they think doesn't matter and keep what does.

    • by AHuxley ( 892839 ) on Thursday July 25, 2013 @12:08AM (#44377945) Journal
      The next step for the NSA is a small file for every human with enough space for a days internet links, chats, text for life.
      That can bed expanded as they get politically active :)
      The file per person would allow any persons digital life to be tracked back to the first 'connection' of interest.
      In the past all that could be done was to track telephone numbers, fax, computer use and voice prints as found or via contact with a past person or group of interest.
      The past sorting was very quick and left a very small amount of data to be sent to the US from any distant super computing location (UK, Australia)
      ie the NSA is not after http://www.wired.co.uk/news/archive/2013-06/24/gchq-tempora-101 [wired.co.uk] long term.
      They don't want 'big' content long term, they need space for all your ip's used, ports, apps used keywords, links, times, locations, connections to people - all very tiny amounts of text like info for now ie the "initial filter" will go for your pic, movie, sound, text - not keeping it, but might give a facial recognition code string to everybody in the pic. You only need a good voice print every so often...
      Data size has never been the issue, legality, domestic commercial 'help' have been.
    • Why the geographical comparisons?

      I expect that it is because it makes it easier for people to relate to the enormous numbers being talked about while both innumeracy and illiteracy [theweek.com] are a problem.

      I've used an example like that myself to explain to people why the lottery and gambling are nothing to pin your hopes on. (When they draw the ticket, it will be like randomly picking 1 person west of the Rhine/Mississippi to win. Are you that person?)

    • by qbast ( 1265706 )
      Problem with this approach is that you don't know what will get defined as 'interesting' in 5 years. You filter out non-interesting data today and you won't have it when it becomes interesting again.
  • Ummmm.... (Score:2, Insightful)

    by Anonymous Coward

    You do know they lie for a living, right?

  • Why 24/7? (Score:5, Insightful)

    by webdog314 ( 960286 ) on Thursday July 25, 2013 @12:06AM (#44377939)

    I mean, sure, you could record a few million people sleeping for eight hours a day, or watching 4 hours of Simpsons reruns a night, but why? If you're recording the 1-2 hours most people spend on the phone a day (max), then 3 exabytes might actually work out okay.

    • Re: (Score:3, Informative)

      by Anonymous Coward

      General Cole is quoted as saying (I paraphrase slightly), "To find a needle in a haystack, it takes a very large haystack." And a very big computer, I might add. And that single needle will be next to impossible to understand, drowned out by the noise around it generated by all the surrounding similarly shaped needles that aren't quite as shiny, that *don't* stand out.

      The more frightening the headlines about the size in exabites of aggregate data in a data center, the more secure we should feel. Peterson's

  • A billion dollars they're spending. The NIH, the people who fund research that is going to cure cancer, they had their funding cut about 1.5 billion.

    Hey, NSA! I'm thinking highly unpatriotic, violent thoughts right now!
    • by Nyder ( 754090 )

      A billion dollars they're spending. The NIH, the people who fund research that is going to cure cancer, they had their funding cut about 1.5 billion.

      Hey, NSA! I'm thinking highly unpatriotic, violent thoughts right now!

      Well obviously Terrorist kill more then cancer.

  • Even that reduced number struck Internet infrastructure expert Paul Vixie as high

    My uneducated response was "Holy Fuck!". Lucky the experts were there to clarify.

  • Sure, it would require a ridiculous server farm to store *recordings* of every phone call placed in the US, much less worldwide. Add emails, texts, IMs, etc., and the NSA would send hard drive prices through the roof all by themselves.

    But phone *records* are another thing entirely. To store a record of every phone call (timestamps, caller number, recipient number, and maybe GPS) would only take roughly 30 TB a year (@ 500,000,000 calls placed each year). That's only about 2U worth of well-stocked NAS.

    The fo

  • by Black Parrot ( 19622 ) on Thursday July 25, 2013 @01:40AM (#44378315)

    "NSA Utah" is an anagam for "anus hat".

    • Alex Trebek: I'm sorry Mr. Connery, but your answer must be in the form of a question.

      • From the Gentleperson's Guide to Forum Spies:

        Technique #3 - 'TOPIC DILUTION'

        Topic dilution is not only effective in forum sliding it is also very useful in keeping the forum readers on unrelated and non-productive issues. This is a critical and useful technique to cause a 'RESOURCE BURN.' By implementing continual and non-related postings that distract and disrupt (trolling ) the forum readers they are more effectively stopped from anything of any real productivity. If the intensity of gradual dilution is i

  • by nbritton ( 823086 ) on Thursday July 25, 2013 @02:15AM (#44378433)

    The claim that a years worth of phone calls is around 272 petabytes is dead on, it matches up perfectly with some back of the napkin calculations I did a while back based on a published report from the FCC[1]. Depending on the encoding bitrate, the range I had was 107 PB for 8 Kbps audio to 430 PB for 32 Kbps audio. 272 PB is about 20 Kbps, exactly in the middle...

    http://slashdot.org/comments.pl?sid=3871487&cid=44027425 [slashdot.org]

    [1]: http://transition.fcc.gov/Bureaus/Common_Carrier/Reports/FCC-State_Link/IAD/trend605.pdf [fcc.gov]

    The report only documents up to year 2000, but I presumed POTS service had leveled out with the emergence of VOIP and SMS messaging.

  • Obviously the previous reports were wrong. Anybody familiar with computers and storage space knew that the numbers reported by NPR and other "news" outlets were ridiculous. They were saying that the center would hold 5 zetabytes, and would only cost $1.2 billion! That's about 25 cents per TB.

    Best I could tell, NPR et al misunderstood a Wired article from over a year ago. In the Wired article, somebody said that they would eventually like the processing power in the center to exceed 1 exaflops, and then mayb

  • Depends.. (Score:5, Interesting)

    by buss_error ( 142273 ) on Thursday July 25, 2013 @03:07AM (#44378637) Homepage Journal

    On what is kept. If it really is just the metadata and not the conversation, then the storage requirements are not all that large.

    For Landlines, there is a unique identifier applied at the switch. I mis-remember what it's called, but in South Texas, it usually started with BAPA- blah blah blah for several digits.

    For cell phones, there is the OMEI/UDID/ESN. Normally around 14 to 20 digits, usually 15.
    Next, called number, same info.
    Last, call duration.

    I believe it's long been known that using particular words in a telephone conversation would raise a flag. I don't know if that's true or not. If so, lets consider this scenario:

    Call metadata captured and stored - always.
    Call voice session saved to a temporary storage area.
    Call concludes.
    Voice data is analyzed for key words using automation. (Think about when you call your credit card company, and can input your CC number by voice)
    If no keyword flags are raised, delete the conversation after X time (or immediately, who knows?)
    If keyword flag is raised, score by number of keywords, flag conversation for human review, preserve all data.
    After human review, who knows?

    What I think: If preserving our freedom comes at the price of invading all of our privacy, then the terrorists have been gifted with a victory they could have never won for themselves. We have destroyed our freedom with the illusion of security, and now have neither freedom nor security. To draw a parallel, how is having the TSA able to squeeze my balls protecting me? "Dude - don't touch my junk!"

    • Damn, no points to mod this up "insightful"

      Maybe the kind of smoke and mirrors in TFA works on the uneducated and/or uninterested masses, but please don't try to bullshit a room full of engineers and scientists.

    • by ModelX ( 182441 )

      Voice data is analyzed for key words using automation. (Think about when you call your credit card company, and can input your CC number by voice)
      If no keyword flags are raised, delete the conversation after X time (or immediately, who knows?)

      You forgot one important step: voice data is converted to a very low bitrate phoneme-like representation that is good enough for subsequent approximate searches and voice based analytics (speaker recognition...).

    • Time to routinely use RedPhone and TextSecure. I've been using the latter, and it's actually very nice. I've yet to find a friend doing the same though :-(

  • There is a saying: "a translator always translates as it is profitable for the translator".

    The same is with this data eavesdropping and collection. It may be used to collect and trade commercial secrets in order, well, to gain money.

    The people, who open other people letters, were always considered sneaks and dastards. That is why it has always been necessary to obtain the specific court decision for each sustect to do it.

    Was it really necessary to change it? The damage, which this global carpet-eav
  • Did the author of the summary read the article? The article for some reason mentions individualized video feeds for every American which is unrealistic and nothing like the sort of thing anyone has said the NSA is recording. 12,000 PB is far, far larger than the 272 PB estimated to hold all US domestic phone calls for a year, plus the foreign and international calls (which people forgot the NSA captures).

    I recommend people read the archive.org description of the problem of archiving phone calls [archive.org] (TL;DR 2
  • About once a year I send an email to my paranoid friends which includes a few buzz phrases.

    Dear Spooks,

    It is once again time for me to provide you with an update on nefarious activities on the Wild Wild Web.

    While you are clandestinely surveilling me through your prism of delusion, why not take a moment and stand back and ask your self; is what you are really doing protecting liberty or slow chiselling it away.

    Have a good.

  • by Anonymous Coward

    Couldn't all phone calls be converted to text and as a result require much, much less storage space?

  • by Bucc5062 ( 856482 ) <bucc5062@[ ]il.com ['gma' in gap]> on Thursday July 25, 2013 @07:47AM (#44379761)

    I read through many of the posts, the exchange between "cold fjord" and Aca something was cute with its little drama about paid writers (maybe their both paid writers for the NSA or other government agency). yet in all these posts, not one poster talked about the root of this article. Why would the NSA *need* all this space if is not suppose to be collecting information without specific warrants or in bulk against innocent citizens.

    That is the story. It is like y'all have just rolled over and accepted that it is okay for the NSA to even do this, so let's argue about size. My own view is that the NSA does *not* need these data centers for they should not be collecting that much information about everyone in the USA and beyond. I listened to a politician this morning (one who voted to continue funding the NSA's current trawling expedition) tell me that their actions "saved" hundreds of American lives, but if I asked for proof he'd say "I cannot disclose that information". I see, so you can't provide facts on what the program has done to save lives, you can't talk about what the program does though we know it gathers information on people who are not related to any illegal activity, and you ask us to "Trust You"? This is a republican who cries out for spending cuts, but votes to continue funding secret projects.

    Please...

    The spotlight on the NSA is not what it is building, it is on what it is doing, allegedly breaking the law. We should be asking more questions about that, digging into that, pushing Congress to act on that; not on blueprints. That they want to listen in or gather information on bad guys, fine, but when they expand that same action to include everyone then I have a problem.

  • Fill a server rack with Dell 3260 storage units, maxed out at 240TB per server. There is room in each rack for 10 such servers, so that's 2.4 petabytes per cabinet which is twice what the article says.
  • The blueprints are at best a measure of those portions of the facility where they will allow low level clearance contractors, like vetted electricians.

    Even the MCI headquarters in Ashburn has an off blueprints sub basement to intel use, so we should hardly expect less of a facility directly owned by a TLA.

  • It's aluminum foil. Al != Sn
  • They couldn't do their absolute worst case scenario if they wanted to! ...for about another 5 years until storage drives jump an order of magnitude.
  • it's actually like a TARDIS
  • 1.2PB per 42u rack it seems to assume 3TB drives in backblaze style pods. 10 Pods 45 drives per pod 40 drives worth of storage in raid 5 or similar give 400 drives of storage or 1.2PB with 3TB drives 4TB is 1.6PB. Taller racks (telcom style 72ru) nearly double that density and suck to work on making it the perfect choice for government work. But the number could easily be in the 16-30EB range.

There are two ways to write error-free programs; only the third one works.

Working...