Underfunded NSA Suffers Brownouts

An anonymous reader writes "Almost ten years after the an internal report, and a year after a Baltimore sun story warned that the electrical system at the fort Meade NSA HQ couldn't keep up with the growing electricity demand ... the problem has got worse. The 'NSA has had to resort to partial, rolling brownouts at its computer farms and scheduled power outages and some offices are experiencing significant power disruptions'. NSA director Alexander testified to congress about this problem. It is suggested he wanted to add more than $800 million to the 07 budget. A recent public powerpoint presentation suggested 70% of of all intelligence spending goes to contractors. It also included a graph, without numbers, of this spending. It suggests that US intelligence spending is around $60 billion. An internal survey that showed NSA employees have problems trusting each other."

Tag plz

[Tuoqui (1091447)]

can we get a 'haha' tag added?

They spent it already?

[Colin Smith (2679)]

Ah well, better get the printing presses running again.

 

Re:They spent it already?

[_Sharp'r_ (649297)]

It sounds like the problem is a misallocation of funding, not underfunding.

If you can power X equipment, then why bother to purchase X+Y equipment before you purchase more power capacity first?

Having more equipment than you can power is a symptom of spending too much on equipment and not enough on power capacity. It says nothing about whether your total budget is too low or too high.

Wait, I forgot, we're talking about a government agency. They just assume that any money they mismanage can just be used as justification for an additional funding demand the following year.

Must be nice to be able to get more money because you totally screwed up spending the last round of funding. Too bad it's us giving them the money.

Re:

[Martin Blank (154261)]

Their planning begins years ahead of time, often working on systems three generations beyond the ones that they're currently installing. Problems with designs can push power usage for a given system much higher than planned, and it can take time to get the power systems in place. NSA is a naturally paranoid agency; they take all of their information sources and know that someone else is looking at exactly those to analyze them, so they don't want anyone to know exactly how much power they're using because

Re:Doesn't matter.

[creimer (824291)]

They're supposed to have some VERY smart people on staff there.

Except they're working for Google now. It's a big problem that a lot of IT outfits are running into. ;)

Toilet seats

[gravos (912628)]

I think they may have bought one too many $40,000 toilet seats. But this is a serious issue: These brownouts are affecting their ability to spy on us! Something must be done immediately or innocent men may go free.

Re:

[sgt_doom (655561)]

Regarding NSA and "underfunding" - a really crucial point us conspiracy nuts (i.e., informed and educated Americans who've taken - and passed - probability math) like to bring up:

Immediately after the 9/11/01 attacks, the then-NSA directer, General Hayden (now CIA director), went before the US Congress requesting emergency funds. What were the top two expenditures of said funds? (This is public domain information and easily verifiable.)

(1) More security guards, and

(2) Hiring more polygraph examiner

Re:Toilet seats

[jandrese (485)]

Polygraph Examiners spend the great majority of their time giving polygraphs to people applying for security clearances. Since 9/11 the backlog on security clearances has skyrocketed as people got paranoid and started slapping "SECRET" and "TOP SECRET" labels on previously unclass projects. Also, many contractors saw the writing on the wall and pushed harder for all of their employees to get cleared so they wouldn't be out of the loop on new project opportunities.

The security guards should have been obvious since all federal buildings stepped up their security after 9/11. There were tons of entrances that suddenly got a real live guard 24/7 where they used to have just a apeaker you would buzz in with after hours. There were also lots of parking lot entrances that got new guard shacks. The "cleaning house" theory doesn't seem the most likely explanation to me.

Re:Toilet seats

[AHumbleOpinion (546848)]

I think they may have bought one too many $40,000 toilet seats.

Regarding government overspending:

1. If it was a zero G toilet seen and the production run was for a handful of space shuttles and a space station then $40,000 is probably a pretty good price. I suspect this is the source of the $40,000 urban legend.

2. For "commodity" items you can not compare necessarily a military part with a commercial part even when they come off of the same production line, ie. we are not comparing a mil spec part, a radiation hardened CPU for example. Military parts often go through additional testing and this can greatly increase the costs due to a loss of economies of scale. In the field, when a military part is pulled from the box there is an expectation that it will work. In the consumer world it is often cheaper for a manufacture to replace defective parts than to test them. Expecting the customer to return to the store for an exchange is considered acceptable. Alternatively the acceptance standards may be higher. For example no dead pixels being allowed on a flat panel. This requires additional costs with respecting to screening a large batch and cherry picking individual items.

3. I guess there is also the ever popular urban myth that they pad the price of some items in the public budget to hide spending on secret projects. ;-)

Re:Toilet seats

[Deadstick (535032)]

First, the sum trumpeted by Sen. Grassley in 1983 for the "toilet seat" was $640, not $40,000. Second, it was not a seat but a shroud for the toilet assembly, made corrosion-resistant because it was designed for Navy airplanes that are used near salt water -- in other words, it was a complete airplane bathroom enclosure. Not a bad price.

Oh, and the actual seat was included.

rj

Re:Toilet seats

[Jah-Wren Ryel (80510)]

First, the sum trumpeted by Sen. Grassley in 1983 for the "toilet seat" was $640, not $40,000.

And of course, $640 ought to be enough for any toilet seat!

I agree but...

[LWATCDR (28044)]

Military purchasing is a nightmare. I work for a software company and one of the services wanted to buy our software. Okay fine how many do you you want. $200 copies fine we will even give you a discount on them...
Yea right.
First they spent a year testing it. This was none mission critical admin type stuff not a flight control system or anything. After the test they decided they like our software the best and wanted it but they had to put it out to bid. Well they tried to make it a single supplier for this

I wonder

[GreatBunzinni (642500)]

I wonder how much of these problems are really due to lack of funding and how much are just tactics to yank an even bigger chunk of money from the guys in Washington. After all, the problems that they describe should only exist if the person in charge purposedly screwed up the budget.

Re:

[afidel (530433)]

I assume this is for a total Datacenter upgrade. You rarely just upgrade the electric, you usually upgrade all the facility elements together since all the systems are usually designed with some kind of balance, eg you don't put in 200tons of AC for a 100ton electric load. Since commercial datacenter projects are running into the hundreds of millions I have no doubt that with typical government inefficiency a large datacenter project could run to a billion.

All I can say is this..

[Adult film producer (866485)]

Good.

I hope they have more of these problems. They've proven themselves to be a complete waste of money, remember that whole terrorist thing on September 11th? $50 billion/year wasted on these bloated government agencies, abolish them now. And despite having all the resources in the world at their disposal they still managed to screw up the intelligence on Iraq. I am not impressed.

I'm sure the CIA/NSA/DIA/DOA/etc all have very clever tic-tac-toe competitions against supercomputers and think up

Re:All I can say is this..

[rm999 (775449)]

The opposing argument is that September 11th happened because agencies like the NSA were underfunded at the time.

Oh yeah, I forgot to say.

[Colin Smith (2679)]

This is why "the singularity" ain't going to happen.

 

project Turbulence

[wizardforce (1005805)]

New equipment for data processing, as well as some purchased for one of the agency's signature initiatives, the mammoth modernization effort dubbed Turbulence , are among those that have been held up, the senior official said. The lengths of the delays are classified.
They may have a lot of power problems but at least they have good sense naming their projects.

A new kind of DoS attack

[TodMinuit (1026042)]

Okay, everyone around the NSA, turn on all your lights, computers, TVs, air conditioners, and appliances. Operation Dark Storm is a go.

How to meet the budget/electicity supply

[J.R. Random (801334)]

Actually obey the Constitution. If the NSA wasn't doing illegal warrantless searches of every American using the telephone or internet it would need about half as many computers and half as much money.

Doesn't sound like underfunding...

[RyanFenton (230700)]

This doesn't sound like underfunding at all. It sounds like highly misappropriated funds going to prioritized sub-groups with an inherent motivation to see the other subgroups suffering and failing for the sake of their own relative gain. This is completely in keeping with the current administration's modus operandi of finding subgroups in organizations (lobbyists, regulators, etc.), that will play ball, and finding a way to eliminate or functionally undercut everyone else, then blame those who were undercut for the resulting general failure.

Ryan Fenton

Re:Doesn't sound like underfunding...

[RyanFenton (230700)]

When an organization doesn't have enough money to do something, that is known as underfunding. When an organization _does_ have the money, but spends it inappropriately so they can't deal with the issues they are responsible for (including their own internal upkeep), then that is known as misappropriation.

The NSA had the money they needed to deal with their infrastructure problems, but did not. Wasn't this the kind of cooperation and organizational problems the whole "post 9-11" reorganization efforts were intended to fix? I will not argue that it is a failure of previous administrations that this did not get fixed earlier - just that these exact kinds of deep organizational failures coming to a dramatic conclusion are exactly in keeping with this administration's practices so far.

For a small sample of supporting evidence for my arguments, assertions and conclusions, see:

The Republican War on Science (Book) [waronscience.com]
Most of the recent works by John W. Dean (Several books) [amazon.com]
One of many powerfully incisive books by George Lackoff [amazon.com]
Countless other books, including these [amazon.com] ...and most political news appearing outside FOX news for the past 4 years.

Ryan Fenton

Re:

[MillionthMonkey (240664)]

This may be "a failure of leadership spanning ~10 years" because the problems were predicted nine years ago, but they didn't fly out of hand until after 2001; probably after Cheney ordered the NSA to start brute forcing its way through more keyspace than the rainfall in the Tennessee Valley can handle.

Scary!

[misleb (129952)]

And the number one way you know you're being watched is...

When the NSA can't even find enough electricity to power their surveillance and data processing equipment.

Scary stuff.

-matthew

NSA (and GCHQ) are shame to mathematics

[by Anonymous Coward]

As a graduate mathematician, I feel ashamed to see so many of my fellow students either going off to the City or to GCHQ (the UK's NSA); while it's true that the cryptanalysis work done by the latter is one of the few non-academic jobs requiring considerable "pure" mathematical skill, that's really not what the huge amount of money spent on infrastructure is for.

Because the War on Terror/Evil Of The Day really isn't about challenging mathematical genius terrorists to ever more complicated ciphers - yes, GCHQ created RSA a few years before R, S and A, but occasionally beating open academia and having a lot more horsepower isn't ever going to put you beyond the mathematical principles you're faced with (*). Massive horsepower is for statistical analysis of insanely large quantities of data. This might occasionally find you something saucy, but it's mostly going to allow you to profile, and profiling reduces risk - past trends are a useful indicator of future performance, whether you're analysing a financial market or the behaviour of groups of humans.

None of this will help if some random guy decides, tomorrow, to commit some nefarious deed involving an IED - something I'd say 90% of graduate scientists either have the knowledge to do, or could read up on overnight. Which goes to show that the reason everyone's not blowing everyone else up is not because there are any technological measures in place to stop them, but because by and large, for whatever reason, people don't want to.

(Oh, and the NSA/GCHQ do have some obvious legitimate uses - such as decrypting messages between known ne'er-do-wells. If that's all they did, I might even like them.)

Oh, and before people forget, the problem of whether the NSA is allowed to spy on Americans is easily solved in principle by GCHQ and NSA doing the dirty work for their friends across the pond; in practice, an extra-judicial agency couldn't care less anyway: he who is not accused (for there is no-one allowed to witness the crime), is not judged.

(*) This is why I love my discipline. Men can only discover mathematics, never beat it!

Bake Sales!

[rueger (210566)]

To paraphrase an old favorite,wouldn't it be nice if we lived in a world where daycares and hospitals had all the money they needed and the NSA had to hold a bake sale to buy servers?

Re:

[Cosmic AC (1094985)]

Wouldn't it be nice if we lived in a world where we didn't have to spend money on defense? We could use the money for other things, like gumdrops, and lollipops, and rainbows, and magical unicorns.

Obvious solution here

[cheros (223479)]

Install a generator that can use contractors as fuel. Solves two problems at once :-)

It has as beneficial side effect that it also reduces the amount of people leaning on healthcare, so everyone wins.

The only challenge I can see is that you have to take into account the amount of alcohol these people consume. Any oven should be able to use the spontanous combustions that may occur. Maybe turning them into biofuel may be better.

Sorry, heavy lunch :-)

Rolling brownouts? Uh, no.

[by Anonymous Coward]

There is no such thing as an intentional rolling brownout.

A brownout is usually caused by a short or a transformer melting down which results in an under-voltage leading to a blackout. A brownout is when you still have electricity but it's not at the required voltage or power level.

I think they mean rolling blackouts.

Re:

[aaarrrgggh (9205)]

Actually, a brownout is generally a loss of frequency due to system overloading. In the good old days of resistive loads, letting the voltage sag 10% would drop the power consumption by 20%

With inverse-impedance equipment being the norm today (switch mode power supplies, electronic ballasts, and VFDs), a brown-out is much more likely to create localized outages as individual feeders become overloaded.

But, what they are most likely really referring to is aggressively scheduled maintenance to allow for upgra

It is an axiom...

[vorlich (972710)]

that the purpose of a bureaucracy is to provide employment for the bureaucrats.
Arthur C. Clarke has suggested that the greatest threat to civilisation is bureacracy.
The 19th century French writer Balzac once said that 'bureaucracy is a giant machine operated by pygmies'.
Sadly bureaucracy is often reminiscent of Homer's Duff Beer - the answer to and the cause of all our problems.
I guess I didn't have to think too much for this post, just pasted in a lot of fondly remembered homily!
Outstanding!
Hmm, forget to mention girls or drugs - they are always popular. Did manage to get beer in though.
Fourth wall? What fourth wall? People read this? No, honestly...

Duh!

[thewiz (24994)]

An internal survey that showed NSA employees have problems trusting each other.

They're spies! They're trained not to trust anyone!

Captain Obvious strikes again!

NSA is not the problem

[HangingChad (677530)]

This is the real problem:

A recent public powerpoint presentation suggested 70% of of all intelligence spending goes to contractors.

The NSA is subject to Congressional oversight, contractors are not. 70% of our intelligence spending is unaccounted and unregulated. It's not the NSA you need to worry about spying on you, it's AT&T. When questions started surfacing about their role in spying on Americans, they responded by asking Congress for a liability shield. AT&T doesn't depend on Congress for their budget, the NSA does.

Reducing power demand is easy

[Skapare (16644)]

Reducing power demand is easy. I've already done this in one server facility. Just change the voltage being fed to the computers with dual voltage or wide voltage range switching power supplies from 120 volts to 208 or 240 volts. The power supplies will on average use about 3% less power. Additionally, because the total current being used is less, the heating losses in the wiring leading to the computers will be significantly reduced (although it is usually only 1% to 2% of the total power demand). In the case of 3 phase power systems, a substantial current will be present on neutrals, causing a lot more loss (and in some cases a potential fire hazard). By connecting computers between phases in 208Y/120 volt power systems (line to line instead of line to neutral), the accumulation of currents on the neutrals will be eliminated. The currents on the phase conductors will be greater, but not by as much. The power lost heating up the conduits will be less. Alternatively, they could switch certain power systems to 416Y/240 volts and reduce the current even more (although this would require going back to line to neutral connections).

I just wonder if the NSA already knows this. Maybe the analysts do, but what about the facility managers?

Re:

[afidel (530433)]

Better yet go with DC power supplies. If you design a datacenter from the getgo to be DC you can save a significant fraction of total power usage. You save on the DC->AC inverter in your UPS and on the AC->DC step in the power supply in the servers. Add to that the lower AC demands from the more efficient power usage and your total savings are around 20% of total power budget, which is significant.

Underfunded?

[ScrewMaster (602015)]

Personally, I'd say that they aren't so much underfunded as they are badly managed. The problem is that it can be very difficult to distinguish between the two, since both cases result in serious functional issues.

Re:

[MSTCrow5429 (642744)]

It's a bureaucracy. Any large bureaucracy, where one cannot make direct causal relationships between events and is subject to faceless administrators, breeds paranoia and mistrust.

Re:No shit....

[Original Replica (908688)]

I don't think it's quite paranoia, more of a problem of dividing evryone into "goodguys" and "badguys". Once you start treating the very citizen you are suppoed to protect like "badguys" you stop trusting anyone.

."What we need is fundamental change in the way we manage NSA and what we expect of management and ourselves," concluded the study, which was led by George "Dennis" Bartko, the NSA's deputy chief of cryptanalysis.

Yeah we need a serious change, like admitting that all this cloak and dagger, sorry that is classified, need to know, bullshit is the cause of most of the terrorist problems we have today. Drop the secrecy, and disassemble these above-the-law organizations. Dealing with policy in the open is the only way to keep it honest. When the government is dishonest with the nation about policy you do not have democracy, you have "democracy theater"

Re:

[Original Replica (908688)]

Sure we could publish the launch codes, because we don't really need more than about 10 nukes anyway. Get rid of all the rest. Nukes are zero deterrent in modern urban warfare. The sole continued purpose is to prevent the use of nukes against us by another nation. What nation in the world would not be decimated by losing their 10 most populous cities? Every nuke beyond that is just a terrorist target or accident waiting to happen. It's easier to keep 10 locations physically secure than 50 or 100 locations,

Re:

[c6gunner (950153)]

I'm sorry, did you just seriously suggest that the military shouldn't have secure comms?

Your level of ignorance is astounding.

Re:No shit....

[Original Replica (908688)]

No. I said the Military has no need of secret operations outside of a declared warzone.

But why shouldn't I have the military equivilent of a police radio monitor? I grew up in Virginia Beach next to the largest Navy base on the East Coast, why shouldn't I know when flight manuvers will be practiced over my neighborhood? or that will an amphibious assault training exercise at 4am on the beach at the base? or that there was and accident on one of the ships? or anything else going on in a peacetime military? If they are the might behind the Democratic will of the people, why can't the people know what they are doing? Unless of course they are doing something that the average citizen would find to be abhorrent, like:

"Sophisticated military technology was illegally transferred from a major U.S. company in Lancaster, Pennsylvania to South Africa and Chile and, from there, on to Iraq. The Iraqi-born designer of a chemical weapons plant in Libya set up shop in Florida, producing and then shipping to Iraq chemical weapon components. The CIA, the FBI and other federal agencies were made aware of the operation and did nothing to prevent it. "During the 1980s and into the '90s, senior officials of both the Reagan and Bush administrations encouraged the privatization of foreign policy, certainly towards Iran and Iraq. http://www.jonathanpollard.org/iraq.htm [jonathanpollard.org]

Re:No shit....

[Short Circuit (52384)]

To illustrate, let me describe something about military secrecy. (I read it years ago, so I paraphrase, but it'll give you a rough idea.) If you have one "level 1" document, the classification of your collection is merely "level 1". If you have one hundred "level 1" documents, your collection is actually "level 2".

Why? Because inferences can be made from collections of documents. With enough data, one can read between the lines.

So it's in the military's best interest to keep as much "innocuous" information as concealed as possible. You may only know when a SEAL training operation is happening on your beach, but with knowledge of enough occurrences of such operations, analysts in Russia, North Korea or Iran can realize when they need to be more watchful of their coasts. You may only overhear mention of different lot numbers of 20mm cannon rounds, but analysts could use that knowledge to know if the US is ramping up production of aircraft ammunition.

That's why.

Re:No shit....

[mrjohnson (538567)]

Lol. Leave the defending the country stuff to the people who know how to do it right.

Because you don't need to know. You don't need to know callsigns or military strategy. You don't need to know a unit is out there practicing with blanks and might be vulnerable to attack. You don't need to know how to call for air support, which is a skill that could be used against us if an enemy got one of our secure radios. *We* don't need to think about civilians listening in when we're trying to train. And a thousand other things.

Oh sure, it might be *way cool dude* for you to know about ops, but it's our fucking lives. If you haven't noticed, we have enough to worry about.

Re:Way to edit, guys!

[Wonko the Sane (25252)]

Did anyone else notice that the summary consists of several only slightly-related facts about the NSA, and only the first one really matches the headline?

Oh boo-hoo...

[by Anonymous Coward]

So the NSA doesn't have enough electricity to illegally spy on my phone conversations and e-mail correspondence?

Cry me a fucking river.

Re:

[Simon80 (874052)]

My thoughts exactly, from the moment I saw the headline.

Re:

[bbagnall (608125)]

The problem isn't that it's underfunded. The problem is that it is a government agency. All government agencies grow all out of proportion to their usefulness and they are incredibly inefficient. Solution: Get rid of it or drastically downsize like 80%.

Re:

[SnapShot (171582)]

Read the editorial in today's NYT. They've already downsized. Most of the people working there aren't government employees anymore. A majority are government contractors. This, my friend, is the triumph of Republican-lead privatization [nytimes.com].

NSA still using 12AX7 valves?

[flyingfsck (986395)]

Thermionic valves used for digital processing are run at low power. It makes them last longer and they don't need a winde analogue power range with digital data. That is essential otherwise the big digital thermionic valve computers would never have worked.

Maybe the NSA should try upgrading to transistorized computers now. A BC107 uses much less power than a 12AX7...

Re:

[dbIII (701233)]

Gold plated solutions are OK in the short term or in small numbers, but if you have time shipping in some diesel genators the size of shipping containers makes more sense and is actually on site. The expense of running those subs is worth it for range but using them as generators when other options are available makes little sense. A disaster is a different story when you can get it onto the spot.