NSA Drowns In Useless Data, Impeding Work, Former Employee Claims 120
An anonymous reader writes in with this story of confusion at the NSA due to the flood of data they harvest. "Some of the documents released by Mr. Snowden detail concerns inside the NSA about drowning in information. An internal briefing document in 2012 about foreign cellphone-location tracking by the agency said the efforts were 'outpacing our ability to ingest, process and store' data. In March 2013, some NSA analysts asked for permission to collect less data through a program called Muscular because the 'relatively small intelligence value it contains does not justify the sheer volume of collection,' another document shows. In response to questions about Mr. Binney's claims, an NSA spokeswoman says the agency is 'not collecting everything, but we do need the tools to collect intelligence on foreign adversaries who wish to do harm to the nation and its allies.'"
Solution... (Score:5, Funny)
Re: (Score:3)
That is just warehousing data they can't process. Snowden and the commentators say that encryption is still good, it still works. At best that allows them to process chains of related data if they get a break.
All standards are tested but some standards are mo (Score:2)
http://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryption-codes-security [theguardian.com]
http://www.washingtonpost.com/blogs/the-switch/wp/2013/11/04/how-we-know-the-nsa-had-access-to-internal-google-and-yahoo-cloud-data/ [washingtonpost.com]
https://en.wikipedia.org/wiki/Turbulence_(NSA) [wikipedia.org]
Re:All standards are tested but some standards are (Score:4, Insightful)
That's all very nice, but be clear -
Bruce Schneier: Crypto works [youtube.com].
Yes cold it is very nice (Score:1)
Re: (Score:2)
LOL The world now understands tame US crypto as used, sold and tested is junk.
You keep repeating that, but it still isn't true. (Did you even bother to watch any of it?) All the available evidence is that the math is still protective. The problems are other places.
I think the NSA would probably be happy to see your scenario. Just think, part time visual basic programmers around the world turning out "secure" products to protect you from the "Yankees." Of course they will guarantee their own work, it's from their elite programmers, their own local genius that can't be questioned.
Everybody wins Cold (Score:1)
Trade deals, banking, political parties, political leaders around the world, NGO's, anti war protesters, law reform groups, environmentalists... commercial and scientific developments...end users are all at risk.
As the video you posted stats bulk collection of data is now cheap and easy. At the 43 min and 46 min point in - "we have made surveillance too cheap"
So long term, where the NSA and GCHQ got in thanks to junk encryption standards, so can ex staff, former staff and any group that can h
Re: (Score:2)
I guess it's my turn to LOL now. Bulk collection is cheap because encryption isn't generally used. When governments legally force the turnover of keys that won't change. Although there may be some spots where security is stronger, it is likely we'll see more actual junk being produced in the future. Perhaps you recall the early days of PCs when many vendors did their own thing instead of relying on DES. How many of those products would hold up to NSA, FSB, or China? And that is before you get into the
Re: (Score:2)
Re: (Score:2)
LOL, sorry, no. DES was only ever intended for unclassified data and was limited in strength. The record is clear that NSA strengthened the DES algorithm against attacks not publicly known at the time. The best anyone ever did against full strength DES was pretty much brute force (linear was very late to the game, and limited). That is what the DES Cracking project was about, finally putting a bullet in DES to get the next standard going. Now we have AES, and nobody can really claim that it is weak, ca
Re: (Score:2)
Re: (Score:2)
If so then nobody but NSA knows about the technique despite decades of trying. The password and brute force are pretty much it as far as anyone else knows. Even differential and linear are hardly useful.
I suppose there is an advantage to spreading rumors that DES and AES have a back door. Then more people will use weak crypto, and NSA gets the bounty.
Re: (Score:3)
"When governments unethically and immorally, but legally force the turnover of keys that won't change."
FTFY
I have a better idea. The police forces and security services should do actual police work, instead of eavesdropping on the entire population. Detective work and investigations are labor intensive, but the US constitution demands that such labor be used instead of just spying on everyone.
Bugs are NSA's best friend (Score:2)
Crypto (likely) still works now. The NSA wants to snapshot everything they can so that as their code cracking capabilities expand they can go back and decrypt old data as desired.
Yup imagine that a bug like debian's openssl bug is discovered.
That mean that the NSA can suddenly go back through all these archives and decrypt what they can.
Note: this is different from brute forcing. And brute forcing is NOT going to happen. Modern cryptography has reached the point where brute forcing is not merely difficult (like back in the time of Enigma) but beyond what could theoretically be possible with current mathematics and current physics while still even having a margin in case of some bugs
Re: (Score:1)
If we could only convince the spammers to encrypt their spam.
NSA drowns (Score:2)
the sorrows of NSA, drowned in an information cocktail, Binny o Binny why did you leave me
the woman spoke
It's not actually a problem. (Score:5, Funny)
Because it's only simulated drowning.
Re: (Score:3)
That's because "data management practitioners" spend their time practicing data management. I bet if you asked the "data analysts" about it, they'd say most of the important work dealing with data is in the analysis, but they still need to waste 20% of their time on data preparation and integration.
Re: It's not actually a problem. (Score:4, Informative)
That's because "data management practitioners" spend their time practicing data management. I bet if you asked the "data analysts" about it, they'd say most of the important work dealing with data is in the analysis, but they still need to waste 20% of their time on data preparation and integration.
Actually the number we quote is analysts spend 60 - 80% of their time manually prepping their data for analysis if they don't have a solution in place. Its a BIG problem. Just because you can ingest everything in the world doesn't mean you should.
Re: (Score:2)
If you have an infinite budget, it makes sense to do that. The NSA comes pretty close.
Re: (Score:2)
It's just metadrowning, the emotions you feel alongside the actual drowning. They don't identify you, your trauma, or the hot date-on-the-side you were with when you fell into the ocean though (we found that through your Facebook page).
Re: (Score:2)
Well played sir, no mod points right now but you deserve more for that!
the answer: collect useless data (Score:2)
Re:the answer: collect useless data (Score:5, Insightful)
Yep. When your job is to find a needle, the best strategy is always to pay top dollar for a few million haystacks and see if there are any needles there.
Re: (Score:1)
Didn't find a needle? We need more haystacks! There has to be a needle in one of them!
Re: (Score:2)
Well, to be fair, other parts of the US government are very very busy manufacturing new needles all the time.
There is no questions that there are needles which can be found.
But if that haystack is still out of reach by now, that needle isn't likely to stab anyone, so is it worth searching for?
Re: (Score:1)
Just give the NSA time, they'll find a bigger magnet and box of matches. The trick is to start making those needles out of something indistinguishable from the rest of the hay.
same old same old (Score:5, Insightful)
And the first analysis is: what sort of data should we collect to make analysis easier? But of course, if people actually analyzed the process itself, someone would have already pointed out that the only way to measure cost-effectiveness is to have an actual goal in mind. Collecting everything you can get your hands is an easy goal to state.
Stating why all that data will help you prevent attacks on America instead of being viewed as an attack on Americans is a whole lot harder to articulate.
Same old same old.
It's a lot easier to invade a country than it is to state what peace would really have to look like.
Re:same old same old (Score:5, Insightful)
Collecting unanalyzed data is a waste of time and effort. Period.
It is much, much worse than that. Collecting unalyzed data that, in more nefarious hands, can be used for extortion and political manipulation, in part because it was collected en-masse, is a criminal violation of spirit of the 4th ammendment to the U.S. constitution, if not the interpretable letter of it.
Not only that, but if in order to collect it, you had weaken the security systems used by the masses for their communications, you are basically making all those systems easier to attack for everyone. This is what has happened, both directly with things like the $10M to RSA, and indirectly, just by having a quid-pro-quo where all the tech companies are blissfully happy to not invest in real security for their users, because the more influential government overlords are totally cool with it. They leak the vulnerabilities they discover that they want fixed, and enjoy a massive trove of vulnerabilities they keep for themselves (and unknown numbers of others clever enough to discover them as well)
Re:same old same old (Score:4, Insightful)
It is much, much worse than that. Collecting unalyzed data that, in more nefarious hands, can be used for extortion and political manipulation,
Ummm...that's the whole point of collecting the data. It has nothing to do with national security. That's just the cover. It's about power and control.
Real Message: (Score:5, Insightful)
We have all this yummy data we gorged on, and we can't digest it all.
Obviously, we need a bigger budget for more contractor analysts and hiring Google to write better analytical tools.
Re: (Score:1)
Obviously, we need a bigger budget for more contractor analysts and hiring Google to write better analytical tools.
Uh, why hire Google when you can just tap their internal traffic and analyze it?
Also, you have to go at the analysis strategically. You start with analyzing the data of the most dangerous people: senators who are critical of increasing the NSA budget. That way, the problem sorts itself out. Preventing terrorist attacks, in contrast, prevents future funding, thus endangering the interests of the U.S. domestically and abroad, and has to be avoided.
Any casualty that can be blamed on terrorism is worth rough
Information overload? (Score:2)
Be friendlier to foreigners .... (Score:2, Interesting)
An easier solution .... treat foreigners as you would have them treat yourself or your compatriots. Apply the same standards of "justice" that you would meet out on your own citizens. That means no torture, no dronings, and respect for international law. In the end a much more successful strategy, and certainly a far cheaper one. Foreigners are not inherently evil, nor are they all plotting your demise. They are people who deserve equality.
Re: (Score:2)
On a certain level this is their job. (Score:1)
The argument is that they have to "see everything" to see as many potential threats as they can. At a surface glance this makes sense.
At anything beyond a surface glance, you can see how mission creep happens and oversight is effectively nullified in the process.
Not all surveillance is necessary, without question the vast majority of it serves no functional purpose beyond its own self-certification.
The lying certainly isn't helping anyone trust them.
Wasn't that the problem (Score:5, Interesting)
The NSA knew about some of the 9-11 hijackers, but it was lost in the noise (and in lack of interdepartmental information sharing). The solution, suck in more noise? Makes little sense to me.
Re:Wasn't that the problem (Score:5, Funny)
Makes little sense to me.
You're obviously too intelligent to get very far in intelligence work.
Re: (Score:3)
Here: http://www.pbs.org/wgbh/pages/frontline/homefront/view/ [pbs.org]
The NSA knew about some of the 9-11 hijackers, but it was lost in the noise (and in lack of interdepartmental information sharing). The solution, suck in more noise? Makes little sense to me.
I don't think that is quite right.
NSA speaks out on Snowden, spying [cbsnews.com]
Gen. Keith Alexander: Well, the reality is if you go and do a specific one for each, you have to tell the phone companies to keep those call detail records for a certain period of time. So, if you don’t have the data someplace you can’t search it. The other part that's important, phone companies-- different phone companies have different sets of records. And these phone calls may go between different phone companies. If you only go to one company, you'll see what that phone company has. But you may not see what the other phone company has or the other. So by putting those together, we can see all of that essentially at one time.
John Miller: Before 9/11, did we have this capability?
Gen. Keith Alexander: We did not.
John Miller: Is it a factor? Was it a factor?
Gen. Keith Alexander: I believe it was.
What Gen. Alexander is talking about is that two of the 9/11 hijackers, Khalid al-Mihdhar and Nawaf al-Hazmi were in touch with an al Qaeda safe house in Yemen. The NSA did not know their calls were coming from California, as they would today.
Gen. Keith Alexander: I think this was the factor that allowed Mihdhar to safely conduct his plot from California. We have all the other indicators but no way of understanding that he was in California while others were in Florida and other places.
Re: (Score:3)
Re: (Score:3)
Possibly, but note this section from the article [go.com]:
... sources said, even if the messages had been translated sooner, it would not have been of much use because the messages were too vague and had no context, with no details of time, location or the nature of the event referred to.
The sources did not consider the information to be a smoking gun, and described it as the sort of chatter that is intercepted constantly, and is seldom of use.
Re: Wasn't that the problem (Score:1)
Based on what should the sayings of General Alexander be trusted? Given that he has repeatedly lied about other things, how do people reason - why would he NOT lie here as well?
I want to understand the thinking. Following this debacle from outside of the US has been interesting to say the least, though occasionally, like now, puzzling.
Re: (Score:2)
I think the first thing to consider is that many claims are made, but not all hold up under examination. They would prefer to not have to say anything, it is the nature of their job. To understand some of the theater going on you may want to read this [commentarymagazine.com].
Big Data (Score:1)
The belief that as the size of a pile of shit increases, the probability of finding a pony approaches 1.
Like FBI before 9/11 (Score:2)
After the fact it was discovered that they had lots of clues. The problem is how to link them together when you've got so much in your files.
Re: (Score:1)
They were trying REAL hard not to see those clues at the top levels of our government. Bush was personally warned on his Crawford ranch by an NSA agent.
There *might* have been some motivations to miss the 9/11 attack "clues", just like there were motivations to deny the USS Liberty bombing/strafing incident.
Re: (Score:1)
And for lying about "weapons of mass destruction". And for building 7 to collapse.
Re: (Score:1)
Because economics is too complex for you, you think colonialism is still a good idea, eh? Oh right, it's because he's BLACK that you're uneducated...
Fucking good. (Score:2, Funny)
Good. Let's create some more useless data for them, I'm starting a second Tor node and a Freenet node tonight.
Re: (Score:3)
My attention span is too short to read that comment.
Your ingenious technique for not drowning in useless data is much more cost-effective than anything the NSA will come up with.
I don't get it (Score:2)
Foreign adversaries.
Like the Germans, French, Spanish, British, Israel and other Americans?
Re: (Score:2)
You misunderstand them. 100% of the world looks like the enemy. They don't even exclude themselves. The odd thing is that they're wrong. Some people actually support them.
You Should Have Those Tools (Score:5, Insightful)
"we do need the tools to collect intelligence on foreign adversaries who wish to do harm to the nation and its allies."
Ahh, good, something we can agree on. You should have those tools. And you do have them, even without the dragnets. Here's how they work:
1. Pick the person who you believe wishes to do harm to the nation and its allies.
2. Start collecting surveillance.
3. Present to an appropriately skeptical judge the reasons that you believe that person wishes to do harm to the nation and its allies.
4. The judge will decide whether your evidence amounts to reasonable suspicion.
5. As long as the judge agrees, you can continue the surveillance.
It's a pretty cool system, really. It ensures that you get the surveillance on people who really do appear to be up to something, while protecting the vast majority of people who are innocent.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
misinformation campaign (Score:5, Interesting)
Yeah, this 'employee' is claiming that they actually asked to collect less but were forced against their will to collect more than they can handle? Flat out bullshit.
They know the cats out of the bag so now they're just going to run with "We've got more information than we can use, so you really have nothing to worry about us hoarding all your data and in fact the more we collect the safer you are!"
Where have we seen this before? Oh that's right, "Pay no attention to the man behind the curtain!"
(captcha: seducing)
The sock puppets have new talking points (Score:5, Insightful)
This has never been a problem due to fast sorting, keywords, voice prints, numbers called and cheap storage.
GCHQ and the NSA could get every call from Intelsat back the late 1960's for sorting and indexing. Once you have the total 'in' and 'out' points of any nation as its telco networks is constructed: https://www.eff.org/deeplinks/2013/08/dea-and-nsa-team-intelligence-laundering [eff.org] shows how easy a lifetime of collection can be and looks like under one small program
Re: (Score:2)
Isn't it curious that the NSA seems to have more leaks now after Snowden than before?
You would think there would be more scrutiny.
I can imagine two scenarios;
1) There is some welcome internal discussion bleeding out to question what the NSA is doing to itself and if it's actually useful to collect all the data.
2) Misinformation is trying to make it look like the NSA is a goofy information hoarder drowning in it's collection of bits and bytes and was never able to track or control anything. That's right folk
Re: (Score:2)
To stay in their countries and be free, they had to play the review/court/cleared game. Snowden understood the total chilling option of any US court even with US political protection and good cleared lawyers.
The real long term struggle seems to have been between the NSA, GCHQ and political leaders over allow
Re: (Score:2)
Tracked and logged, probably. At least everything significant (and a lot that isn't, of course). Decoded? Well, no. Many cyphers were weakened, but some are secure. Anyone who really cares can use a secure code. But possibly not a secure public key encryption. That depends on the person that you are trying to reach have the same secure mechanism that you do, and THAT requires pre-arrangement. And the govt. has acted to weaken the standard public key systems.
FWIW, one-time-pad systems have never been
Re: (Score:2)
Whoa, weird (Score:2)
I've said it since the Snowden leaks first came out, there isn't a way to process all of the data that is generated on the internet. And I feel that this whole bullshit concept about the NSA collecting all of the information on the internet is another way to dowse for illegal activity (dowsing as explained here [ted.com]) Meaning that as long as people believe 'it has the power to do such' (because it was fucking expensive to build that Utah dat
Think East German (Score:2)
Any of the above could be politically sensitive to current or former political leaders, their backers and top staff.
If only you can be found before your story is published, open court work or protest starts
The spying was never for terrorist and here is why (Score:2, Troll)
Terrorist can use any words they want, common phrases but given a different and agreed upon meaning within their dialog constraints.
On the other hand and within the timeline there was need to have an ear to the public in order to know how to respond in the cover up of 9/11 (Building 7 was not hit by a plane, It obviously was taken down by demolition and what it contained needed to be removed to help the cover up.) This is verfied!
What the government knew for certain is that they could create a feedback loop
Re: (Score:2)
I didn't agree to have my taxes spent this way!
Re: (Score:2)
Previously an article on slashdot of them wanting more data collection ...... in total contradiction to this article. http://slashdot.org/comments.pl?sid=4590265&cid=45767805 [slashdot.org]
Re: (Score:2)
Don't assume that everyone who works for a company wants what the CEO wants. Some of them think he's stupid for wanting some things, and consider other things much more important.
So those articles aren't in contradiction, you're just hearing from different voices.
Two words: Binney. Thin Thread (Score:4, Informative)
Thin Thread
http://www.businessinsider.com/nsa-whistleblower-william-binney-was-right-2013-6 [businessinsider.com]
http://en.wikipedia.org/wiki/ThinThread [wikipedia.org]
http://www.whistleblower.org/program-areas/homeland-security-a-human-rights/surveillance/nsa-whistleblowers-bill-binney-a-j-kirk-wiebe [whistleblower.org]
Binney.
http://www.newyorker.com/online/blogs/backissues/2013/06/takes-the-nsas-surveillance-programs.html [newyorker.com]
http://www.democracynow.org/2012/4/20/exclusive_national_security_agency_whistleblower_william [democracynow.org]
http://publicintelligence.net/binney-nsa-declaration/ [publicintelligence.net]
Reinstate him as DNI.
The point is that they can target YOU (Score:5, Interesting)
This mass collection is not about what they can process or correlate with terrorism or whatever. This massive amount is dangerous because they can target individuals. You simply can not assume that all this power will be used for the good of the nation, the inner workings of this huge system are manned by humans. They are prone to corruption, bribery, self interest and so on.
This much power with this little accountability is just bound to be used for personal gain. Imagine if some worker of this system decides he really does not like his neighbor guts. He could target that individual and discover that for example he is having an affair and the disclose that information to cause harm to that individual in particular. Well change that neighbor to some politician that is contrary to the current governing party.
The funny thing is that Metal Gear Solid 2 foretold all this more than a decade ago.
Re: (Score:2)
I wish I had points to mod up your MGS 2 reference.
I'm still waiting for remote controlled soldier's like in MGS 4.
Excellent... (Score:1)
...how can I help?
No, seriously - I tried to start discussion in a previous "The NSA is sniffing your dirty boxers" thread about the possibility of an easy-to-use browser / email plugin / app / etc. that would encourage Joe User to increase the amount of "noise traffic" he generated. E.g., something that would tack a bunch of Terror Words onto the end of every email, but more practical and less scary to use. Encourage people to automatically participate in conscientious objection to surveillance the way t
Re: (Score:3)
That would help some physical sites. Get people thinking about crypto - the historical ways in during pre ww2, ww2, the cold war, 1990's and via the good news from Snowden.
Re conscientious objection - support mainstream and alternative media, legal rights groups and educators all over the political spectrum.
Learn
Re: (Score:1)
I recall reading that spam makes up some 70% of internet traffic. Get your keywords into spam, and your noise propagation will massively skyrocket. Can you take over a botnet and repurpose it? That should be your goal, if so. If not, you might get involved with encryption of some kind. There's plenty of room for extra noise in encryption streams; throw in a few keywords into headers or tack it onto hash algorithms and you might have something as well.
I don't think you're going to get much traction with get
Drown 'em with Tor traffic (Score:1)
The NSA hates Tor [torproject.org]. So running a Tor Relay is a great and safe way for us to actually do something about the NSA.
Hmph - Nice PSYOP. (Score:1)
They are playing the injured naughty puppy. Please, what better way to alleviate your privacy invasion fears than to make you think they can't even handle all of the data. Surely, it's digitized, compressed and permanently stored for future data mining purposes should you ever become a person of interest. I mean really. The future FBI won't even have to profile people the traditional way, many of us are already doing it for them (hello FB).
All a crock of sh... (Score:2)
I've given this capability a long and hard thought. This interception only works during an economic war and does nothing during a real war. Once a real war kicks off on any global scale, these types of interception capabilities get turned off because countries will sever certain cables and links.
Companies that are hosted in the cloud will get disconnected destroying them in hours.
The purpose of NSA data collection (Score:3)
Is not for terrorism, or even drug fighting. Its a tool for the Democrats or Republicans, whoever is in power, to snoop on their political opponents and line their pockets by stealing civilian secrets. Look at the IRS scandal, look at Fast & Furious / Gunwalker. Nothing is beyond this out of control, corrupt as heck govt. Probably more corrupt than Russia or wherever in the world, they just were able to hide most of it (until Snowden).
Never, but Never (Score:4, Interesting)
,
But never, ever dare ask why so many wish to do harm to the Imperial Us and our henchman, upon pain of treasonous death.
A Useless Post (Score:2)
...to make the NSA's job even harder.
This is why (Score:1)
Re: (Score:2)
huh.. stasi did a lot of "meaningful" things.
just not any good things.
but there is a law, if the budget of the one who is controlling secrets is a secret, then his budget will be unlimited - and that has consequently ends up being more expensive than it is worth, but it takes the state to crumble to expose that, since where the money is going is a secret.
Re: (Score:2)
Have an interesting book buying list, travel: sooner or later a database will sort a lot of people's files for human security review.
The Stasi moment - that flood of new files, limit cleared staff and the political demands to find something to show the tame press.
The what can the gov do? A sneak and peak? More logging of web 2.0
Re: (Score:2)
Real world example I know of personally: Have a (nominally Christian) boyfriend from a country where the prevailing religion is Islam. Bang - straight onto the list. So much so that the pair in question even picked up a tail of spooks at least once when on holiday.