Snowden Used Software Scraper, Say NSA Officials 227
An anonymous reader writes with this excerpt from the New York Times: "Intelligence officials investigating how Edward J. Snowden gained access to a huge trove of the country's most highly classified documents say they have determined that he used inexpensive and widely available software to 'scrape' the National Security Agency's networks, and kept at it even after he was briefly challenged by agency officials. Using 'web crawler' software designed to search, index and back up a website, Mr. Snowden 'scraped data out of our systems' while he went about his day job, according to a senior intelligence official. 'We do not believe this was an individual sitting at a machine and downloading this much material in sequence,' the official said. The process, he added, was 'quite automated.'"
Stunning. (Score:5, Insightful)
Who'd have thought? Experienced IT guy didn't manually download each file!?
"Inexpensive and widely available" - I hope they don't mean some evil subversive communist open-source tool.
Wow... (Score:5, Insightful)
Knock me over with a feather, spooks. You fucking hired people to build what is probably the largest collection of signals intelligence scraping systems on the planet, targeted at a wide variety of differently structured systems. Why would you even consider, except as a last resort, the notion that you are dealing with a bunch of noobs?
(Oh, incidentally, maybe you should spend a bit less time reading everybody's email and work on that 'hilarious leaked diplomatic calls' problem, I'm told that sort of thing used to be your job at some point in the past...)
...and that makes it better? (Score:4, Insightful)
If the network can't identify that something accessing the network sporadically and in repeated succession is a bot and should be stopped maybe the NSA shouldn't have access to this much data to start with....
What if a legitimate foreign hacker was able to get in and do the exact same thing? Obviously, they have very shitty standards when it comes to network security - you'd expect thousands of honey pots, ability to intercept attempted attacks, flat out network filtering of these kinds of requests. But alas, that would make sense!
When you are a criminal (Score:2, Insightful)
You use proper tools.
Re:How Many More NSA Employees? (Score:5, Insightful)
Now the question is, how many other NSA contractors / staff / moles / spies have been doing the same thing, without Snowden's intention to disclose their behavior?
I'm sure the NSA assumes they have moles, and none of the data Snowden released is a surprise to the Russians or Chinese.
The NSA was just not prepared for the truth to leak to their real enemy - the general public.
Useless NYT article .. (Score:5, Insightful)
This is retrospective ass-covering cyberbullshit. It is precisely at the edge that the security attacks would come from. What they were doing putting such material on Web servers and Wikis beggers credulity. Didn't senior management not realize that as keepers of the nations secrets they would be subject to attacks both internally and externally. Given the state of non-security at the NSA I would suspect that Snowden wasn't the only hostile with access to the “the keys to the kingdom”.
Re:When you are a criminal (Score:5, Insightful)
When you are anyone trying to do anything efficiently (such as the legally questionable automated gathering and storage of records of millions of phone calls and text messages?), you use proper tools.
A friendly reminder: (Score:5, Insightful)
There's absolutely zero reason to believe anything the NSA says about how Snowden got the documents, or indeed, about anything. They believe they are entitled to lie to congress, so the public isn't even a question.
Re:Stunning. (Score:5, Insightful)
Why?
Because, so far, every single thing that Snowden has said has turned out to be true when cross-checked. And, so far, every NSA official spokesperson has been caught repeatedly lying.
Re:Stunning. (Score:5, Insightful)
Well if you knew a SIEM system had rules which might trigger alters if a database backup is started off hours or if the backup files are accessed for one. As apposed to normalish get query logs with 2XX results, its likely been trained to ignore.
Questioning if who you might eventually leak the data to will have the technical chops and resources put the information together from the database file, as opposed to just reading through a bunch of handy precomputed html pages and office documents for another.
Re:Wow... (Score:4, Insightful)
Re: ...and that makes it better? (Score:5, Insightful)
"I'm all for whistleblower status, but Snowden should have been DEAD, HORRIBLY"
You sure downt sound like you are for whistleblower. I dont many people forget that the NSA violated many US laws.
Re: ...and that makes it better? (Score:5, Insightful)
You know, whether you agree or disagree with what Snowden did, that in no way justifies killing him without a... oh, what was that quaint thing we used to require? That's right, a trial. Rule of law, and all that. I think that's what the country was based on originally.
Of course, it's embarrassing for the NSA that Snowden waltzed out with so much confidential information, and arguably he should have been ARRESTED within 24 hours of "that flight to Hong Kong", but killed? To even think that sort of thing is disturbing.
Having said that, I am glad he managed to get away, since his revelations are shining an absolutely necessary light on the murky behavior of our government and its actions. An educated populace is necessary to ensuring our freedoms and for too long the government has been hiding its wrong-doings from the ones it purports to serve. Whether Snowden acted as a foreign agent, or for his own advantage, or out of idealism, his actions were necessary and should not be so readily scorned.
(oh right, and fuckbeta and all that jazz. It's gonna be hard doing that boycott tomorrow)
Re:When you are a criminal (Score:5, Insightful)
Also, I don't care what think your motive is, you don't turn over classified documents to the enemy.
And there it is ... the free press is nurb432's enemy.
Snowden is a patriot; the NSA is treasonous (Score:5, Insightful)
The idea of military specialists of whatever type being employed against the society they belong to, is treasonous and fucking retarded no matter what legal acrobatics are employed in their defense.
You may have some sort of mystic devotion to the law, but I believe laws are made by (generally corrupt) men for their own interests, and I am familiar enough with the world outside the borders and political influence of the United States to know there is an enormous difference between legality and rightousness. The U.S.A. may not be the kind of country where you are expected to bribe every public official however minor -- we generally reserve that for higher office. It takes a special kind of idiocy to use military forces against their homeland, though.
Government at its core is the body to which we have delegated our inherent right to violence -- a right being defined in this case as something which cannot be taken from you. We delegate this right to others, specialized in its use, with the express understanding that [a] as applied to civilian life, the exercise of violence by police will be applied fairly and equally as men can manage, and [b] that the unrestricted expression of this (as embodied by military force) be only employed against our enemies. War is hell, and we do not bring hell home.
Snowden is a patriot, and the NSA is treasonous -- whether or not the law can be made to serve whichever purpose. Beyond all other argument, potentially felonious violation of the law is so common with the continual proliferation of laws that lawfulness cannot be the only measure of either justice or rightousness. May all those who support the NSA have a fair trial.
Re:Wow... (Score:5, Insightful)
From TFA:
Agency officials insist that if Mr. Snowden had been working from N.S.A. headquarters at Fort Meade, Md., which was equipped with monitors designed to detect when a huge volume of data was being accessed and downloaded, he almost certainly would have been caught. But because he worked at an agency outpost that had not yet been upgraded with modern security measures, his copying of what the agency's newly appointed No. 2 officer, Rick Ledgett, recently called "the keys to the kingdom" raised few alarms. "Some place had to be last" in getting the security upgrade, said one official familiar with Mr. Snowden's activities. But he added that Mr. Snowden's actions had been "challenged a few times".
So they knew he was doing it, even questioned him, and he still got away with the data. To the people who maintain the NSA has the best and brightest security people perhaps they (NSA security) should use that expertise to improve their own security instead of weakening everyone else's.
And yes, this is precisely why they must not be trusted with the data they are gathering due to mass surveillance.
Re:How Many More NSA Employees? (Score:2, Insightful)
... to their real enemy - the general public.
That is such a load of crap.
Why? I'm quite sure that most governments at the very least had a general idea of what the NSA was up to before Snowden's leak, it's only members of the general public that would get painted as conspiracy theorists and ignored whenever the they tried to draw attention to this sort of thing.
Re:How Many More NSA Employees? (Score:3, Insightful)
As others have said: "How so?"
It's a logical conclusion based on the available evidence: No safeguards were in place to defend against an analyst stealing data and giving it to someone else, despite this being an obvious threat the NSA could not possibly have been unaware of.
No such measures were taking until someone (i.e. Snowden) leaked this information to the public. Add this to the extremely negative way in which the NSA and the entire administration talks about journalists reporting on this, and the response to other whistleblowers and this really is the most likely explaination.
Re:Stunning. (Score:3, Insightful)
I think many here are missing the point. Point the Moon with your finger and the fool will look at the finger.
The entire point about the use of automated tools to scrape data here and there on the NSA network is that Snowden wasn't going at the only data he needs to prove the point he says he wants to make. He was just grabbing a full load of data hoping for some of it to prove something that could make him a credible whistleblower. This is playing against him if he would have to convince a judge he is a "legitimate" whistleblower just wanting to free America from the dictatorship of the NSA. He did grab way too much data and some of it having no relation at all with the point he supposedly wants to make. But worst, he just gave everything to a third party rather than just what was necessary to prove that point about the NSA and the violation of US citizen rights.
So, the point is not about the automation of the data scraping, it is about the indistinct data scraping itself. And NSA may make a point here.
Re:Stunning. (Score:5, Insightful)
You're ALL missing the point.
This is not a "news item" because he used some "arcane technology" from the view of the uninitiated.
This is a calculated iteration in the advancement of declaring web automation tools "instruments of terror" and "cyber weapons", for prosecutorial purposes.
You vilate a ToS with these, and WHAM! Terror suspect!
Re: ...and that makes it better? (Score:4, Insightful)
You know, whether you agree or disagree with what Snowden did, that in no way justifies killing him without a... oh, what was that quaint thing we used to require? That's right, a trial. Rule of law, and all that.
If there were to be a trial it is almost certain they would exclude pretty much all avenues of defense [techdirt.com] that support what he actually did and why. Rule of Law is no more in this country. Just ask Aron Schwartz, Bradley Manning and the host of other whistle blowers prosecuted by the self proclaimed most open administration in history. If they want you gone they simple twist the millions of laws that exist and make up new interpretations if that's not enough. But you will be gone.
Re:Stunning. (Score:5, Insightful)
Actually there are options in wget for that.