Former FBI Agent: All Digital Communications Stored By US Gov't 621
New submitter davesays writes "CNN anchors Erin Burnett and Carol Costello have interviewed Former FBI Counterterrorisim specialist Tim Clemente. In the interviews he asserts that all digital communications are recorded and stored. Clemente: 'No, welcome to America. All of that stuff is being captured as we speak whether we know it or like it or not.' 'All of that stuff' — meaning every telephone conversation Americans have with one another on U.S. soil, with or without a search warrant — 'is being captured as we speak.' 'No digital communication is secure,' by which he means not that any communication is susceptible to government interception as it happens (although that is true), but far beyond that: all digital communications — meaning telephone calls, emails, online chats and the like — are automatically recorded and stored and accessible to the government after the fact. To describe that is to define what a ubiquitous, limitless Surveillance State is."
Jupiter Tape? (Score:3, Insightful)
I doubt they have the storage capacity.
Re:Jupiter Tape? (Score:5, Interesting)
More than storage capacity, they need software and/or manpower to analyze everything. More likely his superiors just lied to his face about this or he was paid to say such things to make people think twice about doing any rebellious shit.
Re: (Score:3)
More than storage capacity, they need software and/or manpower to analyze everything.
It's doable, whether it's doable currently I can't say. Remember that the STASI [wikipedia.org] did exactly this, and were able to comb through all personal communications.
Re:Jupiter Tape? (Score:4, Insightful)
Remember that the STASI [wikipedia.org] did exactly this
The Stasi did only did the easy part: monitoring everyone.
The didn't do the hard part: keeping it secret.
Re:Jupiter Tape? (Score:5, Funny)
There was a nice joke about the "inconspicuous" nature of Stasi surveillance:
Q: How can you tell when the Stasi has bugged your apartment?
A: You find an unexplained large cabinet in the apartment, and on the street a trailer with a diesel generator has parked...
Re:Jupiter Tape? (Score:5, Insightful)
Re: (Score:3)
"Oh, I know why you don't like interception capabilities, Mr Senator, but what can you tell us about your frequent visits to sexymilfinbestialbondage.com ?"
Re:Jupiter Tape? (Score:5, Interesting)
Not to assert one way or the other whether he's telling the truth, but...
It's much more sensible to record everything and keep it for a short while and then begin a process of attrition. If everything is accessible for 1 hour, that's pretty powerful because you can freeze data after an event happens and look for what you need. After one hour, maybe only certain things and certain people are tracked for up to a day... then a week... a month... a year...
Re: (Score:3)
Re:Jupiter Tape? (Score:5, Informative)
There's not a lot of storage necessary. Not what you're thinking. Text messages and chats are very small in size. Phone conversations are very small using the right codecs. I also heard once about 8 years ago that the US government was buying up symmetrix like they were going out of style.
Honestly, I believe it. It's entirely possible.
Re: (Score:3)
I doubt they have the storage capacity.
ha ha, no. [slashdot.org] They totally do. Did you know that in 2011, 680 million [quora.com] drives were shipped? Do you really think a couple exobytes of data would even be a blip on the radar? That's only a few thousand drives.
Re:Jupiter Tape? (Score:5, Interesting)
http://www.zdnet.com/blog/storage/worlds-1st-exabyte-storage-system/1266 [zdnet.com]
All US telephone conversations per day approx: 1.5 Petabytes
Fits easily.
Room for 2 years worth, or are you going to tell me US govt 3-letter agencies don't spend much on data centers.
https://en.wikipedia.org/wiki/ECHELON [wikipedia.org]
Re: (Score:3)
Oh and,
http://www.popsci.com/technology/article/2011-05/every-six-hours-nsa-gathers-much-data-stored-entire-library-congress [popsci.com]
Re: (Score:3)
He could be talking about the ubiquitous and insecure nature of all on line digital communications. Meaning, everyone is constantly leaving trails that the government can access years later. Which seems far more plausible.
But who knows, maybe they have above top secret alien positron brain quantum foam storage technology (or insert your own fantastic technobabble) buried at area 51...
Aquinas Hub anyone? http://en.wikipedia.org/wiki/Deus_Ex [wikipedia.org]
Re:Jupiter Tape? (Score:5, Insightful)
They may sift it for key-words, and record who, where, and when, but not necessarily the entire conversation or transaction.
Re:Jupiter Tape? (Score:5, Interesting)
Well we already know they have the means to do keyword searches on a great deal of data. Carnivore has been debated openly before congress so I would definitely have to agree thats the minimum capability to expect.
Re: (Score:3)
I would say no to ROT13, probably can be monitored more or less in real time. I would also think that any particular juicy targets for the real intelligence community (I'm not talking about the bullshit they use to warmonger and perform psyops thats constantly flaming Iran and China) but the real people doing real work that might actually be making a real difference behind the scenes, if they do in deed exist and it is at all possible (I don't have any idea) would be using more then just interception, they
Re: (Score:3)
What about lolcrypt? http://lolcryption.master5o1.com/ [master5o1.com] :P
They Might Be Giants (film) (Score:3)
Re: (Score:3)
Actually after reading through all of the discussion here it sounds to me much more plausible that the government might actually be trying to log everything off a few major backbones. I could go either way. I always assume someone may be watching what I do on line. I've always operated like that anyway. Now there are options for making anything you do on line hard decrypt. There's other ways of hiding real information in plain sight, or using good encryption as well as other methods to obfuscate the sources
Re:Jupiter Tape? (Score:4, Interesting)
Actually after reading through all of the discussion here it sounds to me much more plausible that the government might actually be trying to log everything off a few major backbones
That's what I'd do.
But I am certainly not qualified to go about telling anyone how to covertly access a computer system anonymously. Nor would I pretend that I could. That would require resources, mobility, knowledge, and access to things I don't have. Nor would I want to do any of that anyway. At least not to post what I do.
Not really. All you need is a pringles can, a wifi card with an SMT connector, and a couple pieces of software. Anonymity and mobility is easy, and the knowledge isn't specialized.
The problem isn't access, but entropy. The more you use a system, any system, the more ordered the access becomes; That is, before you access the system, you could say the entropic space to search to find your attempt is infinite, and with every interaction, the entropy reduces until eventually it reaches a threshold where the pattern becomes statistically unique. What most people don't realize... is how little time and interaction is required to reach that point.
The more you use a system, the easier it becomes to tie that use to an identity. Via comparative analysis with other identities, it eventually becomes possible to link it to a specific person. What this means then, is that in essence, no matter what methods you use to access a given system, the mere act of accessing it, independently of anything else, decreases your anonymity. And the more access, the less anonymity.
You can increase the complexity but you cannot prevent convergence to unity. Anonymity gradually drops to zero.
Re:Jupiter Tape? (Score:4, Interesting)
Re:Jupiter Tape? (Score:4, Insightful)
The average cell phone usage is 459 minutes/month * 300 M cell phones / 2 * 60 sec * 3 KB/sec = 13,000 PB/month (uncompressed).
Re:Jupiter Tape? (Score:5, Informative)
The average cell phone usage is 459 minutes/month * 300 M cell phones / 2 * 60 sec * 3 KB/sec = 13,000 PB/month (uncompressed).
Why the "/ 2"/ Assuming that every phone call made from a cell phone is also to a cell phone? And not doing compression, but doing dedup?
I don't know how others use theirs, but most of my phone calls aren't social, but to businesses and their land lines.
Anyhow, CTIA [ctia.org] lists 2.30 Teraminutes yearly per December 2012. Presumably that's also counting cell-to-cell twice, which I'm sure the three letter agencies would record twice too (if nothing else to record what was said when there's a drop-out in the connection on either end). That's 138 Teraseconds, which at 3 kB/s would be 414 PB per year, before compression.
That's a far cry from your 13,000 PB per month (or 156 EB per year), and spread out over multiple providers absolutely doable from a capacity viewpoint. Especially since it doesn't have to be online for a year, but can go on tape. If ten datacenters recorded this, with a fluctuation of 40% in data density between them, and flushed everything to tape within a week, each would need less than 2 PB of online storage.
But do I believe they do so? No. If they did, they wouldn't have a way to mine the data. It would possibly be useful as evidence after the fact, but not for monitoring purposes. It's way too much data.
Re:Jupiter Tape? (Score:4, Informative)
Nobody needs to actively mine the data. The goal would be to collect it. Once you've collected it, you have the ability to follow leads you wouldn't have been able to follow had you not captured it in the first place.
You become aware that an individual may be a person of interest. Ordinarily you'd begin your investigation at that point. With this technology you can now go 'back in time' and figure out not only who that person spoke with, but exactly what was said in those calls. It would be incredibly useful.
I could even see Executive Branch lawyers convincing themselves that this was legal, provided the communications were not actually accessed without some sort of due process.
Of course, the problem with this theory is that it would be very hard to implement, since it would require massive and detectable changes to local telco infrastructure. On the other hand, intercepting wireless communications could be done without any such tampering, provided that the government could obtain a database of SIM credentials for decryption.
Re: (Score:3)
For the Boston Marathon bombers, this would have been a perfect investigative tool. Once you have the phone number of a target, you simply scan backwards through all of their recorded calls.
When I say nobody needs to mine the data, I don't mean nobody every looks at it. I simply mean that you don't mine it in real time. You simply record the text along with the call metadata, and wait until you have some specific targets to investigate. At that point you construct a graph from that starting point, and go ba
Re: (Score:3)
There's a huge difference between this claim and lawful intercept on demand -- meaning that a formal request is made to the Telco to intercept such and such number for a period of time, then the calls are re-routed to special recording equipment.
In this case you'd need to have active real-time recording capability for every call made on every switch in the entire national phone network. You'd also have to hide this capability from the techs who work on the switches and/or swear them all to secrecy. That wou
Re:Jupiter Tape? (Score:4, Informative)
There's no uncompresed 22kHz audio anywhere in the phone network system. You simply budget 64kbits/s for one direction in a voice call, that's also called DS0. That's what the analog last mile gets converted to and hauled as worldwide. It's 8 ksamples/s, at 8 bits per sample, using nonlinear A-law or mu-law lookup table. Every fax or modem connection gets hauled that way as well, and it works by design :)
Re: (Score:3)
cell phones (Score:3)
Re:Jupiter Tape? (Score:4, Informative)
I've heard rumors that AT&T captures all, but never anything that confirms that. Perhaps he meant that the major carriers provide streaming replicas of all traffic to the government, who then archives some (or all) of it. But I know for a fact that "all" is just plain wrong.
Re:Jupiter Tape? (Score:5, Informative)
"'ve heard rumors that AT&T captures all, but never anything that confirms that. "
It most certainly is confirmed. In a court case some years ago, a technician outed that the government had installed a splitter in a special room in one of their exchanges, which fed ALL of their digital data straight to the government. The telcos involved admitted that it was only one of many such. Mass collection, and no warrants involved, anywhere.
In fact, that was the whole reason Congress voted to give telcos immunity, remember? How short our memories can be.
Re:Jupiter Tape? (Score:5, Insightful)
I have been wondering for several years where the public uproar over this is. It's a crime against The People... as defined by our own laws.
Re:Jupiter Tape? (Score:5, Informative)
Sovereign immunity [wikipedia.org]... and it has existed for far longer than this nation has.
Re:Jupiter Tape? (Score:5, Informative)
A switch room that contains a deep packet scanner is not the same thing as sending ALL internet traffic to a storage system. More likely it's just a tap.
The thing that makes me doubt this is the cost. To funnel a copy of the internet to the Feds would require building a shadow internet plus storage for the whole thing.
Re:Jupiter Tape? (Score:5, Interesting)
"A switch room that contains a deep packet scanner is not the same thing as sending ALL internet traffic to a storage system. More likely it's just a tap."
No, the public record is clear. It's not a "switch room", it's a splitter. And yes, the technical and expense implications of that have been debated and re-debated, re-hashed and triple-warmed-over.
They are splitters. And they send ALL the digital data on fiber that enter those exchanges directly to government. No packet inspection (at those locations, anyway), and no "taps". Just a "Y" in the fibers.
(Yes, I realize that technically it's quite a bit more complicated than that, because it involves amplifiers and lots of other things. This, too, was brought up in the court case. But that's what it is. It's in the public court records. Again: that's why the telcos were given immunity. Where were you when this was all going on?)
Re:Jupiter Tape? (Score:5, Informative)
Here is Klein's statement.
https://www.eff.org/files/filenode/att/SER_klein_decl.pdf [eff.org]
The splitter sent the internet traffic to a secure room.
And another interview with Klein:
http://www.pbs.org/wgbh/pages/frontline/homefront/interviews/klein.html [pbs.org]
It's pretty obvious that room contained a Narus DPI. End of story.
Re:Jupiter Tape? (Score:4)
For the less tech oriented, what is a Narus DPI and what does it do?
Re:Jupiter Tape? (Score:5, Informative)
DPI = deep packet inspection. A TCP/IP packet contains header(s) and body. The header tells you the address and some info re the protocol. The body is the content. Most internet infrastructure only cares about the header. Something that is capable of DPI will recover the content of some types of packets. Which ones depends on the capability of the DPI unit you use.
So if you want to search for email sent to joe@there.com you need to use DPI because the email header is in the body.
Use of a splitter is step one for DPI.
This article talks about the AT&T / Mark Klein incident:
http://en.wikipedia.org/wiki/Deep_packet_inspection#United_States [wikipedia.org]
Re: (Score:3)
AT&T doing warrant free capture of arbitrary domestic communications is a matter of legal record. They were granted immunity from prosecution when their cooperation with NSA monitoring was exposed by a whistleblower employee. One of the critical facilities was referred to as "Room 641A". There's a reasonable Wikipeda article at http://en.wikipedia.org/wiki/Room_641A [wikipedia.org].
I've seen no reason to believe that this practice has stopped, merely to believe that they've gotten a bit more subtle about it. The bent f
Re:Jupiter Tape? (Score:5, Insightful)
So the guy is lying?
Probably. He is trying to make a name for himself as a consultant. This claim will give him some publicity. If his claim was true, hundreds of people would know about it, and all of them would know they were breaking the law. Some of these people would be in government, but many of them would be in telecom companies, that would have no reason to cooperate, and plenty of reason not to (losing customers, end of career, prison time, etc.). Of course, no amount of logic or absence of evidence this will stop the conspiracy theories (see below). Of course, if it actually was true, the FBI would probably hire shills to go on Slashdot and spread disinformation, and try to convince everyone that there was no vast conspiracy, so why should you trust me?
Re:Jupiter Tape? (Score:5, Insightful)
Re:Jupiter Tape? (Score:5, Insightful)
Probably. He is trying to make a name for himself as a consultant.
He worked in the FBI's counter-terrorism division. I don't think he needs to "make a name" for himself. His resume already says enough.
If his claim was true, hundreds of people would know about it, and all of them would know they were breaking the law.
And as we know, government officials never break the law. Glad we cleared that one up.
but many of them would be in telecom companies, that would have no reason to cooperate,
They have guns. Lots of guns. Feeling lucky, punk?
). Of course, no amount of logic or absence of evidence this will stop the conspiracy theories (see below).
"Conspiracy theories by a former official in a credible position to know these things." FTFY.
Of course, if it actually was true, the FBI would probably hire shills to go on Slashdot and spread disinformation, and try to convince everyone that there was no vast conspiracy, so why should you trust me?
Why would the FBI give two shits about a geek news site? And why would they need to convince you, me, or anyone else, there wasn't a "vast conspiracy"? You're making a straw man here. A big one.
Re:Jupiter Tape? (Score:5, Insightful)
By this logic we should also believe Bob Lazar about alien technologies. Extraordinary claims require extraordinary evidence and the word of one insider means squat.
His "extraordinary" claim is that the government is doing what Google has already done: Indexing the entire internet, with the only difference being one of scale. I think if a private company started by a few college kids can do that, a government with nearly a trillion dollars in operating budget can come up with something.
So your argument is that they will cooperate or be shot? This isn't even a sane way for a conspiracy to operate.
Correct. But then, it's not a conspiracy. He's suggesting a corporation wouldn't cooperate with law enforcement requests. That's patently absurd; Corporations have little to gain and loads to lose if they decide to go against the government. Multi-billion dollar corporations aren't known for behaving like belligerant teenagers.
The point is that the scale of the conspiracy necessary to pull this off is vast enough to make this extremely unlikely.
Except it isn't a conspiracy; we're talking about a national 'darknet' that taps key points in the internet and then mirrors that data to a processing facility before being stored in a relational database. It's not a conspiracy, numerous government officials have already gone on the record as saying this technology exists, today, now. It's not classified. It's not a secret. They've come right out and said this capability exists.
LOL
A former agent for the counter-terrorism branch of our largest federal law enforcement agency talking about the technology used in counter-terrorism is about as credible as it gets, dude. LOL all you want, but your own cred is the only thing the rest of us are laughing at: You're an internet pundit.
Re: (Score:3)
You assume the FBI would have to go out and hire people to act as shills, as opposed to already having them on payroll.
Maybe I only pretended to assume that to divert your suspicions, and I have actually been on the FBI payroll for the whole time.
Re:Jupiter Tape? (Score:5, Interesting)
So the guy is lying? Perhaps. Or just exaggerating. But I doubt there isn't more than one data center for this very purpose. The question is what kind of hardware would be necessary to compress all the data live.
If he wasn't lying, he would be arrested for violation of his LIFE LONG NDA you sign when you take a job with the The FBI.
So if he is picked up for tax evasion or some similar nonsense charge, then I'll start to believe him, but until then, I suspect he has a book he is peddling now or in the near future.
People should remember just how terrible Americans are at keeping a secret. Someone would have leaked this long ago, just as the secret room at the AT&T switch center was leaked within a couple months.
It wouldn't come from a lowly guy hyping a book.
Re:Jupiter Tape? (Score:4, Insightful)
People should remember just how terrible Americans are at keeping a secret
How long did the Manhattan Project employ thousands of people before anyone figured out what they were making?
Re: (Score:3)
About zero days? [wikipedia.org]
Re:Jupiter Tape? (Score:5, Insightful)
If he wasn't lying, he would be arrested for violation of his LIFE LONG NDA you sign when you take a job with the The FBI.
That's such a load of bullshit I'm surprised I'm the first to call you out on it. The overwhelming majority of the FBI's records are public; they're called court cases. And yes, the agents can discuss them. The agents can even discuss the methods that they use. The only thing agents can't discuss is material related to an active or ongoing investigation, or material that has been classified. There no evidence that either condition has been met here.
People should remember just how terrible Americans are at keeping a secret.
Yup. Everybody knows how to build nuclear weapons, stealth bombers, ICBMs, because we're terrible at keeping secrets. So, tell me, what's the maximum listing angle that any of our nuclear submarines can operate at before they automatically shut down? You don't know? Okay, how about this one: When is the next high-energy test of the HARRP? Don't know that one either? Umm, how about a real easy one: Who's the pilot of Air Force One? Ah, didn't think so.
Someone would have leaked this long ago, just as the secret room at the AT&T switch center was leaked within a couple months.
Yeah, it's a super duper big secret that the government can tap phones and digital lines. Dude, you make this leak sound like it was some kind of blow to our nation's intelligence operations, rather than having all the relevance of knowing the President ordered his latte with skim milk this morning. It wasn't a secret; It just wasn't advertised. There's a big difference. You won't find our nuclear missile silos in North Dakota on google map with the words "Secret Nuclear Missile Silos Here" underneath; But that doesn't mean they aren't there, nor does it mean that there's extra-special effort being taken to keep them secret. They just aren't advertised -- everyone knows they're there.
It wouldn't come from a lowly guy hyping a book.
Yes, a "lowly" agent of our largest law enforcement agency, discussing something that he did professionally for many years, and the tools he used to do that job should be trusted less than random internet pundit "leaking" the same information.
Re: (Score:3)
...he would be arrested for violation of his LIFE LONG NDA you sign when you take a job with the The FBI.
Never heard anything about that from my FBI friend.
Well, of course not -- he wasn't allowed to tell you about it ;^)
Re:Jupiter Tape? (Score:5, Interesting)
Narus
For more than 26 years, Dell has empowered countries, communities, customers, and people everywhere with the right technologies to realize their dreams.
Blue Coat
Dell is a strategic reseller & global systems integrator for Blue Coat’s products. Blue Coat’s products are available through the Dell Software & Peripherals catalog for a variety of Secure Web Gateway, WAN Optimization & Visibility solutions.
Dell's Sunnyvale offices are at 909 Hermosa Ct Sunnyvale, CA... not on the same street, but physically adjacent to Blue Coat's campus. Its building is about 40 feet from Blue Coat's... for Dell employees, it's a shorter walk to Blue Coat than it is to some of their own cars in the parking lot [google.com].
Spelled out: Blue Coat and Dell work together to sell governments equipment to monitor their citizens' communications. And so do Narus and Dell.
Re: (Score:3)
Compress? This is the gov't. If they can capture everything, they have the money to store it raw. Whether it takes 1x storage space or 10x storage space is only dollars, which they print.
Even given unlimited dollars, there is only so much hardware that the world's electronics industry is capable of producing each year. Therefore even the all-powerful MIBs will have limits on what they can afford to do.
From what I've read, a lot of surveillance doesn't bother analyzing the actual content of the communications at all; instead they just keep track of who was communicating with who at what time(s), and use statistical/data-mining techniques to draw conclusions from that.
Re:Jupiter Tape? (Score:4, Funny)
As long as you're okay with them redacting half of it.
Just how much storage capacity would one require? (Score:3, Insightful)
Someone correct me. It just doesn't seem feasible.
Re: (Score:3)
Pshaw. I've got 26 fault-tolerant terabytes just for my media. Every piece of hardware is available from regular vendors like newegg, amazon, microcenter, etc. It fits in a mid-tower case with room to spare and the capacity could be doubled by switching to 4tb drives.
Google was expected to pass an exabyte of data years ago. Amazon's somewhere around that range with their cloud services. Facebook claims a petabyte of duckface pics and videos. And those are companies that are designed to be making a pro
Re:Just how much storage capacity would one requir (Score:5, Interesting)
They do have good off line secret installations. Where security is really important. I went to Ft Huachuca for computer security training and they did a fairly decent job for low level IT staff there. The instructors and some of the other people at that base genuinely knew what they were doing.
DoD care a lot less about every day logistics systems for barracks assignments then they do about keeping under wraps their mission planning or god knows what else, I couldn't get anything out of anyone that mattered.
I'm no longer serving. But you can rest assured there are some in the service that are good at what they do.
Re:Just how much storage capacity would one requir (Score:5, Interesting)
Think of that: you're adding 640 terabytes to your database every minute.
Re:Just how much storage capacity would one requir (Score:4, Funny)
Good point (Score:2)
see subject
Seems unlikely (Score:4, Interesting)
I'm sure what he's saying is true, in a very broad sense. Cell phone conversations, texts, and major/popular version of things like video chat (skype), IM (yahoo messenger), and general social media (facebook, twitter, etc).
This guy seems to be implying that the government has some kind of man-in-middle technology that intercepts and records *all* traffic, which simply isn't true. Unfortunately, either he or the news agency is trying to paint the whole thing as just that.
Re: (Score:3)
I don't doubt it at all. Maybe they can't get every little scrap but you can bet that anything they can get they will get. I wouldn't doubt that they have a lot more capability for this than you think either. I remember what was state of the art 30 years ago in the commercial world and it's incredible the changes and I'm more than confident that intelligence gathering has made at least as many leaps and bounds.
Re: (Score:3)
Tracert is ICMP, and can easily be routed differently than other traffic. ICMP also relies on hops telling you that they exist, both by generating an ICMP response and by decrementing the TTL. It also wont have any way of knowing whether a switchport is being mirrored (in fact, there is no technical way for an end user to discover that).
Re:Seems unlikely (Score:5, Interesting)
This guy seems to be implying that the government has some kind of man-in-middle technology that intercepts and records *all* traffic, which simply isn't true.
The majority of the internet goes over telecom links. The few parts of it that don't, almost always have at least one hop that transits one of the major carriers networks. They don't have to monitor "all" traffic. They just have to monitor one of the hops in the chain.
All IP traffic can be reduced to a stream. TCP/IP has some extra error-correction options to keep it all in order, but a stream is a stream nonetheless. And when you start looking at very large data sets, you're going to quickly discover that the majority of it is just a copy of another set of data.
People here seem to think that monitoring all network traffic is unrealistic because of the storage considerations, but they don't have to store every byte; Just the unique bytes. If you download the CNN homepage, the storage application doesn't need to hold onto that entire transaction; It can just record the headers and timestamp, and then reference the same stream that a few hundred thousand other people also downloaded.
Most of the internet's traffic isn't encrypted, and so the amount of entropy on it is low, despite the very high bandwidth. This statistical fact paired with shannon's laws, which in turn are based on the laws of thermodynamics, provide the basis of a practical surveillance solution.
When you add in intelligent filtering, the amount of data to be stored drops even more. You probably don't need to worry about terrorists communicating via Netflix for example; And that makes up a significant chunk of internet traffic (look it up; it's a surprise).
The other thing about intelligence assets is that they all have a 'use by' date. The more time goes by, the less valuable the data becomes. Eventually, you reach a point of diminishing returns; That is the point at which you can safely delete the data. It doesn't matter whether it contained terrorist communications or the next 9/11 or not... if you haven't found it by the cutoff time, it's worthless.
Combine these attributes and what this man is saying is, in fact, achievable. Now... processing that data and turning into useful, timely, and accurate intelligence... that, people, is a whole 'nother can of worms. And realistically, where the bulk of the resources is going to be. Storage is a non-starter. Analysis is the bitch of it.
Re: (Score:3)
Most of the internet's traffic isn't encrypted, and so the amount of entropy on it is low
Most of the internet's traffic however is compressed, on which the entropy is high. Voice communications and video streams are definitely all compressed. Web pages can be sent zipped and often are.
A lot of content is also dynamic, containing timestamps, unique identifiers or personalisation. Webpages with any dynamic content (like a page from CNN) is likely to have subtle differences each time they're requested, making deduplication harder.
Please! (Score:4, Interesting)
Either this is true and so secret even most of law enforcement doesn't have access or it simply isn't true. Having run a large enough telecom operation to deal with CALEA I can say for sure that law enforcement very much needed our help to do anything with our customers' communications. Not only did they need to come to us with proper warrants in the first place, but they barely had enough technology sense to be able to do anything with it. Anything more complicated than taps and CDRs never even came up.
Citizen reply. (Score:5, Interesting)
Clemente: 'No, welcome to America. All of that stuff is being captured as we speak whether we know it or like it or not.' 'All of that stuff' â" meaning every telephone conversation Americans have with one another on U.S. soil, with or without a search warrant â" 'is being captured as we speak.' 'No digital communication is secure,' by which he means not that any communication is susceptible to government interception as it happens (although that is true), but far beyond that: all digital communications â" meaning telephone calls, emails, online chats and the like â" are automatically recorded and stored and accessible to the government after the fact. To describe that is to define what a ubiquitous, limitless Surveillance State is."
Dear US official;
All of my communications are sent via encrypted proxy, and set to stream constantly. The proxy dumps into Tor and a half-dozen other networks. I originally did this for shits and giggles, to see how hard it would be. I will admit the latency is a bit higher than doing it locally, but it is very usable in spite of this. I also signed up for the Tor Cloud project and run an EC2 micro-instance to help others do the same.
Originally, I just did this as an experiment, but after reading things like what you're saying and realizing that we've become a surveillance state on par with Iran, China, and North Korea (where did they get their filtering and monitoring hardware from again? Oh right: We gave it to them), I decided to keep it.
I don't do anything special with my super duper encrypted "all the things" setup. I wish I could say I was some elite ninja hacker or something, but all I really do is browse internet forum sites and read the BBC news, and you know, download a few TV shows here and there. I'm one of those people that doesn't have anything to hide per-se, but when I live under the tyranny of a government that has turned their citizens into the enemy -- the attitude that we're all criminals or potential criminals, and must be monitored pre-emptively, I feel like it's my duty to frustrate the hell out of people like you.
So I have been helping friends, family, and strangers, set their computers up the same way. Yeah, I know, some of them will probably use their newfound freedom and anonymity for evil, but frankly, even a terrorist attack a week and all the rantings in the world from you (that may even be justified) about how criminals can use this technology for their own nefarious purposes, doesn't deter me.
You crossed a line; Morally, ethically, constitutionally. By criminalizing the average citizen, you have become a bigger danger than all the terrorists, all the "real" criminals. You are corrupt, dangerous, and seek to undermine our democratic way of life. You hide in the shadows and see conspiracies everywhere, and are convinced of your own righteous cause. You are as dangerous as a religious fundamentalist, because just like their dogmas, yours demands absolute purity. There will always be more justifications to invade the privacy of others.
So I will continue to teach anyone who wants to, how to fight back against your tyranny. You're a threat to the way of life of not just myself, but my peers. You're a danger to all Americans -- you view us as the enemy. Your own people.
You've lost your way.
Re: (Score:3)
Might want to watch this Defcon presentation. Trusting a random tor node is a bad idea.
http://www.youtube.com/watch?v=kLt_uqSCEUA [youtube.com]
Re:Citizen reply. (Score:4, Insightful)
Have you written a detailed HOWTO article? That would be more helpful than just helping your immediate acquaintances.
Re:Citizen reply. (Score:5, Funny)
Dear US official; All of my communications are sent via encrypted proxy, and set to stream constantly. The proxy dumps into Tor and a half-dozen other networks.
Dear girlintraining,
We're in ur USB keyboard driver
Recordin all ur passwerds
- lolMIBs
Re: (Score:3)
(where did they get their filtering and monitoring hardware from again? Oh right: We gave it to them)
You might want to address your "open letter to a US official" to corporate America as well... they're happily selling the tools of oppression for profit. Come to think of it, they make big profits off selling guns and ammo as well.
Re:ps. (Score:5, Interesting)
unless of course the freedoms that you want are verboten by that power..
Re:Citizen reply. (Score:5, Insightful)
Not that America is perfect, but you have no fucking idea what tyranny is.
tyr-an-ny, n.: (source: dictionary.com)
1. arbitrary or unrestrained exercise of power; despotic abuse of authority.
"All of that stuff is being captured as we speak whether we know it or like it or not."
2. oppressive or unjustly severe government on the part of any ruler.
"meaning every telephone conversation Americans have with one another on U.S. soil, with or without a search warrant -- 'is being captured as we speak.'"
3.undue severity or harshness.
"'No, welcome to America."
Would you care to revise your statement, Mr. Internet Pundit?
Re: (Score:3)
It's actually worse then that in China. I read about some version of a Chinese town hall meeting were the local authority asked a guy there what he thought about a religious cult. Some form of Buddhism that believes in what not. There had been some conflict with members of this religion and the authorities before. The guy said he didn't think anything bad about the cult. They then proceeded to beat the guy senseless charge him with insanity and drag him off to prison.
He died a few months later of self induc
Checks and balances, anyone? (Score:2)
This is probably the most telling bit of it:
CLEMENTE: We certainly have ways in national security investigations to find out exactly what was said in that conversation. It's not necessarily something that the FBI is going to want to present in court, but it may help lead the investigation and/or lead to questioning of her. We certainly can find that out.
Basically, this capability exists, can and does get used, BUT the fruits of which aren't directly presented in the courts, because that would divulge too much as to its existence. Instead, it gets used to get the suspect to admit what might otherwise be unattainable through a normal interrogation.
Now the scary part:
This could probably directly provide evidence for not just the Boston Marathon case, but many many other criminal cases in this country right now. Fo
May we assume (Score:2)
That in that case classification is done more or less automatic.
Down to the individual, we may assume.
And yet, they are unable to prevent [insert latest bombing or amok event here].
Clearly a government gone anal.
Timothy stories (Score:2)
No one else would post such obvious crap.
It is impossible for the government to record all digital communications because they aren't privy to them. Unless you mean to tell me they have a back door in my open source mail servers which communicate on their own private networks within my organization over physically secure channels.
They don't ever get access to a lot of digital communications so they can not possibly be recording it all.
typical timothy story, so blindly false, yet somehow the moron keeps pos
Re: (Score:3)
No one else would post such obvious crap.
eh?
it doesn't matter whether you agree with it, or not. It doesn't matter who posted in on slashdot. The fact remains a former FBI official made this statement, not once, but twice, to CNN. Is it true? who knows, but don't go and blame the messenger.
Re:Timothy stories (Score:5, Insightful)
While the FBI might not be the sharpest tool in the shed they have infinite resources thanks to the national security black budget. Granted if you have a private network that doesn't peer with any of the big backbones like AT&T then your a probably safe. Once your voice/data hops onto a major backbone like AT&T your data has been sold to the US Government. There are even allegation that this system is contributing names to the no-fly list through heuristic language analysis of voice traffic. To see if you networks are safe, use the word "bomb" as often as you can and if you aren't added to the no-fly list, your networks are probably safe :)
I think the real solution to the problem is to start generating massive amounts of meaningless data until the spooks run out of storage.
Security and Privacy are an illusion. Welcome to 1984 about three decades late.
lost email? (Score:5, Funny)
Could I ask them to restore that email I accidentally deleted last week?
Not true (Score:3, Informative)
I work for a moderate sized phone company. Customers in the millions. The government has no link into our systems. WE can't even record all of your calls and monitor all your internet traffic. Think about it this way, your ISP likely doesn't even have enough bandwidth to provide you with the speed you're paying for some of the times... Netflix on a friday night for example. Do you really think they have the extra bandwidth to ship all that data off to the government as well? Phone calls are a whole other animal, and are mostly still analog. Duplicating that would involve upgrading the switch... an at least 30yr old piece of obsolete equipment... It just doesn't make sense. Sure, the government could pay for all this stuff... but it would be a HUGE project. Everyone in the company would know. The equipment in our data center is very obvious... we all know what each piece does. There's no mysterious black box in the corner... and there's no way they could be tracking everything without us knowing. There would be at least 1 piece of weird equipment somewhere. I've neither seen nor heard of any such equipment. On top of that, all that data would be meaningless without access to our databases. Capturing the data or phone calls raw would just give you a mac address or phone number. You wouldn't know who was using those numbers. So you'd have to query our database... a database that changes regularly... new systems come online all of the time. So they'd have to have access from outside of the company, so holes in our firewall, make SOAP requests into our system, Have an active user account, make requests to dozens of different DBs, hundreds of Tables, know how all their joins work, know when system changes go in, and on and on... No such thing could happen without the entire company knowing about it. It's just not possible.
hepting v at&t, trailblazer, turbulence (Score:4, Insightful)
i know that you work some place and thats impressive but there are just mounds of evidence that prove you are wrong. and several other comment threads above with the same idea.
"i work for a telco, there is no way this is possible"
"ok thats great, except for several well known court cases that prove the exact opposite of what you are saying"
(OT) mod DOWN (Score:3)
Why do moronic moderators keep modding up idiotic statements such as these from people who think absence of evidence is evidence of absence? Who ARE those guys, here "Charliemopps", that they think they should have known everything? Since when is THEIR cooperation and knowledge necessary? Even if they knew all that was going on everywhere in their OWN company (which not even the CEO does), that company is not connected to others, right?
This comment is STUPID STUPID STUPID.
Sorry, I've had enough after readin
Re: (Score:3)
I'm sorry, but you don't know what you're talking about. If the phone is working off a remote, then yes, it hits a DSLAM, gets converted to digital, is sent up a trunk to the switch then gets down converted back to analog before re-entering the switch. This is not on any network or anything. It's just a point to point data line. The majority of the phone lines coming off the switch however are serviced directly by the switch and never converted to digital. This is because the switch is placed in the most po
Re: (Score:3)
The fact that you're so naive about how modern trunking works and yet you're still commenting is sad. We don't use AT&T (or rarely) They are expensive. Also most phone calls are STILL local. They happen entirely within our own equipment. All long distance calls that go from one of our territories to another, again, all happen within our own equipment... because it's cheaper. Even calls that happen into some of AT&T, SPRINT and others may still happen entirely within our own equipment if the customer
Re:I should be shocked and appalled... (Score:4, Interesting)
Maybe his statement should have instead been "the Bells, Verizon, and TWC capture all and forward it to the government." I've heard rumors of that related to AT&T, but never any confirmation. But to say "all" is simply false. Maybe they keep all they get, but I know for a fact they don't get "all".
Re:I should be shocked and appalled... (Score:5, Interesting)
We didn't capture anything that wasn't ordered by a court order, and even then, only the bare minumum to meet the court order.
You didn't need to capture anything. According to him, the government was doing it for you. (Or rather, for them.)
Re: (Score:2)
the government was doing it for you. (Or rather, for them.)
For all of us.
Re: (Score:2)
If it's NOT true, it's hard to figure out what other possible purpose there could be for that gigantic Utah data center the Feds are building.
It'd also be hard to figure out why the government was so adamant they have the need to wiretap communications without a warrant, since (pre 9/11) they could already put wire taps into place and then file the reasonable cause paperwork anytime within the subsequent two weeks.
Re:I should be shocked and appalled... (Score:4, Interesting)
Re: (Score:3)
Actually, it's been my experience that the folks who work on the physical infrastructure usually are the ones who know where the taps are with the best degree of accuracy. They have to, so that they don't inadvertently screw with the link.
Re: (Score:3)
Er, you folk all asleep or something? "FBI Confirms 'Magic Lantern' Project Exists. And Carnivore gets renamed DCS1000" quote from 2001.
The only argument still going on is whether to let the local traffic department mine the data and Jail the lot of you. Oh and to admit that they collect the data so that the phone conversation can be played in front of the judge before sentencing.
How naive can you people get? its been bloody obvious for years that "Terrorists" have been caught before they did any harm usin
Re:I should be shocked and appalled... (Score:5, Interesting)
In 2007 we were using "A single NarusInsight machine can monitor traffic equal to the maximum capacity (10 Gbit/s) of around 39,000 256k DSL lines or 195,000 56k telephone modems. But, in practical terms, since individual internet connections are not continually filled to capacity, the 10 Gbit/s capacity of one NarusInsight installation enables it to monitor the combined traffic of several million broadband users.". The Wikipedia page doesn't seem to have any real updates since 2007. Of course traffic has increased since then, but I doubt they bother to store streaming video of Justin Bieber from YouTube - which is reputedly 98% of all bandwidth consumption apart from pron.
What was the size of the LHC storage by the way? Oh, that's right, in 2010 it was "About 50PB of tape storage, handled by a set of robotic storage hardware. Still, they've been finding that disk storage is working well, and have scaled that up to 20PB worth of storage." http://arstechnica.com/science/2010/08/lhc-computing-grid-pushes-petabytes-of-data-beats-expectations/ [arstechnica.com]
However the good news is that in 2011 "Our annual data consumption was estimated at 9.57 zettabytes" on the internet. A difference of 21-15=6 orders of magnitude. http://www.engadget.com/2011/04/07/our-annual-data-consumption-estimated-at-9-57-zettabytes-or-9-57/ [engadget.com] So unless NarusInsight can find and throw away a million times more Bieber than your snarky comments on Sub Reddit, "Revolutionary rodents against the government" they don't have that on disk yet, But they could probably record all telephone conversation.
I seem to recall that rumor used to have it that only all calls in and out of the USA were monitored, it would not be at all surprising to find that the capability to monitor all internal calls were available. The only reason it might not be happening is that the transcontinental calls route through a finite set of fiber or satellite links, whereas call data on the internet in the USA could route through a very much higher set of nodes that would need to be monitored to capture the data.
Re:I should be shocked and appalled... (Score:5, Informative)
> I seem to recall that rumor used to have it that only all calls in and out of the USA were monitored,
That looks like the NSA's legal requirements to monitor only foreign communications. They were prohibited from monitoring domestic communications, that was the responsibility of the FBI. Unfortunately, "Homeland Security" was created in the wake of 9/11 specifically to merge and organize data among the various intelligence services, and part of the result is that you can't effectively prosecute one agency for overstepping its bounds by going to the other agencies. They can all rely on Homeland Security to cover for them with "Patriot Act" court free search orders, or groundless "national security" orders that prevent even disclosing that your clients have been monitored.
Homeland Security is an extremely dangerous concentration of monitoring and investigation power. I sincerely hope that the antipathy of the more specialized intelligence agencies continues to hinder their growth.
Re:I should be shocked and appalled... (Score:4, Insightful)
We can't 'know' this is false, but . . . we can look at what the implications would be if this were true.
This would require vast storage, incredible database crossreferencing, would imply certain kinds of information be available not only without warrants, but without ever needing to pull the original data. Not only would warrants be redundant, so would National Security Letters.
All without a single patriot in the government going public and blowing the lid off this, yet simultaneously putting this information in the hands of someone willing to shoot their mouth off on CNN.
Can, in theory, all this be true? Sure. It could happen. *Practically* can all this be true? No - too many conspirators have to work invisibly, never tipping their hands, never making a mistake. Just don't buy it.
Pug
Re:I should be shocked and appalled... (Score:4, Interesting)
No. Data taken from warrants and NSLs can be used in court and the FBI can admit they have it and not worry about giving away their capabilities by acting as if they have it. Data taken in a dragnet like this could only be used secretly.
Except that it has been revealed. People just seem to keep forgetting, like they forget the Tuskeegee experiment, like they forget the Gulf of Tonkin "incident", or various other nasty things the government has done.
Re:And to think they told meg... (Score:5, Interesting)
Ahhh if only the government would be so kind as to freely back up all the classic usenet celebrity fakes of Gillian Anderson and provide them free of charge on the open internet as a public service. This world would be on the right path indeed.
Re:Logistically impractical (Score:5, Interesting)
Think of the sheer amount of storage, electricity, infrastructure, personal, computing resources and so on that you would need in order to perform this feat. The numbers would be boggling and would account for a significant portion of the worldwide sales of all hard drives, tape back ups etc, etc.
Well, the internet was clocking about 21 exabytes per month in 2010. However, the overwhelming majority of that traffic is redundant; if you remove the redundancy in the data set and then compress it, you're probably looking at less than an exabyte of data over the public internet. You can reduce that further with whitelists; Traffic from Netflix, for example, is probably not going to contain super secret terrorist communications.
So let's say you can cut that down to only record the most relevant 5%. That's about 1 exabyte. How much [storagemojo.com] would that cost? Well, in 2008, they guesstimated this to be about $400 million. A single stealth bomber costs about $2.1 billion [yahoo.com]; So the yearly storage costs of "the internet" is about 2 stealth bombers. -_-
So at least as far as the data storage is concerned, I think it's well within the government's budget. Now, making that data usable and analysis of it... hooo boy... that's gonna be the bitch of it. But storage? Solved.
Re:the NSA has a shadow market of IT work (Score:4, Interesting)
if you read James Bamford's books, you will begin to realize that most of the major US computer companies, from Cray to IBM, were propped up directly and secretly by the NSA to build supercomputers for it, secretly, years before the technology would reach the public.
I don't need a book to know the government funds technology improvements; They freely admit [darpa.mil] it. It's not exactly super spy secret stuff -- they created the internet. It's a safe bet that they continue to work on similar things.
Re:Logistically impractical (Score:5, Insightful)
Seriously, you mean a data center like this can't handle the traffic?
http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/ [wired.com]
and the 5 million people (as of 2011) with security clearances aren't enough?
http://www.wired.com/threatlevel/2012/07/security-clearances-increasing/ [wired.com]
and the NSA recruiting at Defcon and math colleges all around the country isn't happening?
http://www.federalnewsradio.com/411/2890348/NSA-hiring-reforms-serve-as-model-for-government [federalnewsradio.com]
These guys have cash and are all of their activities are shielded under FISA and the National Security Act and State Secrets Privilege.
http://www.aclu.org/national-security/fix-fisa-end-warrantless-wiretapping [aclu.org]
It's happening, it is a reality, and it is more than possible. Even with an inside whistle blower, the courts will not limit the power of the government to spy on us.
https://en.wikipedia.org/wiki/Room_641A [wikipedia.org]
The only thing we really have going for us is the Catch-22 on the use of the data. If it is every used in a trial, chain of custody and 4th amendment issues likethe exclusionary rule will suppress the evidence since it was obtained without a warrant. The only thing that stands in the way of the NSA and fully implementing 1984 is the 4th amendment.
Re:Logistically impractical (Score:4, Informative)
From what I've read, the legal argument against this being an illegal search is that the entire dataset isn't searched, it is stored. They store the communications. When they want access to the data on a particular person they get a search warrant to access the stored data. I don't agree with that, but that seems to be the theory.
Here is [nytimes.com] a short video on an NSA whistleblower about the Utah datacenter and the types of things they can do with that much data.