An Education In Deep Packet Inspection 126
Deep Packet Inspection, or DPI, is at the heart of the debate over Network Neutrality — this relatively new technology threatens to upset the balance of power among consumers, ISPs, and information suppliers. An anonymous reader notes that the Canadian Privacy Commissioner has published a Web site, for Canadians and others, to educate about DPI technology. Online are a number of essays from different interested parties, ranging from DPI company officers to Internet law specialists to security professionals. The articles are open for comments. Here is the CBC's report on the launch.
Deep inspection up your authorities (Score:5, Funny)
Re: (Score:3, Funny)
If there's a Slashdot achievement for getting a +5 on a Goatse link, you just missed your chance at it.
Re: (Score:2, Informative)
It wouldn't be the first time [slashdot.org] somebody got +5 for linking to goatse [goatse.cz]m im just waiting for the editors to let it slip into a summary.
Re: (Score:3, Insightful)
Re: (Score:2)
Well, I already run an OpenVPN tunnel to a ...ehem... unrelated server of mine, registered in some offshore island, by some company that is... ehem... not mine...
Not that expensive, and that server keeps no logs of my activity at all, and runs everything you can imagine. SElinux, IDN, DPI-firewall, honeypot, remote security audits and so on. Of course bribing the government keeps foreign inquirers off my back. They have a vivid business down there, based on being able to trust that they don't give foreign q
Re: (Score:2)
Oh, and, yeah. My e-mail address and everything else go trough that pipe too. Everything's fake. So in case you thought you are a wise crack for pointing out that I did not post anonymously: I simply don't need to. :)
Re: (Score:2)
Re: (Score:1)
It wouldn't be the first time somebody got +5 for linking to goatsem im just waiting for the editors to let it slip into a summary.
If the AP believes so strongly that aggregation steals their content instead of drives more visitors to them, they may yet resort to using Referrer headers to make off-site links to their stories redirect to goatse.
Re:Deep inspection up your authorities (Score:5, Insightful)
How would the authorities like to be deep inspected?
That's a good question.
This summary mentions education about deep packet inspection. To me that's a very simple thing that boils down to a few questions:
Do you want your ISP and potential unknown/unaccountable parties to be able to easily monitor, intercept, and record some or all of your Internet traffic? Do you want profiles built on this information that will compromise your privacy and could be used to serve advertisements or to micromanage your Internet usage? Do you feel like QoS, which will be the given reason/excuse, is such a good and desirable thing that it's worth all of these disadvantages?
Like so many things that are not the result of overwhelming customer demand, this is a bad idea that is open to all sorts of abuse.
Re:Deep inspection up your authorities (Score:4, Insightful)
Good luck deep inspecting that crap
Re:Deep inspection up your authorities (Score:5, Insightful)
it's just going to push more and more protocols to use TLS wrappers and to use random "legit looking" ports (like 20, 21, 80, 443, 110), a la Skype and most IM clients nowadays Good luck deep inspecting that crap
That's true. You'd think that "spam vs anti-spam measures" alone or "windows viruses vs windows virus scanners" alone would have taught us, by now, how to recognize an arms race when we're about to start one. This is what I mean when I say that our culture does not value foresight.
Re: (Score:3, Funny)
Our culture doesn't value foreskin either (aside from grinding it up for use in cosmetics).
Such a thought is sure to put any intact man in your position, causality.
Re: (Score:1)
My neighbor got burns on his scalp and had that done. The funny thing is that his name was already Mr. Forehead.
Re: (Score:1)
Re: (Score:2)
Apparently it's going to push more people to use fixed width fonts too.
Re:Deep inspection up your authorities (Score:5, Interesting)
Do you want your ISP and potential unknown/unaccountable parties to be able to easily monitor, intercept, and record some or all of your Internet traffic? Do you want profiles built on this information that will compromise your privacy and could be used to serve advertisements or to micromanage your Internet usage? Do you feel like QoS, which will be the given reason/excuse, is such a good and desirable thing that it's worth all of these disadvantages?
The Internet will become more like an airport: all your packetages will be subject to inspection without need for a warrant or probable cause and denied travel accordingly.
Re:Deep inspection up your authorities (Score:5, Funny)
Re: (Score:2, Redundant)
Re: (Score:2, Offtopic)
Re: (Score:1)
And I am in this thread too!
Re: (Score:2, Funny)
Re: (Score:2, Insightful)
HAHA... this reminds me of the circa 1997/98 near-bust (or was it an actual bust?) of a famous sports player who got caught up in a Mountain View Police raid on a "massage parlor". His plea to the cops to not be cited or charged was that he wasn't there having sex; he was getting "deep tissue therapy"....
In hind site, umm, hind SIGHT, ummm, hell, RETROspect, this may have been a form of "deep PACK IT" inspection. If things were non-condomnable, it might have ended up as a 32-bit insemination, vice inscripti
The description's a little "excited" (Score:5, Insightful)
It's a hacky technology to implement QOS because folks don't like setting the QOS bits and protocol in the headers. Usually because some Microsoft firewall only allows http on port 80 (;-))
It's the use of it by the famous "men of good will but little understanding" that is bad, plus of course the use of it by men of ill will.
--dave
Re:The description's a little "excited" (Score:5, Insightful)
The former category is much more dangerous. At least most people recognize ill-will when they see it. By far people with good intentions and no comprehension of the "law of unintended consequences" do more damage to the world than do people with openly evil intentions.
No politician ever increased state power by saying "I'd like to see this nation become a totalitarian state and you should support me because this law will bring it closer to that goal." They do it by saying "this is for your safety" or "this is to stop terrorism" and the people who mean well and don't understand the damage they can do will eagerly eat that shit up. That's true whether or not the politician himself believes anything he is saying.
Re: (Score:3, Interesting)
Well, that brings up an interesting philosophical question --
Re: (Score:2, Insightful)
Meaning well does not remove you from the causation of harm.
Example:
Just because I don't understand that shooting someone with a gun can kill them doesn't mean I didn't cause that person to die. It just means I am an ignorant fool who killed someone.
Then again if you follow the train back far enough we can just blame *insert how you think the world came about here* for all evil.
Re:The description's a little "excited" (Score:5, Insightful)
That's why it makes more sense to look at it in terms of enablers who could have chosen differently. The people could study statecraft and propaganda techniques. They could study dictatorships like the Third Reich or Italy under Mussolini to learn how these leaders came to power by preying on the desperation and the weaknesses of the people. They can familiarize themselves with the sorts of excuses and justifications that are given for the expansion of state power. They can learn argumentation and research so that they are equipped to investigate things on their own instead of requiring that premade conclusions be spoonfed to them. In short, they can shed the naivete and the ignorance that must be present before such horrors can arise.
Any literate adult with Internet access can do all of these things. The only obstacle they could encounter would be their own laziness or unwillingness. I would say that we have a responsibility to do these things because everything that is good about the way of life that we presently enjoy depends on an informed citizenry. Our civilization is on the decline because people think this does not apply to them, or they think that someone else will take care of it, or they think that the latest celebrity-worship is more important.
The evil politicians are like organisms in an environment. The environment in which they thrive consists of ignorant people who are far too naive and trusting and do not guard themselves against being deceived. If you set up this sort of environment, those organisms will appear in it and will prosper. Thus, I believe it is the people and their ignorance and lack of priorities that are far more to blame, for they provide fertile soil without which this organism could never succeed. It should be assumed that evil men will come along who will try to take advantage of our way of life to suit their selfish purposes. We should be prepared for this and well-able to deal with it by never rewarding it with the power it seeks to have. We are not. We think our enemies are our friends because they know how to tell us what we want to hear. That is the problem.
Re: (Score:1, Insightful)
You, my friend, do not understand human nature.
Re:The description's a little "excited" (Score:5, Interesting)
As I understand it:
ISPs don't implement the QoS (Type of Service) field because (back before THEY needed it for services) Microsoft deployed an IP stack in Windows that "improved" their own file transfers and other IP traffic by demanding high QoS for everything.
Because of that (and the threat of bad guys cheating) the ISPs don't trust the field when coming from a customer. So there wasn't a strong driver for implementing QoS in the ISPs and backbone
IMHO the right solution is for ISPs to:
- Write service level agreements that guarantee a certain bandwidth of high QoS traffic - for the whole feed to the customer, not per flow.
- Start honoring the ToS field and policing the data rate at the edge router, and
- When a packet would be dropped for exceeding the data rate for the enhanced service, instead REWRITE THE ToS FIELD for best-effort delivery (or whatever lower service level seems appropriate) and try to forward it under those terms.
That way:
- The ISP doesn't have to classify the flow according to traffic type to give the user high QoS for his critical services.
- The ISP doesn't have to do a packet-recombine if the packet is fragmented to identify the flow for the trailing fragments (which don't carry the TCP/UDP port number).
- The user / application can specify what special handling he / it wants.
- Applications that try to "cheat" can only do so up to the bandwidth cap for the special handling. (But the user paid for that. So he can use his bandwidth for whatever he wants. It's not "cheating" any more.)
- Excess traffic will still go through as well as it does now.
- A "cheating" application WILL hurt the user's own really-needs-high-QoS service, giving users and applications providers an incentive not to request excessive QoS. (But it won't hurt ANYBODY ELSE's traffic.)
- Authors of applications that need high QoS will have an incentive to specify it, since doing so will work.
Easy Fix (Score:4, Interesting)
Re: (Score:3, Interesting)
Charge more for higher QoS. Give a discount for lower QoS.
That's a given.
I'd expect plans to include some small amount of VoIP quality 2-way high-QoS as standard (about a couple phone calls' worth plus whatever is needed for the plan's special services). Higher amounts could be obtained by subscribing to a higher-priced plan or dynamically-configured as needed - perhaps for a fee (like dialing a toll phone call or subscribing to a pay-per-view).
Want your packets to get reserved bandwidth and better treatme
Re: (Score:2)
Re: (Score:1)
Re: (Score:1)
Let me get this straight.
You propose that ISP's should count packets tied to a subscriber account in such a real-time fashion as to put this logic to use in rewriting the packet headers in-transit?
Once you're done actually building a system even capable of that, how do you keep such a bad-ass future network from becoming Skynet?
Real-time accounting is hard enough--having the networking devices actually make decisions about header mangling based on that accounting on a packet-by-packet basis is not something
Re: (Score:1)
Re: (Score:2)
Sorry, it ain't gonna work. You're still trusting the end-user to set the QoS properly. That's just not going to happen. Even someone with good intentions who wouldn't otherwise try to subvert the system is going to get suckered into using a "web accelerator" that munges the QoS. And then he's going to complain about crappy VoIP quality, because he doesn't know he used up all his high-quality allocation downloading a torrent of the Star Wars Holiday Special.
Any system which relies on the cooperation of th
21st Century Government Work (Score:5, Informative)
Taking a quick look through the content at the government site, I must say I'm surprised. CC licensed content, links to external resources, a collection of international points of view. I'd be truly impressed if they'd managed to get all these folks in a room together.
Regardless, kudos to Canada for hitting the 21st century.
And I was doubly impressed to notice the absence of web beacons / analytics scripts.
Re:21st Century Government Work (Score:5, Informative)
Re:21st Century Government Work (Score:4, Funny)
It's also snowing.
Re: (Score:2, Funny)
We've recently been blamed by the US as the major source of film piracy.
I thought that was China...
no wait.. its Russia...
Can they ever get their facts straight?
Re:21st Century Government Work (Score:5, Funny)
Can they ever get their facts straight?
What are you talking about? Everyone knows it's the terrorists. It's always been the terrorists. We will fight them with our allies: Canada, China and Russia.
Re: (Score:2)
I defy you to find a reference to it anywhere in history [archive.org]!
And don't give me that "here is some paper that proves it" crap, everyone knows that paper can be counterfeited. Only the internet [wikipedia.org] can be believed.
Re: (Score:1)
This kind of open and honest debate is what makes me proud to be Canadian. The Office of the Privacy Commissioner of Canada seems to 'get' the technology of DPI and why we should be concerned about it.
On a side note I'm not so proud of the dirty oil being produced here in Alberta. But, at least no one is being killed for our oil. - well except for migratory birds [worldwatch.org].
I'd be more impressed ... (Score:2)
... if they'd managed to build a web site that displayed correctly (or displayed the essay collection AT ALL) on Firefox 2.0.0.8.
obligatory (Score:5, Funny)
Re:obligatory (Score:5, Funny)
I did not know you could do that with a kielbasa, you dirty, dirty young man.
Re: (Score:2, Funny)
Oh come the fuck on, what it is with retards with mod points?
That was a troll?
No, not in the slightest. A troll would be me suggesting that whomever moderated the above as troll go fondle themselves with a razor blade while watching their mother sate the insane raging lust of a Brahma bull.
the above was a joke...*sigh*
Re: (Score:2)
Re: (Score:1)
Deep Inspection is not the Problem (Score:4, Insightful)
Re:Deep Inspection is not the Problem (Score:4, Insightful)
It's a technology that almost no one wants except for those who are in a position to abuse it. That makes it difficult or impossible to view it as a "neutral" thing.
Re:Deep Inspection is not the Problem (Score:5, Insightful)
It's a technology that almost no one wants except for those who are in a position to abuse it. That makes it difficult or impossible to view it as a "neutral" thing.
I've seen quite a few "good" uses of DPI, from filtering out content trying to contact worm control channels to gathering statistics on Web site usage for academia. You can use DPI to slow down traffic going to any video hosting site not paying you a kickback or you can use it to filter out a DDoS attack on one of your network's clients. The technology is useful today, but we do need legislation to keep it from being abused.
Re: (Score:1)
It's a technology that almost no one wants except for those who are in a position to abuse it. That makes it difficult or impossible to view it as a "neutral" thing.
What about this? http://dpi.priv.gc.ca/index.php/what-is-deep-packet-inspection/ [priv.gc.ca]
DPI has been used for several years to maintain the integrity and security of networks, searching for signs of protocol non-compliance, viruses, malicious code, SPAM and other threats.
Are you suggesting people don't want a less SPAMy, more secure internet? There's more to it than "oh noes, the isp's are spying my internets!"
I'm not saying I want them to, there's just more to it than some people realize.
Re: (Score:2)
It's a technology that almost no one wants except for those who are in a position to abuse it. That makes it difficult or impossible to view it as a "neutral" thing.
What about this? http://dpi.priv.gc.ca/index.php/what-is-deep-packet-inspection/ [priv.gc.ca]
DPI has been used for several years to maintain the integrity and security of networks, searching for signs of protocol non-compliance, viruses, malicious code, SPAM and other threats.
Are you suggesting people don't want a less SPAMy, more secure internet? There's more to it than "oh noes, the isp's are spying my internets!" I'm not saying I want them to, there's just more to it than some people realize.
You and another person suggested using it to thwart spam or worm attacks. I am replying to you since the other person was more reasonable. That is, he did not say "are you suggesting people don't want a less spamy [sic], more secure internet" as though that's the same thing as criticising another wrong solution that cannot solve our problems. The way you did that reminds me of people who say "you mean you don't want to be safe from terrorists?" when you point out that it's wrong to infringe on civil libe
Re: (Score:2)
It's a technology that everyone except abusers should love, if used well. Think of it this way, when congestion happens, something must be dropped. So, what do you drop? Do you drop random packets? Or do you identify someting that's drop-tolerant and delay-tolerant and drop those first (up to some point where you'll be dropping more)? Personally, I'd think that people would be happy that their VoIP is priori
Re: (Score:1, Insightful)
"Do you drop random packets? Or do you identify someting that's drop-tolerant and delay-tolerant and drop those first?"
There's no need to look into the packet to tell what the priority should be. Check the header, and see what priority the user gave it, but limit the amount of "high priority" traffic per user if there is congestion. The user doesn't gain anything by "cheating" and labeling their bittorrent or FTP as high priority - all they'll do is hurt their VoIP or streaming video.
The TCP headers alrea
Re: (Score:2)
So you'll happily let my malware through at high priority, simply because it puts a high priority in its TCP header?
Cheers dude.
Re: (Score:2)
It's a technology that everyone except abusers should love, if used well.
Of course. If you don't have anything to hide, you have nothing to fear.
Re: (Score:2, Insightful)
Re: (Score:1)
The post office reading every letter mailed is the modern day equivalent to DPI.
How in the bloody fuck can you say the post office reading every delivered letter on any terms is good at ANY time for ANY reason?
DPI is the same, there is no legit usage for this tech NONE AT ALL.
Deep Panty Inspection (Score:5, Funny)
I think you can find that elsewhere... (Score:2)
Deep Panty Inspection
Performed for (and/or on) you by the Female Body Inspectors in cooperation with the Clitoris Investigation Agency.
(note: I didn't say pus*y or any other naughty word)
Encryption stops this correct? (Score:5, Insightful)
Doesn't a good encryption system stop DPI from giving any useful information?
Re:Encryption stops this correct? (Score:5, Informative)
Re:Encryption stops this correct? (Score:5, Interesting)
They can however arbitrarily assume all encrypted data to be hostile and filter accordingly...
Not really (Score:2)
Re: (Score:1)
Oh, don't worry, customer! You can still bank securely using our "Transparent Security Service" man-in-the-middle atta-er, I mean, proxy server! Just click OK to that certificate there. It's safe, I promise.
Re: (Score:2)
sure you can do online banking, because your bank has paid the filtering company to be on the whitelist...
Re: (Score:2)
The answer to so many questions is encryption, "they" would love a way to make it painful to use.
Re:Encryption stops this correct? (Score:5, Insightful)
Doesn't a good encryption system stop DPI from giving any useful information?
Any useful information? Sure! There is lots of useful information that can be gleaned even when encryption is used. Who are you communicating with? What protocol are you using? By looking at packet timing and packet sizes, much more information can be obtained than you might think, such as: are you web surfing vs. interactive keyboard login? Are you tranferring large files or reading short web pages? And if the structure of the web pages of the target site is known, the size of the packets transferred might even reveal which pages you were visiting. Some have even reported the ability to make educated guesses about keystrokes in interactive sessions based on timing of packets. Admittedly some of these features will have to wait for the next generation of DPI technology, but even today, a great deal of information can be collected.
Re: (Score:2)
Some have even reported the ability to make educated guesses about keystrokes in interactive sessions based on timing of packets.
So that's how the Comcast employee was able to beat me at CS -- he knew my bunny-hopping pattern!
Re: (Score:1, Insightful)
Yes, analyzing packet sizes and frequencies can work in theory... until it is put into practice, because then it would be trivial for the encryption users to rewrite their servers and clients to send random-sized encrypted packets at random intervals and mess up any information you may have gained.
Re: (Score:2)
Why would that mess anything up? The whole point, usually, is to throttle people using P2P software like BitTorrent. All your suggestion would do would be to put a big neon arrow over their heads.
Re: (Score:2, Insightful)
Re:Encryption stops this correct? (Score:4, Insightful)
I was going to say that wont work very well because of VoIP but as most ISPs are phone companies they probably dont want VoIP working too well either.
Re:Encryption stops this correct? (Score:4, Interesting)
Take a look at:
SSLIA [netronome.com]
Deep packet inspection inside SSL sessions. It's not the only one either.
Re: (Score:1, Interesting)
Uh, no, not with some of the new devices comeing along for firewalls. I know my company is preparing to upgrade their firewalling (global presence, 100K+ employees) with such technology explicitly for 2 "big" problem areas as they see it: 1. Intellectual property theft; 2. "Inappropriate" site access (porn, etc.).
As I read the proposal (as part of a wide list of reviewers as potential stakeholders due to my 2-bit role in web administation) I was appalled to realize that this is commercial off-the-shelf te
Re:Encryption stops this correct? (Score:4, Informative)
Mostly. However, many DPI boxes are now including heuristics as well. Encrypted stream of 20k in and out to a single IP? Sounds like VoIP to me, toss it in that bucket. Encrypted 10k out and 1M in to a single IP? Sounds like a file download, toss it in that bucket. Encrypted 10k in from 20 hosts and 5k out to 15 hosts? Sounds like P2P, toss it in that bucket. If I can make such guesses easily, someone smarter than me has already coded those in and knows what your encrypted data is. Not what it contains, as that's never the goal of DPI, but what you are using it for. So encrypt it all you want. They'll know what you are up to anyway. Unless you do someting like Tor downloads and then it'll look more like P2P and you'll get even worse performance. Or, as someone else mentioned, if it isn't easily identifiable encryption, then they treat it like the least desired traffic. Hide all you want and they can still get you for it.
Your Action, My Reaction (Score:5, Interesting)
I go for encryption, SSL, and HTTPS. Even my slowest home system can easily handle this.
Re: (Score:2)
Re: (Score:1)
Arms race of course as various and ever changing schemes are used to make the encryption appear to be something else.
Re: (Score:1)
Tell them that they are welcome to pay more for an "enterprise class" connection, at which point I disable the check.
Re: (Score:2)
Re: (Score:3)
I go for encryption, SSL, and HTTPS.
Then how do you log in on slashdot?
It takes two paranoid people to use encryption. We really need everyone to be paranoid, which is a hard sell.
Even my slowest home system can easily handle this.
Can the server you're slashdotting handle one thousand (or million) times what your home PC can handle?
Sometimes, when I'm transferring stuff around at home, ssh (sshfs) or kcryptd is the bottleneck, not the disk/wire.
Damn, that makes me sad :(
Re: (Score:3, Interesting)
But there has to be a better way for establishing the 'CA - domain' trust. Why isn't the trust chain 'ICANN CA - country domain operator CA - registrar CA - domain'?
But first you need DNSSec anyway, otherwise you can validate the PKI chain, but not that everybody is who they say they are. (For example: Registrar CA's should only be valid on DNS records where they are listed as the Registrar.)
After that, de
Question... (Score:1)
Re: (Score:2)
> If an ISP decides to inspect all traffic, doesn't this make them responsible for the
> traffic?
No.
Re: (Score:1)
Well, thanks for clearing that right up...
No Tales from the Encrypt (Score:3, Interesting)
Unencrypted data will always get you in trouble. There is no reason in the year two thousand and nine to send or receive anything over the internet without encapsulating it in a SSH or SSL tunnel. Whine all you like about performance hits, but if the technology has reached the point where your residential ISP can look inside every packet you send to see what's there - in real time - then the point has come to spend some processing power on protecting your data in mid-flight, or invest in some encryption hardware.
I'm more than half convinced that this is how everything =inside= a LAN should communicate with each other, too. The firewall should allow port 22, port 443, and drop the rest.
While we're at it, everything should be firewalled right at the VLAN, on the switch.
Re: (Score:3, Funny)
The obvious solution is to block or severely slow down all encrypted traffic (that is, all traffic the ISP can’t interpret). This would have the obvious effect on online banking, which could be solved by the ISP’s computers handling it: The SSL tunnel stops at your ISP, which inspects the decrypted packets before handing them to you. You know the ISP isn’t going to do anything bad with the information because they told you so (in specific, there’s both a contract and fraud laws stopp
Re: (Score:1)
Actually it should work like this
1) Measure the bandwidth usage of encrypted data per month.
2) If it is over some limit, throttle the speed.
That way torrents will work for a while and then slow down. Even a throttled connection should be able to handle online banking. This is only on the cheap service.
I'd also sell a more expensive service with higher limits, static IP addresses and less contention. In fact I'd have a load of options, at the top of which you'd basically be able to max out the connection 24x
Re: (Score:2)
There are currently two "prime time" zones in the day and if you use too much bandwidth during those times you get your service cut in half until midnight.
Sure, they are giving us a free upgrade in speed, but it's totally asynchronous and potentially Phorm riddled.
On the upside, they are reliable (once you have it working), cheap (enough), let you have any port you want all the time and don't throttle BitTorrent.
How is it protected? (Score:2, Insightful)
i guess i'm the one who has to ask (Score:1)
The uneducated .. (Score:2, Insightful)
.. boggle my mind.
Here's what I say to all you paranoid conspiracy freaks ..
go ahead and encrypt your dang traffic. The Internet companies don't really care about the CONTENT of your traffic.
Rather, they want to know what TYPE of traffic you're using - file transfer, web browsing, voice, video.
You think I'm wrong that they don't care about your content. I'm sure you think I'm wrong - because every one of you posting on this thread is f*cking paranoid.
But I can tell you first hand - they don't give a damn.
Yo