ISPs Using "Deep Packet Inspection" On 100,000 Users

dstates writes "The Washington Post is reporting that some Internet Service Providers (ISP) have been using deep-packet inspection to spy on the communications of more than 100,000 US customers. Deep packet inspection allows the ISP to read the content of communications including every Web page visited, every e-mail sent and every search entered, in short every click and keystroke that comes down the line. The companies involved assert that customers' privacy is protected because no personally identifying details are released, but they make money from advertisers who use the information to target their online pitches. Deep packet inspection is a significant expansion over tools like cookies in the ability to track a user. Critics liken it to a phone company listening in on conversations."

So what's the status on IPSec?

DNSSec and opportunistic IPSec should put an end to the snooping and throttling once and for all.

Re:So what's the status on IPSec?

In response to another article, I said that we should start encrypting all of our traffic and asked for programmers to start adding that functionality and making it the default so that even unsophisticated users' trafic would be encrypted.

But with the revelation the other day that the Bush administration believes the Fourth Amendment (right to privacy and protection from searches without cause), this becomes just another good reason to get cracking with all traffic encrypted.

http://yro.slashdot.org/article.pl?sid=08/04/03/1219200 [slashdot.org]

People already do

its called tor [torproject.org].

Encrypt everything.

Thats it, I say webservers move to SSL only transactions. All other plaintext transmissions should get encrypted at the endpoints transparently. Then when the government whines about not being able to find the terrorists they can blame datamining companies that paid for their election campaign. Then they can make a law that forces a back-door, which would create a need for some nifty-ass steganography [wikipedia.org] which would lead to massively excessive processor and network overhead (encryption and steganography respectively) for the most basic of transactions which would lead to NSA funded algorythms to find these hidden messages which would. . .holy shit it's almost 10AM, I need to hit the sack.

Re:Encrypt everything.

Thats it, I say webservers move to SSL only transactions.

I agree completely, but keep in mind that even with encryption, ISPs can still collect quite enough information on us to put together a truly impressive profile. Sure, they won't know exactly what you read, but if you visit Erowid, I'd call it a good bet you don't want recommendations on a cheese to go with dinner.

For targetted advertising purposes, the simple "where" counts for 90% of the "what".

Re:Encrypt everything.

The problem is that SSL happens before any HTTP does, and SSL is a general mechanism that can be used for any kind of TCP connection.

How does the webserver know what to give you when foo.com and bar.com map to the same IP address, and the browser requests something like index.html that exists on both? This works only because when the browser makes the request it also tells the webserver which domain it was trying to access. The browser sends something like this:

GET /index.html HTTP/1.1
Host: foo.com

Now, this breaks for SSL, because SSL happens before the connection is established, so there's no way to decide which certificate to use based on the domain.

To fix to this is adding the support directly to SSL. rfc4336 contains a mechanism to do this with TLS.

Re:Encrypt everything.

Wrong RFC. That would be RFC4366 [rfc.net],

Re:Encrypt everything.

It's beyond me why this hasn't happened already.

As far as I know, IIS and Apache don't quite support TLS yet (although it's in-progress) which means every SSL-enabled website would have to be on it's own unique IP/port...making the IP 'crunch' even more of an issue.

time for some hactivism

Let's start turning over rocks in the private lives of telcom CEO's and see what scurries out. I'm sure they won't mind, it's in the interests of an open society and free debate, don'cha know.

Re:Btw. is your ISP Knology?

1. Find his adress 2. Intercept his snailmail (which later is returned). 3. Scan it and post it to our small group of Slashdotters. 4. Ask him if he thinks that this is a violation of his privacy? 5. ?? 6. Profit!

7. Go directly to Federal-pound-me-in-the-ass-prison for postal fraud. Do not pass go, do not collect $200.

Seriously, if the USPS, UPS or Fedex started doing this can you imagine the outrage? Yet somehow it's ok to do it with electronic communications? WTF?

Re:Btw. is your ISP Knology?

Fedex and UPS DO do this.

Fedex and UPS open your packages to look at what you are shipping so they can sell that data to advertisers?

rather they're searching through it looking for things that look suspicious

Did you even bother to RTFA? Wait, dumb question around here. This has nothing to do with looking for 'suspicious activity'. The ISPs in question are allowing third-party companies to build profiles of their users by spying on their traffic in order to do targeted advertising.

Good luck with that

Never mind that it's evil, or that it's a great step to losing their common-carrier status.

Never mind that it's a true violation of privacy.

Never mind that I block cookies pretty well and I run with NoScript most of the time and I don't see very many ads, and besides, half of the time I'm inside my employer's VPN.

But even more than that, I have seven other users in my household, half of them teenagers. If they want to sniff all of my NAT-ed packets coming out, they're going to discover that I'm a geek who has four Facebook sites, likes art and hates it, plays Runescape incessantly (the 10-year-old), likes the Wiggles, and works as a beauty consultant. So go ahead and hand me the ad for the latest XBox game (I hate games). Offer my kids server hardware, and see if you can get my wife to click on fun games to play with the Backyardigans. Oh, wait, you already do. It's called "not targeting advertising", and it's free.

So what we have is a thoroughly broken high-cost borderline-illegal absolutely-unethical service offered to advertisers in a difficult economic period. By people who we all hate a lot, and who will rapidly become targets for everything from blocking to legislative action to you name it.

I knew there would be some kind of career move for spam kings in the future. I just thought it would pay better.

I predict a less than stellar outcome for these idiots, and they deserve every painful moment.

Re:Good luck with that

However, you still get more accurate data on user trends as a whole - you no longer have the old problem of the fact that only the sort of people who fill in surveys will fill in your surveys, and they're not generally a representative sample.

Any data at all on user trends more than their competitors will help advertising companies make money.

Re:Good luck with that

Never mind that it's evil, or that it's a great step to losing their common-carrier status.

They don't have a common-carrier status to lose.

Re:Good luck with that

> If they want to sniff all of my NAT-ed packets coming out, they're
>going to discover that I'm a geek who has four Facebook sites, likes
> art and hates it, plays....

Silly person, they are much smarter than that. Each of those PCs can be identified, see previous slashdot articles on the subject. Especially since each PC in a network serving a diverse family as you are describing will probably have obvious differences in OS and browser versions. Then there is detailed packet header inspection (DEEP INSPECTION, remember?) to seperate out OS subtle version differences, etc. And each PC/account will offerup different cookies to the same websites like Google.

NAT won't stop them. SSL won't stop them. Laws might. This sort of snooping isn't 'like' listening in on phone conversations. It IS listening in on conversations.

ssh tunnelling + squid

I pay for a dedicated server (essentially colo but they provide the hardware) from a company with a decent AUP. I put linux on the server and run squid on a non-standard port, allowing connections from localhost only. Then from the machine I'm surfing from I tunnel into the squid server. Say squid is running on port 1234 and sshd is running on 4567:

ssh -f -N -L 1234:localhost:1234 -p 5678 my.squid.server.com

Configure firefox to use a proxy to localhost:1234 and all traffic is encrypted to the squid server.

Of course, I could just use Tor, which is great, but can be slow. In fact, you could run a tor server on your colo machine and have all tor traffic bounce off of the server, which would be pretty fast if you leave tor running as a daemon and dedicate a decent amount of bandwidth to the tor network.

There should be a law

It's illegal for anyone to open mail not intended for them. The same should be done for electronic communication.

And if I hear one libertarian say we need less laws, I'll puke. It's as if they though they had a magic wand and all the troubles of the world would disappear by removing government. Unfortunately, the world hasn't worked that way since we left the caves 12,000 years ago.

Re:There should be a law

We *do* need fewer laws. However, the ones that remain need to be effective and of value, and actually enforced.

The law to protect your right to privacy already exists, it just needs to be enforced. Creating more laws doesn't help with lack of enforcement of what is already there.

How are they to deliver targeted advertising?

If these are the ISPs (as opposed to the visited web sites) doing the spying, then how are the advertising companies involved supposed to deliver the content? Are they going to use the same "deep packet" method to inject the advertising? If the advertising delivery is away from that deep packet inspection, then how do they identify which user was interested in penis enlargement products vs. which user was interested in replica watches? Or are the ISPs going to lock-in the IP address, now?

Listening in? Um, yeah.

Critics liken it to a phone company listening in on conversations.

Um, my ISP IS my phone company. If they can get away with reading my emails and stuff like this comment, what's to stop them from listening to my phone calls? We're really at a crossroads: either the law makes ISPs common carriers with no interest in, or control over, content like a real phone company, or we lose most of the potential of the communications tech revolution.

Up to 2 years imprisonment

If you do this in the EU. Packet pauyloads are off-limits without court order. You may not even store them.

"Customer revolt"

FTA:

For all its promise, however, the service providers exploring and testing such services have largely kept quiet -- "for fear of customer revolt," according to one executive involved.

Guess what pal..the word is now out.

Ever get the feeling the the Internet just isn't worth it anymore?

Deep Packet Inspection Not For Ads

I have a bit of history with two large service providers in the US. While I have not been involved directly with the deep packet inspection teams, I have had direct contact with all of them and helped them design networks using this technology. The technology was never sold to upper management as a way to track our users and target ads to them. It was never intended to capture a web page hit that was directed at a specific company to see what that consumer was interested in. Instead, it was always meant to monitor users (and more importantly, user aggregates) and determine what kind of traffic they were sending.

It was, and is, always about the network profile. If they find out that 10% of the traffic on the network is VoIP traffic, they want to design the network shift this traffic to have lower latency.** If they find out that 50% of the traffic is BitTorrent, they may put rules in place around such services. In my opinion, the service providers that I have dealt with do not have the technology in place to target down to the user. Also, they do not appear to be developing this technology.

**Some can argue that providers are instinctively evil and want to destroy this traffic, but I'm not going to fight this here.

What's the difference

The difference is that in the first case, the data passes through a dumb machine that compresses, caches, etc. The result is cached like it is expected (RFC 2616 is pretty clear about that), even though it is done transparently. No need to keep logs about who downloaded what.

In this case, the data is explicitly mined, by a company interested in building a profile of each user. It doesn't say it is limited to web traffic only, only that "Nor does NebuAd record a user's visits to pornography or gaming sites or a user's interests in sensitive subjects -- such as bankruptcy or a medical condition such as AIDS.", which I doubt both on technical grounds and because it is a market and someone will want to take advantage and "The company said it processes but does not look into packets of information that include e-mail or pictures." which I think is in contradiction with other parts of the article and even if they didn't, it's a matter of time before they do.

Basically, it's the intent that counts. The ISP can intercept everything they want because they're in the middle. When they start doing so for reasons that are not part of maintaining the communications as specified (like forwarding, maybe firewalling and proxying depending on the conditions), alarms should go off.

Re:Slashbot hypocrisy once again

So, it's bad and evil and wrong if a computer at your ISP reads all your packets for marketing research purposes, but when Slashdot's favourite pet company Google does the exact same thing with all your messages in Gmail, it's perfectly fine and justified?

Yes. You may use gmail or not, and if you do then you agree that they will use your email contents for advertisement.

No one authorized ISPs to inspect packets for any purpose.

However if they provided their service at the same price google offers gmail in exchange for authorization to inspect packets, I'm sure there would be lots of people willing to take the deal.

I think Slashbots need to get their kneejerks straight.

And I think whoever modded you insightful was on crack.