Questionable Data Mining Concerns IRC Community 306
jessekeys writes "Two days ago an article on TechCrunch about IRSeeK revealed to the community that a service logs conversations of public IRC channels and put them into a public searchable database.
What is especially shocking for the community is that the logging bots are very hard to identify. They have human-like nicks, connect via anonymous Tor nodes and authenticate as mIRC clients. IRSeeK never asked for permission and violates the privacy terms of networks and users. A lot of chatters were deeply disturbed finding themselves on the search engine in logs which could date back to 2005.
As a result, Freenode, the largest FOSS IRC network in existence, immediately banned all tor connections while the community gathered and set up a public wiki page to share knowledge and news about IRSeeK. The demands are clear: remove all existing logs and stop covert operations in our channels and networks.
Right now, the IRSeeK search is unavailable as there are talks talking place with Freenode Staff."
What's the big deal? (Score:5, Insightful)
Re:IRC is still alive? (Score:5, Insightful)
I think DALnet has done quite well handling abuse. We've switched our infrastructure over to an anycast model that seems to have made us fairly resilient to DOS attacks, and we have made major progress in dealing with drones and abusing bots.
Re:But isn't anonymity a privacy right? (Score:1, Insightful)
Re:What's the big deal? (Score:4, Insightful)
Many of us out there started our technical exploration on IRC. Some people get into computers and then find IRC. Some are the opposite - find IRC and then get into computers. I can credit IRC and the people on there with my entire career choice.
Re:good idea (Score:3, Insightful)
If that question is asked as frequently as you make it seem to be, the person asking it could have found the answer with a websearch. The fact that they didn't search the web tells you that they certainly won't use an irc search engine first either.
Re:IRC is still alive? (Score:5, Insightful)
Min
Re:I've got money... (Score:2, Insightful)
Probably not. I strongly doubt they would put the logs on the web.
Re:What's the big deal? (Score:4, Insightful)
Ensuring the Privacy of Internet Communication (Score:3, Insightful)
The second type of communication is peer-to-peer. A user sends a message to a specific user. Examples include e-mail, phone communication, and the like.
Anyone can ensure the privacy of peer-to-peer communcation. Consider two users who want to exchange e-mail messages. First, the users pick a reliable encryption tool (which are readily available on the Internet) and an encryption key. Then, each user encrypts a message before sending it via e-mail to the other user. Even the NSA will be unable to crack the message (if the users pick a good encryption tool).
Encryption can also be applied to voice communication. The users can use an Internet-phone software application to communicate by voice via the Internet. Each user merely needs to encrypt the data packets before sending them to the other user's computer.
If you believe that someone (e.g., a Russian spy) is wiretapping your regular (mobile or landline) phone, then do voice communication via the Internet. In Russia, most people use cell phones, so they just need to ensure that the phone has a data-communication mode in addition to the regular voice-communication mode. To ensure private communication, the user switches the mode of his phone to data-communication mode and uses his phone as a modem. He plugs the modem into his computer and then runs an Internet-phone software application to communicate via the Internet. The FSB (successor to the KGB) can record the entire session of encrypted Internet packets, but the FSB will be unable to decipher the communication.
"What happens online, stays online. Forever." (Score:5, Insightful)
Sorta (Score:5, Insightful)
Yes, everyone knows about Stalin's brutal mass executions and deportations. Very distasteful business, that. It also created so much resentment that it was unsustainable in the long run.
So it evolved into something more subtle: the idea that somewhere there's a dossier about you, containing a lot of the stupid things you've said in the past. You don't know exactly what or how much. (After all, they were the non-computer kind.) And you don't know when or how it will bite you in the arse later.
Maybe you can kiss any chance of traveling abroad goodbye. Maybe now your chances of promotion or of finding a better paid job, just became nil. Or maybe you're just this far from having to explain it all to the secret police and, if you're lucky, looking forward to a long career somewhere in Siberia. Or maybe it will bite your kid in the arse, if they can't get you. Etc.
In a nutshell, the idea was that you don't have an expectation of privacy. Anything you say, even nodding approvingly when comrade Piotr swears at the government at the pub, might become permanently attached to you and a factor in which way your future goes.
Worse yet, how do you know if comrade Piotr isn't an agent provocateur, trying to get you to say something you'll regret?
So people learned to think twice before opening their mouth, and avoid saying anything that might be used against them. It turned them into a mass of isolated (and thus vulnerable) individuals, because not many risked saying (or even listening to) anything that could have been the start of an organized resistance.
And now back to the topic, here's what I wonder: why the heck do we allow the same in the West, if it's done by corporate PHB's instead of the Communist Party?
The effects, way I see it, can be exactly the same: anything you ever say or do is recorded _somewhere_. Be it Google, or such recorder bots or whatever. And in an age where HR drone routinely google employees and prospective employees, it can come back to bite you in the arse.
And to get even more back on topic: even if you started a private conversation with comrade Piotr, how do you know if he's not just baiting you for something to post on Bash?
Yes, nicks are a privacy tool, but for most people it's not as unbreakable as they think. We already know that most ISPs would give away the owner of an IP address without even asking for a court order. Did you ever register that nick? Because if you did, now the IRC server has information linking that nick to an email address. If you think none can be bullied into giving it away, think twice.
Plus, are you paranoid enough to keep _all_ conversation at the level of "I'm evolvearth, you don't need to know my RL name and telephone number"? Well, kudos if you do, but most people don't. For most, online communication seems to be just an extension of RL communication. (And please don't imagine that said in a condemning tone or anything.)
So basically, all these attempts of recording everything we say or do... will they just turn us into some obedient serfs to our corporate overlords? You know, better not say anything that makes you sound like a maladjusted anarchist, because some HR drone will google you. That might be your job you're throwing away there. Better not say anything against the government too, because you don't know when your (current or future) company gets a chance at a government pork-barrel contract that requires a thorough background check. Etc.
Yes, you can password protect channels, do it all in private channels, etc, but I'd say even that might not help you much once enough people learned to just keep their mouth and fear strangers asking about certain matters.
Just some (admittedly pessimistic) stuff to think about, if you're bored enough
Re:IRC is still alive? (Score:5, Insightful)
Back on topic; I already knew about this, and don't see what the big deal is. I often run into chat logs while googling, sometimes they have useful info. Does anyone really consider a public IRC channel to be a private place?
A lot of the things I've said on
Re:But isn't anonymity a privacy right? (Score:1, Insightful)
Thats so nice about humans, they will forget, over time, what you said. Just imagine a person who stands just behind the (small) group you where directing your remark to recording everything you where saying, and than make it available for everyone to enjoy.
Once or twice, and whereever that person would show his/her nose the conversation would instantly die.
Not because what was being said was important, but because we simply do not like to have our words "re-played" by someone who thinks he can demand an explanation from us (maybe knowing nothing (or worse : not caring) about the circumstances those words where spoken in).
Re:IRC is still alive? (Score:4, Insightful)
I've been using irc since about 1991. Our channel doesn't suffer from spam, bots or abuse.
Re:Wow... (Score:3, Insightful)
Because I am a user, not a master! I get help and I try to help. I don't know most of the things. But I occasionally look into the window if some question is asked that I know. I am there to tell what I know. When I don't know, I stay quite.
Or did you wanted someone to tell you as soon as you asked a question that "hey! I don't know. I feel sorry for your problem and it should not have happened, bla, bla?". See, it is all reasonable to expect this reply. But when you are in a community of thousands (mind you, most of the people are connected to many channels at the same time), it is highly impractical to expect someone to babysit through your "IRC-Xperience!".
I mean, duh!
Re:Ultimatum (Score:3, Insightful)
Some channels (particularly support types) will have use for a search bot.
It seems a bit underhanded how they disguised the bots as a human and used tor to hide the activity. Look at the web: the only search engines that try and disguise themselves and which ignore robots.txt belong to spammers. Legitimate search engines obey robots.txt and are easily identifiable by their user agent. They don't disguise themselves as MSIE.
Will degrade the IRC experiance. (Score:3, Insightful)
Re:Wow... (Score:1, Insightful)
Are you new to the internet? Why are you viewing IRC as somehow different from every other chat service? I always see people connected to instant message services who are not actively chatting, and sometimes not even at the computer. IRC is like an instant message service which also allows group communication on a topic of common interest. People remain connected to it 24/7 so that they can keep track of the conversations their friends are having, and then when they have time to return to the computer or to the chat, they join in on the conversation already in progress. And sometimes this results in periods of time where no one is saying anything.
I frequently see people weirded out by this, but then those same people will go IM one of their friends and say, "Are you there?" without really being weirded out if their friend is not present.
Regarding the article topic, people also expect their conversations to be, for the most part, reported only to the people who are in the channel. It's like having a conversation in a party with 20 people there. You might say something that you're okay saying to those 20 people, but you'd be a little pissed if someone had secret cameras videotaping you at the party without your knowledge and putting it on the internet.
IRC is not automatically public (Score:3, Insightful)
As an IRC user I dislike IRSeek's business model and practices very much. Discussions on IRC channels are by definition available only to the people who join in, and making any log available without asking is bad etiquette and in most places it is against the terms of use. If we wanted to make our discussions public, we would speak in a Web forum or USENET newsgroup, or we would use our own logging facility and post the logs on our webpages.
People who believe IRC is dead or don't appreciate it are obviously not worthy of being called nerds. IRC is alive and well, and it is very interesting and useful. Remember that there are many IRC servers across the globe and many channels in them, just as there are many USENET newsgroups. If one network or channel is touched by the Eternal September, go to another server and at some point you *will* find interesting people.
Re:hmm (Score:5, Insightful)
1) You walk into someone's office at work and find a list of the funniest quotes by you, that they had remembered from previous conversations.
2) You find out that they have been secretly tape recording every conversation you had with everyone at the office.
Re:IRC is still alive? (Score:5, Insightful)
That's the fault of the protocol? I'm sure bittorrent is used for kiddy porn too -- but if I pointed that out in an argument against bittorrent I'd have 50 replies pointing out how it's also used for Linux ISOs, game updates, etc, etc.
Re:Ensuring the Privacy of Internet Communication (Score:3, Insightful)
The problem is, it's not quite like that. In your case, if so, I'd find it hard to believe that you had this user idling there for years without you ever once talking to it or figuring out that it was someone's bot.
Re:Sorta (Score:5, Insightful)
Best summary ever (Score:4, Insightful)
Essentially, yes. You've summarized my concerns better than my verbose roundabout style ever could. Thanks.
My only question was just how much such logging bots, "do no evil" Google, etc, just move us closer to... well, slavery. "Do no evil" Google has brought a lot of good, for example, but also brought us the reality where you _will_ be googled by your potential employer, and might suffer the consequences for some dumb thing you've said in freshman year.
Sometimes the road to hell can be paved with good intentions. Sometimes the government is just one of the possible evils.
1. To start with the most important part: If you're a highly qualified expert -- I fancy myself one too -- you have that option. Most people don't. Most jobs involve interchangeable peons. Noone will lose any sleep over whether they hired someone uber-qualified to operate the cash register, or just the obedient peon who doesn't rock the boat. In fact, in most cases it can be argued that hiring the latter is the _better_ thing to do.
What I'm getting to is:
A) Most people don't have that option to be defiant. So if saying the wrong thing can spell even one extra month of unemployment, they'll rather say what a potential employer wants to hear.
B) A world where only the upper 1% experts can afford to speak their mind, is a world which has lost the battle. A small inteligentsia can be bought, arrested on trumped charges, discredited, whatever. Stalin did that too.
If everyone except you is too afraid to even listen to your crusade, you've already lost. You've just become the liability to a totalitarian regime -- either the totalitarian government kind, or the corporate-owned kind -- and they'll find a way to render you harmless.
2. In an ideal world, every employer would be logical like you describe.
In the real world, employers are swamped in resumes, and are just dying for a reason, any reason, no matter how arbitrary or lame, to discard some. Some will just mix them discard the bottom half of the pile. Some smart and successful people argued that you should discard anyone whose email address you don't like the sound of, or whose picture looks unprofessional, or whatever. At least one corporation is using numerology. Add the numbers for each letter in your name (where A=1, B=2, etc), add the digits of the result, repeat the last step until you have a single digit. If it matches the digit for the company's name, you're eligible, if not, noone will even read your resume. At all. Several corporations use tarot. Literally. Etc.
The only thing that matters is having a repeatable criterion, and one that doesn't fall afoul of discrimination laws. So even if you're not allowed to refuse employing someone because they're black, you can safely refuse to hire them because their name sums up to 3. Or because your HR department found something they dislike when googling them.
So even for the top experts, some will realize that they increase their chances of a better job, if they just keep their mouth shut. Even if it's a slight increase, hey, every bit helps. If keeping your big mouth shut gives you even a 1% chance of landing a better paying / more stable / better quality-of-life / etc job, there will be people who'll gladly take that advantage.
For the replaceable peons I've mentioned before? Doubly so. In fact, make it 10 times so.
Re:IRC is still alive? (Score:5, Insightful)
This attitude is widespread, but very problematic, because it is a departure from long standing social norms and communication modes: A free society has a need for public communication which isn't set in stone. If your only options are to keep something private or have it recorded for all eternity that you said it (and when, where, to whom), many important things will not be spoken publicly. It's not so much a problem of privacy or no privacy: A public channel is not private. It's a matter of forgetting the mundane, so that people need not worry about having their every public move inspected and reevaluated later on. The grace of oblivion is not implemented in our information systems. This lack robs us of our chance to change or start anew, and that stifles public discourse. Again, it's not so much the expectation of privacy which is violated by these archives, it's the perceived transient nature of IRC (and Usenet before DejaNews.)
Re:It's DejaNews all over again. (Score:3, Insightful)
I think this is the same thing. There's going to be resistance to the idea at first, because people aren't used to it and nobody likes something that works to change. But there's no reason why the change has to be for the worse and not for the better. I think an IRC log service could actually be pretty cool. Sure, there's a lot of stuff that goes on there, that I doubt anyone is going to care about later, but particularly in the technical channels there's a lot of good information given out from time to time. A good, well-known archive might prevent a lot of repetition, and allow users to make sure they're not asking things that get covered all the time.
There's no way to have a communications system where you're just screaming unencrypted bits out into the ether for anyone who wants to listen to them -- which is basically what both IRC and Usenet amount to -- and not let people archive them. There's no technical solution (you can try to keep blocking the logbots, but it's a losing battle if they're determined), and there's no real legal solution either (you could just set the archive up in some country that doesn't care about user's copyrights).
The IRC community has a chance now to embrace this, and in doing so, find some sort of middle ground (like the "X-No-Archive" header) that wouldn't get them into a fight with the people who want an archive that nobody can win.
Re:It's DejaNews all over again. (Score:3, Insightful)
Comment removed (Score:3, Insightful)
Re:Sorta (Score:3, Insightful)
I refuse to live in fear about what people may think about what I've done or said. I'd rather live with consequences than live in fear. Even if that means I'm first against the wall...
Re:IRC is still alive? (Score:3, Insightful)