Slashdot Log In
U.S. Gov To Spider Internet
Posted by
Zonk
on Thu Feb 09, 2006 10:56 AM
from the dog-house-farm-house-chicken-house-outhouse dept.
from the dog-house-farm-house-chicken-house-outhouse dept.
HopeSeekr of xMule writes "Perhaps as one of the first high profile uses of Alexa's WebSearch Platform, the U.S. government plans to search, link and reference every news site, blog and email on the Internet, using sophisticated AI codenamed ADVISE to do the correlations. Unlike traditional dataveilance like Echelon, ADVISE aims to find terrorists before they strike and even deduce their motivations in wanting to commit their crimes. Part of the breakthrough is a way for humans to view data as 3D holographic images with tech recently used at the Superbowl."
Related Stories
[+]
IT: Holograms Help Protect Super Bowl 287 comments
Apache4857 writes to tell us CNet is reporting that Homeland Security agents monitoring the Superbowl will be doing so in 3D. Using streams from two cameras, the LifeVision 3D system is able to project images onto a 20-inch screen that is equipped with a depth tube. This depth tube makes images appear to rise 30 inches off the screen and sink 30 inches into the screen allowing real world volumes and distances to be displayed accurately. Using this system security officials will be able to search sidewalks, monitor faces, and even peer under vehicles.
[+]
Developers: Alexa Web Search Platform Released 63 comments
Philipp Lenssen writes "Amazon's Alexa is releasing their search index (the same that powers the Wayback Machine) to developers via their new Alexa Web Search Platform. The Alexa framework is not for the weak of heart -- expect to learn how to use their C API, and expect to pay micro-amounts for requests and CPU cycles used -- but it also seems to be more powerful than the rival APIs from Yahoo and Google."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
again.. (Score:5, Insightful)
What if they communicate via
- plain old websites/ftps
- internet storage servers, irc, etc?
- instant messangers
- VoIP
- decentralised networks?
Lets not forget that they can
- obsfucate.. simplest method would be typing stuff into a CAPCHA-like image. OCR has no chance...
- use slang
- encrypt!
It will end up as an intrusion to the privacy of ordinary people unaware of this and/or private communications among companies.
Re:again.. (Score:5, Insightful)
What this amounts to is tracking thought-crimes, how can you know someone is going to commit a terrorist act until they do it? People say lots of things, people think lots of things. Whither freedom.
Parent
Re:again.. (Score:5, Interesting)
This differs from premeditated murder in that, yes, ultimately, premeditation is a thought.. but the key difference is that you were planning a crime. Planning to kill someone is not an opinion. Hating a group of people is an opinion.
It is basically legislating what sort of motivations for doing a crime is worse than some other motivation. So if you rob a bank because you hate banks should you be subject to stiffer penalties? If you kill a gay man because you hate gays how is that anything other than a murder? Hate crime, for me, goes way to far down the path of thought crime. Double plus bad.
Jeremy
Parent
Re:again.. (Score:5, Insightful)
If the purpose of your murder is to incite fear and terror in all gay people, then yes. It is far beyond a death threat - a criminal offense in itself - you've already gone through with it, the only question is who's next. Multiply that with the number of people you've threatened and we can easily put you away for good.
Hate thought isn't illegal, any more than other thoughts. What is pure hate crimes would have been called slander, libel, threats and more if they were done against an individual. You can't treat gays as inferior to straight people without treating a single gay man as inferior to a single straught man. Where does that leave your "All men are created equal..."? That it's okay as long as you insult many enough at once?
Parent
Re:again.. (Score:5, Insightful)
Good question, perhaps this will put it into perspective.
First, let's make this battery instead of murder. Murder is so heinous it is indeed hard to consider a major difference between one derived from "hate" vs. other reasons (being cheated in some way, result of a robbery, etc.). They are both "high crimes" no matter how you measure it, serious punishment will likely occur regardless of the hate component. Battery might be a better crime to illustrate the point, and probably one of the more common uses of hate aggravation (along with vandalism).
A non-hate battery crime usually is an event with some guilt on the part of the victim. Not saying they deserve it, but they probably did something to instigate it (insulted your friend, smashed into your car, etc.). The escalation was likely avoidable by apologizing, running away, or just keeping a cool head about you.
Take a hate-crime battery, and the victim was probably completely innocent, just being black, gay, muslim, etc. and at the wrong place at the wrong time was enough.
The perpetrator in the non-hate battery is likely regretful later, and is probably not an inherently evil person. They may need drug/alcohol treatment and/or anger management classes, etc. They are likely to learn a lesson, and will likely avoid repeating the offense in the future. There likely was no premeditation to it either.
The hate-based batterer is generally not regretful, perhaps even proud and satisfied. They will highly likely repeat it, and there is very little a victim can do to avoid it. This is a MUCH more dangerous person, and the punishment (and/or rehabilitation) needs to be much stronger (IMNSHO).
Another situation is a gay neighbor of mine that got burglarized. He came home to find his home burglarized, and "die fags" spray painted on his wall. I have been burglarized, and it was painful and scary, but I did not take it personally and I was not traumatized. I took it as a random, unfortunate, event not directed at me personally. Some druggy looking to finance his next fix. I could definitely see a difference in these situations, my neighbor was totally traumatized, as would I be. I definitely see the crime perpetrated against my neighbor as a far more serious crime than the simple burglary I endured, even though, at their core, they were otherwise similar.
Also be cognizant of other factors that can aggravate criminal sentences, such as recidivism, no remorse, etc. These are along the same lines as hate crime aggravations. They are all an attempt by society to allow for differentiation between one-time mistakes by the offender, and the much more dangerous criminals that will likely repeat and perhaps escalate their crimes. It is just codifying "hate" as an aggravation at the same level as some of the other factors.
I am not sure how well I am making my point, but I guess the bottom line is if you look at the victim impact, the impact of a hate crime on the victim (including their family and community) is far greater than than a non-hate crime. There is little a victim of a hate crime can do to prevent it. As well as the perpetrator of a hate crime is much more likely to repeat it.
I agree with earlier points, it can be difficult to determine when a crime is hate-based, or not. But in many cases, it is not all that hard. I do agree that assigning "hate crime" status to a crime should not be taken lightly or capricously, and if there is any reasonable doubt, should not be applied. I think (I hope) it is applied very carefully, and sparingly, in real life.
Parent
Re:again.. (Score:4, Insightful)
Not only do I agree that this is a 'war on personal freedom,' i also feel that this project has disaster written all over it. This 'AI' will have to be pretty intelligent to tag and organize all of this content in a meaningful way, and on top of that, those analysing the data will need to be pretty friggin' brilliant to use it correctly.
as you say, "People say lots of things, people think lots of things." I personally feel that there is no one who can honestly or accurately see all comments and verbalized streams of thought for what they are worth - usually just contemplation or teen angst.
while it is certainly *possible* that terrorists might use (or have used) globally accessible modes of communication to plan a major attack, monitoring the news wires and blogs is probably not the most effective way to prevent the attack.
we must continue to demand privacy at all times, however i feel that the push by the top levels of government to gain access to our souls could be our downfall as a society as we distance ourselves from each other in fear of relinquishing too much information.
Parent
The most terrifying quote in the article (Score:4, Insightful)
This is not something "we" need to be willing to do! My civil liberties are NOT YOURS TO GIVE AWAY! I'm terrified that a CS prof at Stanford thinks that it's no big deal that the US wants to spy on its own citizens and deprive us of our rights under the 4th and 5th amendments. (Yes, the 5th ammendment too, since US Citizens have been held on US soil without being charged with a crime, and thus deprived of due process of law.)
How can any educated person think this loss of privacy is "no big deal"? I'm at a loss for words.
Parent
Re:again.. (Score:5, Informative)
No, but it'll sure help keep the lid on political dissent, won't it?
Portions of this have already begun: the data mining only extends prior government watching of the web [washingtonpost.com] for "terrorists" like the ACLU [irregulartimes.com]. But [11alive.com] not [aclu.org] for [sfgate.com] political [icdc.com] speech [thirdworldtraveler.com], of [nytimes.com] course [aclu.org]. Never that. [findlaw.com]
So shut your mouth and shut down your blog and stop commenting here if you don't want to end up on a list of people to be "neutralized" -- like Mario Savio, hounded for ten years despite never breaking a law [sfgate.com].
Savio's "crime" was, ironically, leading the Berkeley Free Speech Movement. We'd do well to remember today 0Savio's words then:
Parent
Wrong name (Score:5, Funny)
BUSH BOMB WHITE HOUSE (Score:5, Funny)
Sorry slashdot.
Re:BUSH BOMB WHITE HOUSE (Score:5, Funny)
I'm gonna shoot some Americans!
Negative. No knock on the door.
I'm gonna blow up the empire state building!
Negative again.
I'm gonna poisen the American food supplies!
Negative yet again. Damn I might as well stop and start downloading some mp3s...
KNOCK KNOCK KNOCK!
Parent
The Erosion of America (Score:4, Insightful)
Proponents of this initiative boast that other data mining systems, such as Starlight, have already proven their worth in the fight against terrorism. However, given the fact that the current administration knew full well [inthesetimes.com] that Osama bin Laden intended to use hijacked airliners as missiles in a terrorist strike, but chose not to act, and that the CIA managed to uncover this information without a wholesale violation of the privacy of American citizens, I really can't see the justification here.
Why exactly does the Bush administration need such vast amounts of information to conduct their 'war on terror'? And why were they unable to use the perfectly good intel they did possess to thwart the worst terrorist attack ever on American soil?
One thing's for sure...it doesn't really matter whether the people OK this initiative or not, as Dubya & Company have amply demonstrated a complete contempt for the law of the land.
The quote that annoys me... (Score:5, Insightful)
Excuse me?
If what he says is true, then it's possible that the technology has been used to protect our lives. Our freedoms are a different matter. Which of the two you consider to be the more important is a pretty strong indicator of whether you're a free country or a police state.
Parent
Spider info (Score:3, Interesting)
Also, does it obey robots.txt?
www.terrorists.evil
User-agent: US-govt
Disallow: /
Robots (Score:5, Insightful)
I don't suppose this is going to honor the rules in my robots.txt.
It won't work. (Score:5, Insightful)
This just looks like the security people are getting desprate and trying to cast a wider net. The secret wiretaps used on citizens was a wide net that seems to have had poor results [nytimes.com].
Exactly how (Score:4, Insightful)
Blogs and news sites are things we publish to the world and are easy to spider. Emails are private communications. In order to monitor them you have to either intercept them in transit or search records on private servers. Even if the email is available via webmail, you have to gain unauthorized access in a way that is generally considered trespass.
Can it detect sarcasm? (Score:5, Interesting)
I highly doubt it.
Can it translate accurately from Arabic to English?
Somewhat more likely, but I still doubt it.
Do they really think the terrorists get their books from Amazon.com?
They are going to spider 2 million porn sites, and keep records of my purchases from Amazon, and never catch a single terrorist.
Sounds like a big waste of time and (my) money.
i.e. business as usual.
My guess is upper management at DHS doesn't understand computers too well, so they hire a bunch of nerds who can program, and give them absurd instructions, like "We need to get a handle on this whole internet thing!"
Or something ridiculous like that.
Something I've always wondered... (Score:5, Interesting)
Why is it that it is always the US government that seems to have been up to all this stuff since WW2 and increasingly even after the Cold War? I thought you were supposed to be the people from the land of the free and whatnot, really suspicious of government intrusion into people's lives, et cetera. Considering that a lot of you are always willing to disparage the Europeans for their love-affair with government, I certainly wish a lot of you would just take the log out of your own eye first... it's your government, despite all the rhetoric, that is horribly control-mongering at home and eager to support whatever right-wing dictator abroad, while ours concentrate more on making sure that kids with cancer don't die in the name of economic efficiency should they be unfortunate enough to be born to parents of financially limited means.
Go ahead, mod me troll/flamebait... at least I won't post this AC.
When "24" is the source of knowledge.. (Score:5, Funny)
Jack Bauer : Chloe, I'm sending you a picture. Can you datamine for him?
Chloe O'Brian: Sure. send it to my screen.
Computer: Blip...blip...blip.
Chloe O'Brian: Jack - it's the well known terrorist named...
Interesting (Score:5, Insightful)
Singlehandedly causing the West to self-destruct is no small potatoes.
War on Terror - my a... (Score:4, Interesting)
Sorry for the language, people, but I feel insulted. Just how DUMB do they think I am?
Terrorists don't use the net. At least not if they're halfway smart, and hell, they are! They ain't some dumb, mindless bomber drones (ok, some are, but look at the US soldiers... same way 'round, just with rifles). The key heads are very bright individuals, they know what they're doing. They know logistics, they know psychology, they know how to build a network right around your feet without you noticing.
Do they use the 'net? Let's assume they do, ok? Let's for just a moment assume they do.
First of all, they WILL NOT use the net for anything but the minimally necessary form of communication. They won't blog, they won't chat, they won't spend time in a bboard, all they do is MAYBE sending some data from A to B. And it won't be much data.
This data will be encrypted by best state-of-the-art encryption.
A good deal of this data will be plain false, and it will be false in a way that they can discern whether the feds were sniffing. Simply for testing their communication channel for being tapped and their key for being broken.
If you consider, all this incredible effort just 'cause some oil countries dared to think 'bout taking Euros instead of Dollars for their crud... it's amazing what some old hydrocarbones can move and shake in this world.
I'm not defending Bush but (Score:4, Insightful)
I think the inclusion of email is what gives this the swarmy, big brother overtones. We've also have ample evidence that the Bush administration can't be trusted. The combination of Bush political flaks with no regard for privacy or the law and large amounts of personal data is what makes it scary to me.
Re:Big Brother ADVISEs you! (Score:5, Funny)
Oh, I'd say in about 5... 4... 3... 2... 1... [signal terminated]
Parent
Re:Is this starting to create a bigger problem? (Score:4, Interesting)
I seem to provide this quotation quite frequently these days. It was said by Lord Hoffman, sitting as a British Law Lord, in their ruling on the UK detention-without-trial fiasco a few months back:
And, unlike the rest of us, the Law Lords sitting in that case presumably did have access to any classified information they required. It's very convenient that the government can always tell us how its draconian policies are protecting us from imminent doom (but they can't tell us how for security reasons). That argument is rather less powerful when its critics include people on the inside who would be well aware of the full facts.
Parent