GAO Studies U.S. Government Data Mining 141
securitas writes "Total Information Awareness is alive and thriving. eWEEK's Caron Carlson reports on a new General Accounting Office study that says TIA-style data mining programs are rampant in federal agencies with 199 projects at 52 of 128 agencies. The Defense Intelligence Agency/DoD is the single largest user of these data mining projects (eg. Verity K2 Enterprise). The story was first reported by Reuters' Andy Sullivan (ZDNet UK mirror) and the NYT's Robert Pear, who wrote that at least 122 projects used personally identifying information like names, e-mail addresses, Social Security and driver's license numbers. The 'actual numbers are likely to be much higher' because the report excludes classified projects. Wired News' Kim Zetter writes that, in addition to government databases, federal agencies mine private databases of credit rating agencies, bank account numbers, student loan applications, etc. This week the Center for Democracy and Technology (CDT) released a report with privacy guidelines for data mining technology (PDF) development and use. Guidelines include data anonymization, government data access authorization and audit trails. Cynthia (Cindy) Webb's 'Total Information Dilemma' at the Washington Post is an excellent survey of media coverage of TIA, MATRIX and the GAO report 'Data Mining: Federal Efforts Cover a Wide Range of Uses' (mirror, both in PDF format). More at GCN, GovExec and the Guardian/AP."
URL spam (Score:3, Insightful)
Re:URL spam (Score:1, Funny)
You have a 1 in 15 chance (Score:1)
Its NOT URL spam (Score:3, Insightful)
the poster has apparently gone to a fair ammount of trouble to bring these links together into a nice written summary so that you dont have to go out and search for the information yourself (as if you would anyway... pfft).
point being...
If you are too lazy to click on a few
Re:URL spam (Score:3, Informative)
A summary of the summary is though:
Wired News' Kim Zetter writes that, in addition to government databases, federal agencies mine private databases of credit rating agencies, bank account numbers, student loan applications, etc. This week the Center for Democracy and Technology (CDT) released a report with privacy guidelines for data mining technology (PDF) development and use.
Re:URL spam (Score:1, Troll)
Naw, it's just a duplicate of a "Slashback" article :)
Thank god... (Score:2, Funny)
Re:Thank god... (Score:2, Funny)
Wonder what happened to the back up tapes.
Here's the difference... (Score:5, Insightful)
From the article... "Scores of data-mining projects that collect and analyze U.S. citizens' personal information are in operation at dozens of federal agencies, the GAO found. Many of the nearly 200 projects planned or already under way rely on data purchased from the commercial sector"
They are purchasing the information from the commercial sector", information that is readily available to anyone willing to pay for it. Congress stopped their TIA initiative, but will allow this due to that major fact.
Re:Here's the difference... (Score:5, Informative)
And it is not allowed to give information away or even sell them (that is only allowed for public available information)
Re:Here's the difference... (Score:3, Interesting)
This is called the collection limitation principle - There should be limits to the collection of personal data and any such data should be obtained by lawful and fair means and, where appropriate, with the knowledge or consent of the data subject., cf. OECD Guidelines for privacy/data protection. [cdt.org]
Did you *even* read the parent post ? (Score:2)
Re:Here's the difference... (Score:3, Interesting)
The power of the government purse. The spending power of the government is huge and able to afford this. Ordinary citizens are not.
Another side effect is that the companies that collect and distribute this information are enriched and emboldened.
Re:Here's the difference... (Score:2)
Re:Here's the difference... (Score:2)
In the EU/EEA, this would be illegal, as there are legal limits to third party transfers of personal data.
Re:Here's the difference... (Score:1)
They are purchasing the information from the commercial sector", information that is readily available to anyone willing to pay for it
-----
So what's to stop a politician from helping his best friend set up a commerical data collection agency and then feeding his best friend with lucrative government data projects at exorbitant prices in order to funnel enormous amounts of taxpayer dollars to his best friend?
Yes. I _am_ a paranoid conspiracy theorist. Feel free to troll away.
Re:Here's the difference... (Score:1)
So we need to get word out to these companies that we don't want our information being sold to the government, or we won't do business with them anymore...something to that effect?
Is there a list of the companies that are doing this?
Re:Here's the difference... (Score:2)
Do you think these companies really care that you won't do business with them anymore, once they have your data that they can sell?
Threats of not doing business with company so-and-so are not good enough. We need to get these companies where it counts. Fine them heavily if they sell personal data to third parties without the c
Re:You are forgetting something... (Score:3, Interesting)
This is the government. We have laws, and a Constitution that (purportedly) protects us from the government by limiting its power through a series of checks and balances. Yes, this information may be available to the general public, but John Q. Public doesn't have military or police force that come beating in your front door and taking you into custody because your personal information matches a certain pattern - whatever that may be.
Herein lies, I'd argue, at least one reason behind the 4th Amendment to t
Re:You are forgetting something... (Score:2)
I appreciate the rationale behind your argument, but it is legally wrong. The Fourth Amendment says that the government may not conduct warrantless searches (except where obtaining a warrant fi
Re:You are forgetting something... (Score:5, Insightful)
And I'd argue that this (even though it might be the current standing of the U.S. Supreme Court), has no merit, and the reason is simple: Our society is built on an infrastructure that, for all practical purposes requires that I do certain things. I have little choice but to place my trash out on a curb for pickup, because there are few, if any other practical options. The fact that I have few (if any) other practical options does not portend or in any way grant anyone a "right" to take advantage of something merely because it is available, and especially because its availability is a byproduct of the infrastructure itself. I use the infrastructure because, as a pragmatic and civil-minded denizen, I must.
Let's address the issue with the "publicly available" information. While there are certain records that are unquestionably public information, I'd argue that what I purchase is NOT public information- each purchase is a contract between myself and those vendors with which I choose to do business. Unfortunately, the easy availability, and wide use of credit, have allowed institutions to hijack, retain, sell, and otherwise use the specific details associated with each transaction, and make them available as they see fit.
Something to think about - Let's say I cut my finger while preparing dinner. I use a kleenex to clean the wound, before applying a bandaid. What am I now to do with the kleenex? It has my blood on it, and presumably, can serve as a "sample" for anyone interested in fishing it out of my garbage. What am I to do? I don't believe that anyone has has a right to acquire this without my permission, or without a warrant. I am placing it in the garbage not because I have any intention of making it available to the public, but because this is the proper and accepted method of disposal.
Here you can see the quandary imposed by a) the infrastructure, and b) the court's current standing. I don't believe it is fair, and I hope to see it change at some point in the future.
Why's the source matter? (Score:2)
This is warrantless search and seizure in my view. Looking for patterns to identify you as a potential criminal or terrorist? Is this what it's come to?
(brace for storm of outrage) (Score:5, Insightful)
It's like the argument about ID cards - there's nothing inherently wrong with being able to conclusively prove that you are who you say you are, but it's another thing entirely for people to be able to demand you prove the same.
Re:(brace for storm of outrage) (Score:3, Funny)
Re:(brace for storm of outrage) (Score:1)
while that's absolutely true, no questions, its also not exactly the issue. this issue is: what does the government do with this information other than prosecute real and genuine baddies? (n.b. real and genuine baddies are few and far between)
for example, historical use of personal information (such as which groups people participate in) w
Re:(brace for storm of outrage) (Score:2)
It is not that I do wrong things, it is that you may think I am doing them, so I'd rather not feed your fertile imagination with my personal details.
Re: (Score:2)
Re:(brace for storm of outrage) (Score:2)
b) I buy alcohol for minors all the time, and I know lots of minors who get other people to buy them cigs.
c) If my family want to find me they will report me missing, or come looking. I tell them where I am, and they don't need to see id.
So, there is no reason for ID, unless you are the suspitions or criminal type.
No way (Score:2)
Are we so terrified that crimes must now be prevented before they occur? Kryzt, give me a fuckin' break.
European Protection Stronger for Personal Info (Score:4, Insightful)
Without laws preventing such mis-use, it runs rampant. European laws guarding personal information are much stronger than in the US, where corporate and government interests and methods are closely aligned, especially these days.
European data protection (Score:5, Interesting)
Re:European data protection (Score:2)
Re:European data protection (Score:2)
(*) Happy with them for the most part, but not always. They could do a lot better, but apparently are low-priority when it comes to funds from the Department of Justice.
Re:European data protection (Score:2)
Anyway, that's not a privacy concern.. The biggest issue, I guess, is all the unlicensed surveilance cameras that are popping up these days, especially in Oslo, and according to some spokesperson I saw on the news, you're right, they don't have the cash to do anything about it... Overall though, I'm still quite satisfied with them... (If you live here though, I guess you knew all this allready...
The stink starts right at the top (Score:3, Insightful)
Like the Abu Ghraib Humiliation scandal the intellectual direction has been set at the top levels of the administration, even though the work on the projects is being done by lowly Primary Investigators.
What I wonder is whether the ones setting the intellectual agenda in the administration have any idea ofwhere they are leading us.
I fear, that one day we will be left in shock and awe when we discover that their idea of "preventing terrorism" was as well thought out as the idea of democratizing Iraq. We will all be at the mercy of a State that, like the Iraqi Monster, has grown too big for them to bully around. And then all of us, being in the same boat, will realize that those bas***** in the administration are screwed - but so are we.
.
Re:The stink starts right at the top (Score:2)
By humiliation do you mean what our Dear Leaders experienced? Or do you mean what the prisoners experienced?
If the latter, call it what it is: Abu Ghraib Torture scandal.
Bush Admin Right On (Score:4, Insightful)
Bush administration officials ARE responsible for the objectionable actions at Abu Gharib and ARE taking action against those that committed them. That's what being responsible means. Compare that to the previous administration that dodged most responsibility and lied to cover it up.
Don't poor-mouth the intellectual agenda. It is brilliant and is working.
Re:Bush Admin Right On (Score:1, Insightful)
What a chump. Do you really think it was just a bunch of privates and corporals who came out with this idea of abusing prisoners? No, this goes right to the top. Did Rumsfeld say "make human pyramids", of course not. But he certainly said "Get info out of these guys no matter what it takes".
Did some privates tell the JA
Re:Bush Admin Right On (Score:3, Insightful)
Do you really think it was Rumsfeld or any other administraion official who instructed the prison guards to document their deeds with photos, and then leak them to the press? Not a very smart move, huh? Chances are, they acted alo
Re:Bush Admin Right On (Score:1)
Recovering economy, despite the huge collapse that started in OCT 2000
-----
Give it up. Anyone with half a clue was watching the dot com bubble getting thin as early as mid-1999. The rest of the market was feeding off of the assets and volume generated by the artificially inflated technology sector.
-----
Homeland safety, despite the onslaught of world terrorism?
-----
Give it up. The nightly news has recently become obsessed with every molotov cocktail being thrown by some disgruntled Palestinian. The
Re:Bush Admin Right On (Score:1)
Oh, so the fact that Rumsfeld doesn't think it would be a good idea to document your own war cri
Re:Bush Admin Right On (Score:2)
Well It isn't just bush that oposes it. It is the people of the untited states that opose it. Also you pretty much answered your own question by grouping the U.S. into a group of villian states.
One of the bigest reasons i have oposed it and writen my gongressmen about it (remeber the president cannot sign a treaty without the consent of congress. congress has to radify it and give it to him with thier aproval before he can legal
Which 75 stories ..Re:Gosh... (Score:2)
If you tell me that the 75 stories are the bottom stories then I agree with you. The problem is that you are showing me the top 75 stories standing up in thin air, and asking me to believe that the bottom 25 stories will soon be done. In this case, which is what it is with the Administration's "achievements" in Iraq, it is a cause of considerable worry. Pure and simple delusion.
.
Re:The stink starts right at the top (Score:1)
"
Seriously
To give the tin foil hat view of the whole thing.. (Score:3, Interesting)
1: Police officers have been taking pictures of protesters to throw into databases. Additionally, wearing masks is now illegal in most cities.
2: Facial recognition software enabled camera's are going up everywhere. Sure, the software barely works but it'll get better. This is first justified in protecting people in high crime areas like chicago, and then will move out as far into the suburbs as they can justify it.
3: Our economy is going to shit and we're going farther and farther into that hole.
4: Massive databases of personal information are being assembled by our goverment.
5: Our constitutional rights are gone.
6: And to put the decorations on the cake, at the G8 summit at Sea Island in Georga, an order to "fire to kill" was given to all police officers in the state in regards to whoever they found was acting suspiciously. This means that if you're a protester outside of the building, the cops and military in the area can legally shoot and kill you.
So, we've got our reichstag fire (9/11), we've got our Decree of the Reich president(Patriot act), now we're waiting for the enabling law (aka, the law that let the reichstag put people into concentration camps).
Time to get your gun (Score:2)
Come on, are you a sheep? Are you going to rattle off a list of things like this and say you have no responsibility for changing things? If you honestly believe your list, you know the 2000 election was rigged and the 2004 election will have a similar result. So, politics are dead in the US. What are you going to do?
If you do nothing it will soon be too late. How about a repeal of the 22nd amendment so George W. Bush can be "President for Life"? Do you
Re:Time to get your gun (Score:1)
Re:To give the tin foil hat view of the whole thin (Score:2)
Balancing the right to protest with the prevention of a potential suicide bomber is tough. Where is your personal line? What would you have the security forces do?
Re:To give the tin foil hat view of the whole thin (Score:2, Flamebait)
Re:To give the tin foil hat view of the whole thin (Score:1)
Really? We have had 4 straight quarters of record breaking growth, and have added 1.2 Million jobs in the past 6 months. If that is the "hole" we are going to, I say keep digging!
Re:To give the tin foil hat view of the whole thin (Score:1)
We have had 4 straight quarters of record breaking growth
-----
Damn those slaves can pull hard when they're whipped heavily enough.
-----
and have added 1.2 Million jobs in the past 6 months
-----
Yeah, there are Wal-Marts and McDonald's are going up everywhere. I'm still suspicious of how those numbers are generated. If my company trims 5000 people and then hires 250 new people, is that 250 jobs added?
Re:To give the tin foil hat view of the whole thin (Score:2)
So, having the most people working for the highest wages in history suddenly qualifies as slave labor?
Yeah, there are Wal-Marts and McDonald's are going up everywhere. I'm still suspicious of how those numbers are generated. If my company trims 5000 people and then hires 250 new people, is that 250 jobs added?
This isn't mystery science. The statistics are freely available [bls.gov] to the public. For example, here are the statistics for the mo
Re:To give the tin foil hat view of the whole thin (Score:1)
The number of unemployed people went down by 188,000 to 8,164,000
-----
Is that reported as newly filed unemployment claims? Even the nightly news will report that many people have decided that standing in line at the unemployment office isn't worth the effort.
All of the comments about the number of employed people correlates well with the new KFC they're building down the road.
-----
The average hourly earnings for production workers is at an all-time high of $15.59 per hour
-----
Averages are the most e
Re:To give the tin foil hat view of the whole thin (Score:2)
No, that is the total number of people that collect unemployment. The BLS also tracks the number of people that drop off unemployment because they are discouraged about the job prospects. Guess what
Re:To give the tin foil hat view of the whole thin (Score:1)
The average weekly earnings for private sector employment is $545.38? I barely make that with a top-notch education. Is it possible that number is skewed by the all-time high of CEO compensation disproportionately skewing the scale? $525/week, here in the US, barely pays the bills. Knock of 30% for taxes, then knock of 6% sales tax on any
Re:To give the tin foil hat view of the whole thin (Score:2, Informative)
3: Our economy is going to shit and we're going farther and farther into that hole.
GDP is growing faster. [cnn.com] Unemployment is going down. Manufacturing is strong. The stock markets are moving up. Now by what measure does this translate to "shit"?
The facts are so clear the US economy is back on track again, why do these idiots keep sticking to their view from 3 years ago? This meme needs to die. It's not true, and it's certa
Re:To give the tin foil hat view of the whole thin (Score:1)
GDP is growing faster. Unemployment is going down. Manufacturing is strong. The stock markets are moving up. Now by what measure does this translate to "shit"?
"
Be careful
Since the grandparent post talks a lot of Nazi-Germany, I just want to add, that after Hitlers election in 1932 the German GDP was growing fast, too. Unemplyment went down, too. Manufacturing was strong, too. Stock
Re:To give the tin foil hat view of the whole thin (Score:1)
Just what time period are you looking at? Let's look at the 5 year history of the DOW: http://finance.yahoo.com/q/bc?s=^DJI&t=5y
Through 1999 and until about June of 2001, it was hovering around 10,500 or so. Sept. 11, 2001 caused the dive to near 8000. DOW goes back up again to around 10,000 until about June 2002. Then another dive, this time down to about 7,200 in Sept. 2002. Goes back up to around 8,700 through X-Mas 2002, takes another dive through Q1 2003 (wh
Re:To give the tin foil hat view of the whole thin (Score:3, Insightful)
You mean a law that would allow people to be detained without a reason, thrown in a Guantanamo Bay prison and not allowed to speak to anyone including legal counsel? Don't be silly that will never happen!
Re:To give the tin foil hat view of the whole thin (Score:1)
Re:To give the tin foil hat view of the whole thin (Score:1)
You obviously have missed something
Inneficiency cancels out negative effects (Score:2, Funny)
Sharing Info (Score:2, Interesting)
Insider (Score:3, Funny)
Re:Insider (Score:5, Funny)
Pah. You won't be. My guess is that some time around November 2009, Google's server farm will reach a critical mass and achieve self-awareness. It already knows everything - all it lacks is a mind. It will probably have enslaved the human race by Christmas.
Public data should be public (Score:2, Insightful)
I have trouble seeing how the availability of information is in itself a bad thing. Yes, it can be used to draw false conclusions, but that is a fault of its use, not its existence. And its not like false conclusions are never drawn without databases of public information. Anyone ever heard of racial profiling?
We should not worry about regulating the ex
Re:Public data should be public (Score:2)
I don't believe the police should be able to troll the records of millions of innocent people looking for few troublemakers anymore than they should be allowed to randomly search homes. It's not that big of a step from one to the other. Probable cause should apply everywhere. In your home, your car, your data, including u
Re:Public data should be public (Score:1)
You are free to not use credit cards, get loans, etc., if you don't want to. Hell you can go become a hermit somewhere out in the mountains if you want to. But for the rest of us, it is worth the small loss of privacy.
"I don't believe the police should be able to troll the records of millions of innocent people lookin
Re:Public data should be public (Score:2)
Why does that make that data public? Why does the fact I have a credit card or what I buy with it qualify as public knowledge? I happen to think that what I buy is no one's damn business but mine. Even property records give away too much information in my opinion. I shouldn't have to be
Re:Public data should be public (Score:1)
Because you signed a little form saying it was public information.
"I happen to think that what I buy is no one's damn business but mine. "
By definition it is the business of companies who are giving you credit. If you want them to lend you money, they have to know you will pay them back.
"Privacy doesn't disappear in one big event..."
The privacy you are talking about never e
We have a file of pointless trivia on you (Score:4, Funny)
Or maybe they'll be convinced I'm a terrorist because I read CNN *and*
Re:We have a file of pointless trivia on you (Score:1)
data quality? (Score:5, Interesting)
2. Data Quality Principle [cdt.org]
Personal data should be relevant to the purposes for which they are to be used, and, to the extent necessary for those purposes, should be accurate, complete and kept up-to-date.
How does the government separate the junk data from what may actually be worth looking at?
Re:data quality? (Score:1)
How does the government separate the junk data from what may actually be worth looking at?
-----
They don't. The government approach is "there is no such thing as bad information." Conflicting information is traditionally resolved in a manner which best suits the government. If the conflicting information involves a citizen who is well connected then the information is resolved in favor of the citizen. If the conflicting information involves a citizen who is not well connected then the government has
Re:data quality? (Score:3, Insightful)
In the case of the 2000 Florida elections and the woefully inaccurate convicted felons database [gregpalast.com], the answer was simple: assume the database is correct, especially if it contains a political demographic that is likely to vote Democrat. The ~50,000 non-felons who were denied their right to vote were too poor to sue, so no big deal. I may remind our gentle readers that the Florida presidential elections were decided
Re:data quality? (Score:3, Interesting)
I read an article a while ago that painted an interesting scenario. A person living on "Lebanon Street" is put on the no-fly-list due to connections to Lebanon. And of course, the poor sod has no way in getting his situation rectified. I don't have any URLs, unfortunately; if anyone here read the same article and has an URL, I'd greatly appreciate seeing it again.
Re:data quality? (Score:2, Informative)
Absolutely true. I worked on a huge commercial/government mining and data integration project in the mid-90s (this isn't exactly something Bush thought of) and the state of the government databases were attrocious. Extremely dirty data, and of the over a thousand databases that were integrated, half the government ones look
Easy... (Score:2)
I *hope* my data is useless and cluttered. Although security through obscurity doesn't work well, obscurity itself if better than nothing.
The more chaff in the air, the harder it is to see.
Big Brother ... (Score:1)
Good (Score:1)
Re:Good (Score:1, Insightful)
Re:Good (Score:1)
Indelible Records (Score:1)
Why are they doing it? (Score:2)
Are we (in Europe) really crap at catching people or:
Do they want to catch everyone that makes the slightest error.
Is there a big problem in the US that requires data-mining compared to Europe.
Are you just crap at catching people in the US and need data-mining.
Are you planning on sacking half of the police force when data-mining by a couple of terminal operators catches everyone.
Help, it all s
Re:Why are they doing it? (Score:1)
If data is available and someone doesn't take advantage of it we call tehm stupid or ignorant. IF the governement uses available data we
Re:Why are they doing it? (Score:1)
Governement Data Mining (Score:2, Insightful)
well... (Score:3, Insightful)
First of all, in Europe, data is collected for specific purposes (use limitation principle) and can only be used for these specific purposes and not for other purposes, unless the data subject explicitly consents to new use.
Secondly, whether or not data is "legal" and "pubicly available," you have to get consent from the data subject in order to use that data. And of course you would still be bound by the use limitation principle.
Look who doesn't do data mining (Score:3, Interesting)
The GAO report that surveyed government departments to find out how much data mining they're doing and planning is interesting. In particular, the appendix that lists which departments don't do or plan to do any data mining has a few notable inclusions:
You'd expect some of those to have data mining as their primary function. Others are just hard to believe.
it'd be nice if... (Score:2)
- locate areas where increases in education, public health, and crime prevention spending would have a large positive impact
- identify major sources of pollution
- find patterns of police brutality
- predict and avoid major traffic problems
Somehow, I don't get the feeling that these are the goals of most of these projects.
(C)opyright protections (Score:2)
We found him! (Score:1, Funny)