Forgot your password?
typodupeerror
Privacy Databases Government Programming Software IT News

A Look Into the FBI's "Everything Bucket" 31

Posted by kdawson
from the what-they-know-and-when-they-know-it dept.
Death Metal notes an EFF report on information wrested from the FBI over the last three years via Freedom of Information requests. The report characterizes what Ars Technica calls the FBI's "Everything Bucket" — its Investigative Data Warehouse. (Here's the EFF's introduction and the report itself.) The warehouse, at least 7 years in the making, "...appears to be something like a combination of Google and a university's slightly out-of-date custom card catalog with a front-end written for Windows 2000 that uses cartoon icons that some work-study student made in Microsoft Paint. I guess I'm supposed to fear the IDW as an invasion of privacy, and indeed I do, but given the report's description of it and my experiences with the internal-facing software products of large, sprawling, unaccountable bureaucracies, I mostly just fear for our collective safety."
This discussion has been archived. No new comments can be posted.

A Look Into the FBI's "Everything Bucket"

Comments Filter:
  • Well, I'm glad someone out there with lawyers is taking advantage of the Freedom of Information Act [wikipedia.org] (FOIA). To see how the EFF has taken advantage of it, their main FOIA [eff.org] with the subpage on this entire DOJ Investigative Data Warehouse topic [eff.org] and all the documents they've collected (some are linked in main story) [eff.org].

    If you are a US citizen, you yourself are able to make a FOIA request [state.gov].
    • by Smidge207 (1278042) on Friday May 01, 2009 @10:29AM (#27786745) Journal

      *sigh* My dear frined eldavojohn is suffering from a case of naivety, I fear.

      In fact most of my FOIA requests have been with the FBI. To date, I've filed 57 requests with them. Of these, 8 have resulted in documents, 18 were "no records" (including cases where records had been destroyed), and the remainder are still open. The most important thing to know about the FBI is that their records system is decentralized. By this I mean that every FBI field office keeps its own records, and there is no centralized database for searching all records at once. Yes, FBI Headquarters (HQ) in Washington, D.C. keep lots of files, but just because FBI HQ doesn't have any records on a topic doesn't mean that the field offices don't. For example, I requested information on an organization called the Youth International Party Line, or YIPL, which was based in New York City during the 1970s. FBI HQ had no records on YIPL, but the New York field office did.

      What this means for you is that you need to send FOIA requests to both FBI HQ and to any FBI field office that you think might have records responsive to your request. This is a pain, but it's not the end of the world. There are about 56 FBI field offices and you can use the same FOIA request letter for each one, so we're talking about at most $22 in postage to spam all of their field offices with your request. This link provides a list of all the FBI field offices, and this other link provides the same list in Microsoft Word mailing label format. The second important thing to know about the FBI is that their records system is broken up into "automated" (i.e., computerized) and "manual" indices. The manual indices are for criminal investigations prior to 1973 and security investigations prior to 1958. By default, the FBI generally searches only the automated indices, since it's much easier for them to do so. If your request covers times prior to 1973, you should add a big bold note specifically asking them to search both the automated and manual indices. Wise up or shut up, eldavojohn, my dear twisted sister.

      =Smidge=

      • Well, that's unfortunate but with the massive failure of Virtual Case File [wikipedia.org] (and at extreme taxpayer expense), you can understand why you have to mail to every field office.

        In fact most of my FOIA requests have been with the FBI. To date, I've filed 57 requests with them. Of these, 8 have resulted in documents, 18 were "no records" ...

        I would consider your story a success story. It seems you don't but you recieved what documents they could provide to you.

        Here's my own anecdotal worthless history of FOIA. I was a junior in high school and was dissatisfied with lunch prices of Aramark (the same people who rape you at arenas and stadia) in our cafeteria. Every month they would systematically increase prices on all products by five or ten cents and it got to be ridiculous not long after. Then they "locked down" the campus so we couldn't leave for lunch. Which really really pissed me off. Yes I could have brought my own lunch but I didn't really like doing that.

        So I asked my friend to ask his dad (lawyer) for a template FOIA and filled it out with three other kids. We signed our names requesting the public high school release all details on their contract with Aramark. Instead, they brought us into an office room and gave us everything. I think that was an attempt to dissuade us but instead we were there late into the night. We had records on everything. What ever teacher was paid, what every contract had been made with an external business, everything. So we looked into the lunch provider history. The school had made some sort of several year contract with Aramark (not uncommon I guess) but that made them the only purchasable food.

        The rest of the story is pretty offtopic. But I found that to be a highly successful and satisfying use of the FOIA on the local level. I'm sorry Federal cases don't sound as profitable and I don't mean to sound naive but it is the Federal Government. You have to expect bullshit bureaucracy there--I'm sure field offices requesting documents from other field office experience the same problems.

        • The rest of the story is pretty offtopic. But I found that to be a highly successful and satisfying use of the FOIA on the local level. I'm sorry Federal cases don't sound as profitable and I don't mean to sound naive but it is the Federal Government.

          You do know that the FOIA only applies to the Federal Government to start with, right? Many states have sunshine laws, which may apply to local governments within the state, and local governments may have their own sunshine laws as well, but those are not the F

  • I got the impression (remember, this is a kdawson green light *sigh*) that it was more of a super gigantic database rather than a data mining system. That is, any information derived from the data is done by humans. Data mining is incredibly difficult even WITH perfectly clean data, much less the gigantic mess that seems to be the everything bucket. Meh, color me unimpressed.

    =Smidge=

  • by Drakkenmensch (1255800) on Friday May 01, 2009 @10:12AM (#27786555)
    ... way out there, and it's done in MS Paint.
  • by iamhigh (1252742) on Friday May 01, 2009 @10:25AM (#27786703)
    I mean first of all Windows 2000 is currently still the most rock solid, impenetrable [slashdot.org] OS you can use. There is no reason to think a more secure OS has been released [slashdot.org] since that time.

    And everyone knows that hardcore web designers use MS Paint. [wordpress.com]
    • by Anonymous Coward

      "I mean first of all Windows 2000 is currently still the most rock solid, impenetrable OS you can use. There is no reason to think a more secure OS has been released since that time." - by iamhigh (1252742) on Friday May 01, @10:25AM (#27786703)

      Windows 2000 (&, later Windows versions) can be made to be VERY secure, by security-hardening them, & secured based on "industry best practices"!

      (Per the multiplatform CIS Tool, which also mind you, runs on various *NIX OS', such as Solaris, quite a few BSD variants (since you noted it in your 2nd url (no MacOS X though afaik)), & yes - Linux too), quite easily - heck, the CIS Tool makes it actually sort of "FUN" to do (almost like running a PC performance benchmark test).

      In fact, Windows 2000 P

      • by Gr8Apes (679165)

        Disabling the server and browser services, among others, are the first things I do with a windows system. Yes, this means no file sharing from a windows machine, but how often do you do that anyways on most home single system setups?

        It's true that this will block most worms, as there will be few open ports left. It still doesn't mean that the system is secure. The fact that much code runs through DLLs with system level access allows for buffer overflows to basically run anything they want at root level, so

  • Something cool [flickr.com] will be born out of this "sea of information"...
  • by bunratty (545641) on Friday May 01, 2009 @11:01AM (#27787097)
    Today's xkcd [xkcd.com] is apropos.
  • Here's a concern. Given the degree to which morons are let enter data to run-of-the-mill DBs like the one in the article, what's the level of accuracy? How many times has some nimrod phoned you up to generate harassment due to bad data? How many times have you yourself called to correct something only to be told "well, the computer says so, it must be right!"

    People will make serious policy decisions based on this bad data.

    Voluminous quantities of stupid and failure are certain to follow.

  • Why? It sounds like it is too mucked up to be very dangerous.

  • Jack Lint:
    It's not my fault that Buttle's heart condition didn't appear on Tuttle's file!

    It's Brazil! [imdb.com] all over again...

    • by Jaazaniah (894694)

      Mod parent up! I can't wait till someone gets wise and publishes a story similar to this.

      On the other hand, I wonder if there's anything to gaming decentralized records, such as attracting attention in the northeast states, slipping off through the border, circumvent the globe, and arrive in LA to do the same shit all over again with reduced risk.

      Of course, I've probably landed on at least one of their watch-lists for simply mentioning it, but hey, what does that say about their organization if some lay-per

  • Is this a bucket or a "Raiders of the Lost Ark" warehouse?
  • "Tyranny, tempered by incompetence"

  • I guess I'm supposed to fear the IDW as an invasion of privacy, and indeed I do, but given the report's description of it and my experiences with the internal-facing software products of large, sprawling, unaccountable bureaucracies, I mostly just fear for our collective safety.

    After the experience I just had today with an agent from a Federal agency, I have to agree. He needed to use a computer that couldn't be traced back to his agency for undercover reasons, and that agency is remodeling and rewiring th

TRANSACTION CANCELLED - FARECARD RETURNED

Working...