Group Files 'Largest FOIA of All Time' (vice.com) 29
Reclaim the Records -- a group of activist genealogists, historians, journalists, teachers -- has filed what may be the largest Freedom Of Information Act Request of all time. The group wants the National Archives and Records Administration (NARA) to release billions of digital images and their associated metadata to the public. From a report: NARA is a government agency that preserves and archives the American government's historical records. It's also supposed to increase public access to those records. To accomplish that goal, NARA partnered with private companies such as genealogical website Ancestory.com to digitize and upload census records, immigration records, and other historical documents. Digitizing these records is a massive task, one NARA likely couldn't accomplish on its own. In exchange for its help, NARA granted the private companies limited exclusivity to the records. That means that billions of documents related to America's history are behind paywalls on sites like Ancestry, FamilySearch, and Fold3.
According to the agreements, the sites were supposed to open up their digitized archive to the public after an exclusivity period of 3 - 5 years. "In practice, this simply hasn't happened," Reclaim the Records said in a blog post announcing the FOIA. "NARA has never actually posted online the vast majority of these records that were digitized through their partnership program, not to their Catalog nor indeed anywhere else where the public might be able to freely access and download the now-digital records. This remains the case today, even when the embargo periods for many of these record sets have been expired for more than a decade, sometimes two decades." Most of these are stored behind Ancestry.com's paywall, in part because Ancestry purchased several of the other sites that NARA had made deals with when they were still independent.
According to the agreements, the sites were supposed to open up their digitized archive to the public after an exclusivity period of 3 - 5 years. "In practice, this simply hasn't happened," Reclaim the Records said in a blog post announcing the FOIA. "NARA has never actually posted online the vast majority of these records that were digitized through their partnership program, not to their Catalog nor indeed anywhere else where the public might be able to freely access and download the now-digital records. This remains the case today, even when the embargo periods for many of these record sets have been expired for more than a decade, sometimes two decades." Most of these are stored behind Ancestry.com's paywall, in part because Ancestry purchased several of the other sites that NARA had made deals with when they were still independent.
About time (Score:5, Interesting)
As an Ancestry subscriber, I have access to a lot of this data, but it is a disgrace that it isn't freely available to all. Some is, of course, but a lot isn't. Time to open the floodgates and let the data be available to all.
Re: (Score:2)
Well, giving them 5 years of exclusivity in exchange for digitization and Internet access is reasonable, but shouldn't go on forever, especially past their agreement.
Re: About time (Score:2)
As I understand, FOIA requests typically involve a 'per page' fee for the results of the application. Not sure why they are going this route for 'billions' of pages of information.
Wouldn't it be easier to bend the ear of a few friendly politicians to force the government records agency to simply force the private third-parties to offer up the data, as they comitted to?
I suspect the issue is that the gov't isn't able to pay for the required infrastructure needed to host the 'billions' of pages of data.
Re: (Score:2)
Re: (Score:2)
Are there any security issues? My bank asked my what my mother's maiden name was as a security question...
It would be nice (Score:2)
Comment removed (Score:3)
Re: (Score:3)
Re: (Score:3)
Are you willing to pay for the cost of it up front? By that, I mean buy the scanners, staff them, and perform QA?
Does anyone ask the same question about the fucking ridiculous military budget that would cover all that work in about half a day?
Yes. (Score:2)
Yes.
Yes, any time the cost of ANY government program, actual or proposed, is discussed you an bunch of other people change the subject to defense spending.
Do you have any comments related to the topic at hand?
Re: (Score:2)
Yes.
Yes, any time the cost of ANY government program, actual or proposed, is discussed you an bunch of other people change the subject to defense spending.
Do you have any comments related to the topic at hand?
I thought tax money spend for no return was the subject.
Re: FOIA is a nice start. (Score:2)
Yes, they do, and the senators and congressmen approve the military budgets without fail.
Digitizing 100 year-old documents so you don't have to join ancestry.Com isn't a good use of federal tax dollars.
Re: (Score:2)
Yes, they do, and the senators and congressmen approve the military budgets without fail.
Digitizing 100 year-old documents so you don't have to join ancestry.Com isn't a good use of federal tax dollars.
Whereas having an armed force able to take on the 10 next largest forces is? Yeah. Always good to prepare for those fantasy wars that are never going to happen.
Re: (Score:2)
The thing is, the embargo policy failed. The target was free release of all of the data in digital form. The price was allow it to be embargoed for 5 years. The embargo part was paid, but the release never happened.
The deal wasn't a bad idea, but now it needs to be enforced.
Re: (Score:2)
Are you willing to pay for the cost of it up front? By that, I mean buy the scanners, staff them, and perform QA?
No. But I also didn't sign an agreement that I would, and then five years later release the data for public access.
Ancestry and others *are* willing to scan it for "free."
I like how you put "free" in quotes, so that you can pretend you aren't lying when you entirely ignore the consideration that Ancestry received in return for this work, that's given them a de facto monopoly from which they've expanded their business.
Still, if (as you insinuate) the data has no value then Ancestry will have no objections at all to handing it over.
Ancestry's complete document library is free (Score:4, Informative)
Ancestry provides public libraries free access to their entire historic library. You can probably go to your local library today and search their database.
(Their DNA databases are obviously not availiable, nor are their user uploaded information.)
Re:Ancestry's complete document library is free (Score:5, Insightful)
Yeah, but if you read the story, it would probably be against the law to charge the government for it, because the government already paid for it. And they already paid for me to have it for "free," too, but I don't.
Re: (Score:2)
But the point of the article is the insinuation that all this data is going to be locked away behind paywalls. Which it isn't. It's all going to be freely available to everyone with a library card.
DNA databases (Score:4, Informative)
Ancestry's DNA databases seem to available to the police without any problem...
Re: (Score:2)
That statement is amazingly wrong on several levels.
Re: (Score:2)
https://www.ancestry.com/cs/le... [ancestry.com]
Re: (Score:2)
You didn't even look at the link that you attached, did you?
"Ancestry does not voluntarily cooperate with law enforcement. To provide our Users with the greatest protection under the law, we require all government agencies seeking access to Ancestry customers’ data to follow valid legal process and do not allow law enforcement to use Ancestry’s services to investigate crimes or to identify human remains."
What makes the comment "wrong on several levels" is that 1) Ancestry does not voluntarily co
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Billions of old documents. Perhaps 100K each. Less than a Petabyte. The legal costs are way above the hardware costs.
Actually, a lot of it *is* available (Score:5, Informative)
And NARA even built a nice freely available API: https://github.com/usnationala... [github.com]
A few problems though: a) the documentation isn't great; b) though useful to the Slashdot crowd, the API doesn't provide great access to the (non-technical) folks submitting this FOIA; and c) it literally provides access to mostly unstructured / flat content; the images themselves, plus a little bit of metadata.
the contracts don't say that (Score:2)
Re: (Score:1)