NSA Makes Contribution To Apache Hadoop Project 102
An anonymous reader writes "The National Security Agency has submitted a new database, Accumulo, to the Apache Foundation for incubation. Accumulo is based on the original BigTable paper with some extensions such as the ability to provide cell-level security. It appears there are some hurdles that must be cleared concerning copyright before the project could be accepted."
It's a trap (Score:2, Insightful)
It's a trap! It HAS to be. /tinfoil
Please trust the NSA. Pretty please. (Score:2, Funny)
It's a trap! It HAS to be. /tinfoil
No, no, it's not a trap, not in the slightest. Just insert your penis into this device... I assure you, it's not a meat-grinder, really, it's not! And I didn't have my fingers crossed when I said that, not even a little bit.
Re: (Score:1)
Helpful tip: Having your fingers crossed while saying something only means what you think it means (that you're lying) in Swedish, not in English. In English, it means you're hoping for a particular outcome, which could be true in this case too, I guess.
Re: (Score:2, Funny)
Re: (Score:1)
Depends on where your fingers are. Not sure where it comes from, but when I was growing up (in England, home of English), crossing your fingers behind your back meant that you were lying.
That's also the case in America, at least in the places I grew up in - and we moved around a lot when I was a kid.
Re: (Score:1)
Re: (Score:1)
Helpful tip: Having your fingers crossed while saying something only means what you think it means (that you're lying) in Swedish, not in English. In English, it means you're hoping for a particular outcome, which could be true in this case too, I guess.
Well, OK, whatever, so what does meat-grinder mean in Swedish? Slang for some body orifice, I'm guessing from context.
Re: (Score:2, Funny)
Well, OK, whatever, so what does meat-grinder mean in Swedish? Slang for some body orifice, I'm guessing from context.
No. It means a meat-grinder, or köttkvarn - the mechanical device which turns lumps of meat into ground meat or minced meat.
Pro tip: don't stick your dick into one of these.
Re: (Score:1)
So it seems from the other replies that "crossing your fingers" in that way is used by at least some English speakers as well. Neat. Didn't know that, primarily because Swedish English education teaches that it is a false friend since "crossing your fingers" is already something else in English.
With this in mind, I'm not surprised that AliasMarlowe is Swedish per the above because I've never heard a native English speaker use it in that context.
Re: (Score:1)
A lot of words and gestures in the US and likely other English speaking nations carry duel meanings. This is probably because of the expansions of the English speaking countries into other nations and territories as well as a one time liberal immigration policy over the years.
Take shag for instance, in some uses, it means sex, in others it describes the look and feel of something like shag carpet which is a thick loose pile of thread instead of a rug someone had sex on/with. That would be a shag on the carp
Re: (Score:2)
I just wanted to step in and express my gratitude for how much I have learned in this thread. I now know where not to stick body parts, what crossing my fingers means, and what a meatgrinder does.
Re: (Score:2)
Huh. I always assumed the etymology was related... As in, a "shag carpet" being thicker and softer than most floor surfaces, it must've seemed like a clever place to practice the various marital arts for quite a few couples....
Re: (Score:2)
A lot of words and gestures in the US and likely other English speaking nations carry duel meanings.
Quite right! Pistols at dawn, then?
Re: (Score:2)
> like shag carpet which is a thick loose pile of thread instead of a rug someone had sex on/with
I dispute the accuracy of that claim, there is no reason that your "instead of" could not be replaced with "as well as". In fact, considering how soft and comfortable shag carpets are compared to other carpets the odds that somebody already shagged on it is much higher than other carpets.
Then again not so long ago shag was a popular brand of particularly strong pipe tobacco, Sherlock Holmes had an affinity fo
Re: (Score:1)
So it seems from the other replies that "crossing your fingers" in that way is used by at least some English speakers as well. Neat. Didn't know that, primarily because Swedish English education teaches that it is a false friend since "crossing your fingers" is already something else in English.
With this in mind, I'm not surprised that AliasMarlowe is Swedish per the above because I've never heard a native English speaker use it in that context.
Um, not exactly, but Google Translate [google.com] was helpful. I spent a few months in Sweden back in the 80's, and have made shorter visits since then, but never picked up much of the language. I'm a native English speaker and have spent decades in various places on both sides of the Atlantic, so consider myself fluent in British and American English and familiar with many local variants (Ontario, BC, Alabama, Florida, Maine, Wisconsin, as well as several regions of the British Isles).
On the "fingers crossed" phras
Re: (Score:2)
Serves me right then, but assuming someone that knows the correct spelling and meaning of "köttkvarn" to be Swedish is generally a low-risk bet.
Re: (Score:2)
Heh? They're perfectly safe. Just don't turn the handle.
Re: (Score:2)
Behind the back it does, but it doesn't exclusively mean that in general.
Re:Please trust the NSA. Pretty please. (Score:4, Informative)
I agree with everyone else who says you're absolutely wrong. In common (American at least) English usage if you say something like "Here's hoping!" or "Did you get the part? I hope so!" and cross your fingers it means you're hoping for an outcome.
If you have your fingers crossed for another type of statement (typically obscuring them), it means you're lying. Typically children's usage.
Re: (Score:1)
I grew up with both.
Re: (Score:2)
In America it also means you are lying. At least in some places.
Re: (Score:2)
Re: (Score:1)
Unless you know something that I don't, the SELinux code has been reviewed several times looking for backdoors and there aren't any. Or is it just that the config is a PITA for you? That I can understand.
Re: (Score:2)
Well, if he is referring to SELinux, he is wrong. If he is referring to many other technologies that the NSA has subverted either thru cooperation with the company or by covertly infiltrating their engineering into the company... then he is correct.
If you run such a company making security products and you strangely have a few workers who seem to be very familiar with the Baltimore metro area but nothing on their resume shows that they worked or went to school there, you may want to take a closer look at t
Re: (Score:2)
Audit the code yourself. It's all there for one to obsess over.
Re: (Score:1)
Huh. That's disturbing. And you've seen this before? Like actual planted employees whose job is to slip in backdoors and exploitable code? And management and the assorted PHBs are what...in on this? Unaware? And the code doesn't get found in reviews? It seems like it would make more sense for the NSA to make dummy corps to develop and sell security software than it makes to plant people and then expect them not to be detected.
I always find conspiracies hard to believe. They require two things humans are his
Re: (Score:2)
The fact that this is happening is well known.
One such example: http://en.wikipedia.org/wiki/Crypto_AG#Back-doored_machines [wikipedia.org]
But I guess that might be too hard for you the believe.
Re: (Score:1)
I had heard about collusion on that level, it was the planted employees I hadn't heard of. I didn't mean to imply that I thought you were lying or whatever other offense your tone implies you believe I meant.
Why an Apache donation (Score:1)
Re:Why an Apache donation (Score:5, Informative)
Apache Brand
Our interest in releasing this code as an Apache incubator project is due to its strong relationship with other Apache projects, i.e. Accumulo has dependencies on Hadoop, Zookeeper, and Thrift and has complementary goals to HBase.
Re: (Score:2)
Protip, a license that restricts what the coder and devs using the code can do, is not really "free". It may protect the end users freedoms but it inarguably does so at the expense of developer freedoms.
When he says GPL is not truly free, he means it, and I dont think anyone involved with the development of GPLv2 and GPLv3 would argue that.
Re: (Score:1)
Works of the US government are public domain, and thus can't be released under the GPL. That's the copyright issue mentioned in the summary.
(I know people here don't read the articles, but don't they at least read the summaries?)
Re: (Score:2)
Re: (Score:2)
the NSA is not a corporation (Score:2)
i think you might enjoy the book "Shadow Factory" by James Bamford,
or maybe you might like the PBS Frontline special about his book, available online at pbs.org (the video is called Spy Factory for some reason)
Re:government never has copyright (Score:5, Informative)
But other companies and individuals that produce works do get copyright. While they may give the government (and even the NSA) a license to use their works, the government can't just donate those works off to Apache without clearing it first. That means any code the NSA didn't write themselves needs to be removed, replaced, or also donated by the owner.
Re: (Score:1)
That means any code the NSA didn't write themselves needs to be removed, replaced, or also donated by the owner.
Unless that code was "work for hire". If so the contractor (individual or company) has no rights to it, just like any other employee.
Re: (Score:2)
I'm not sure i'd want to go anywhere near the work of an NSA "contractor..."
and thus can't grant an explicit license (Score:2)
NSA tries to get vendors serious about security (Score:5, Informative)
NSA has been trying for decades to get vendors to get serious about security, without much success. One of NSA's units is the Central Security Service, the defensive side, which develops and tests security technologies for Government and military use. They have people testing safes and locks, for example.
Back in the 1980s, NSA tried applying that approach to computing, with the Trusted Computer System Evaluation Criteria. [wikipedia.org] Systems were classified from A1 down to D. A very few specialized systems made it to an A level, but most commercial systems couldn't come close.
Manufacturers hated the testing procedure. Software vendors are used to controlling their own Q/A process. The NSA approach came from the test procedures for safes and padlocks - vendors could submit something, and it was tested by NSA personnel against NSA criteria. If it failed, the manufacturer got a list of defects, which was not necessarily complete. The manufacturer could resubmit the product, and NSA would retest it, on a strictly pass/fail basis. No third try was allowed, and failure was publicly announced by NSA.
After a decade of screaming and foot-dragging by vendors, the "common criteria" [wikipedia.org] security scheme replaced the TCSEC in 2002-2005. This is much more "vendor friendly". The most strict levels of the TCSEC criteria were removed. Security evaluation is mostly done by outside labs, not NSA, and the vendor pays for and controls the process. The vendor can keep trying to pass as many times as they want. Failure is not publicized.
A reasonable number of systems meet some levels of the common criteria, but nothing below EAL5 really means much. Windows XP made it to EAL4.
NSA has tried, with NSA Secure Linux, to get people to take mandatory security seriously. NSA Secure Linux has "mandatory security", where there are levels and compartments which create boundaries data is not allowed to cross. Think of everything being in its own sandbox, with limited and tightly controlled intercommunication between sandboxes.
The point of that is not that NSA Secure Linux is a highly secure implementation of mandatory security. It was to get people to implement, modify, and partition applications so that they could work under a mandatory security model. A web browser, for example, would have to be structured so that the parts which could open local files were completely separated from the parts that communicated with the untrusted outside world. This didn't catch on in the browser world, although finally, a decade or so too late, browsers are starting to to run Flash in sandboxes.
NSA keeps trying. This new database is one for which fine-grained access control is possible. The challenge is to write apps that can live with such tight controls. They're trying to get people to get serious about security.
(It's been a long time, but I used to work on this stuff.)
Re: (Score:2)
Most competent sysadmins try to do their best to secure their system, and those worth their salt, succeed to do so. SELinux (and Tomoyo) are painful to use, easy to lock yourself out, and cumbersome. But that's the price to pay, I guess. Some admins decide the price is too high.
Re:NSA tries to get vendors serious about security (Score:4, Funny)
Most competent sysadmins try to do their best to secure their system, and those worth their salt, succeed to do so.
So, um. What does that make the kernel.org guys? ;)
Yeah, I thought so.
Re: (Score:2)
Yeah that's the other thing too. SELinux doesn't "protect" you against attacks more than mosquito repellant doesn't protect you against mosquito bites... there's always going to be a way. But the more precautions you take, less chances of getting hacked you have.
Or in more technical terms, SELinux doesn't protect you from a malicious user hacking into your system more than giving him a regular user account instead of root access. There are exploits to gain root access, and I guess SELinux can be exploited t
Re: (Score:2)
Indeed. Google for 'selinux 2.6.30 exploit' and you'll find one by 'cheddar bay' that's targeted explicitly at SELinux.
Re: (Score:2)
In that article, it specifically says that NSA require covert channel monitoring.
does this explain the Thomas Drake case? (Score:4, Interesting)
You are describing software testing in the 1990s. Thomas Drake was heavily involved in software testing, and worked for NSA contractors until 2001, when he was hired at NSA itself.
After 9/11, he got disturbed with some of their wasteful practices . . . I am wondering if 'vendor friendly' software testing was one of the practices he might have had a problem with.
The DoD IG report on Trailblazer is still mostly redacted... the public is left in the dark about these things.
Re: (Score:1)
Re: (Score:2)
NSA has been trying for decades to get vendors to get serious about security, without much success
Car analogy. Most vendors can barely get their cars to run, so preventing the cars from getting broken into and/or stolen is not the top priority.
It only becomes a priority in places where legislation requires the vendors to worry about it.
Before donning tinfoil hats... (Score:4, Interesting)
Re: (Score:1)
The NSA's lack of street cred is based on a smear job by the NY Times. What the Times artfully hid is that the agency complied with federal law in the wiretapping. They exploited a loophole in the law to support CALEA efforts. However, what they did was legal, and how the agency did it was legal. The folks requesting the information from the agency broke several laws, but attacking the NSA is a lot safer than attacking federal law enforcement agencies who broke the laws.
The NSA has two huge problems right n
Re: (Score:2)
Most of these hurdles and testing are performed by so called "Information Assurance Engineers" who could not hack it as a sys admin. Most of what they do consists of installing from an approved kick-start media, running various scripts to configure and test the machine.
Dispel yourself of the notion of some super security guru setting up this machine. It is some drone following a checklist he/she doesn't really understand using media given to them, running the STIG scripts, running their verification scrip
If our overlords are to record everything we do... (Score:2)
...the least our overlords can do is pitch in on building the databases our overlords are going to store all that crap they recorded about us.
The myth of security... (Score:2)
... the best security programmed in software can and will be breached by other means. This emphasis on security IMHO is misplaced, if you want something secure you don't hook it up to the outside world.
Re:The myth of security... (Score:4, Insightful)
When i gain employment at your company and you are not looking, the outside world is effectively inside the company.
Almost everywhere one of these databases will be used will have employees accessing the systems (remember manning?) and there may be a complete need to access the information remotely which even if the internet isn't involved (T1 loop or something) you have the potential of unauthorized access.
You simply cannot focus on one side of the equation. This focus is for where the other sides can't be effective either.
Re:The myth of security... (Score:5, Insightful)
... the best security programmed in software can and will be breached by other means. This emphasis on security IMHO is misplaced, if you want something secure you don't hook it up to the outside world.
I know my front door can easily be breached by a determined attacker, yet I put a lock on it. Why bother? Insurance requires it for starters. It deters casual thieves for another. Abandoning security altogether is just as stupid as making what you're trying to secure unusable by over securing it. A bit of balance goes a long way.
Re: (Score:2)
>I know my front door can easily be breached by a determined attacker, yet I put a lock on it
Of course, the world is full of towns and cities where people do NOT put locks on, or bother to shut the locks that came with the door.
A lock can reduce casual theft, reducing the casual thieves work better.
This is no less true of cybersecurity. As long as most cybercriminals get away with it most of the time - we won't see a reduction in exploits.
Re: (Score:2)
A lock can reduce casual theft, reducing the casual thieves work better.
This is no less true of cybersecurity. As long as most cybercriminals get away with it most of the time - we won't see a reduction in exploits.
Yeah I agree but good luck with that. Dark side of human nature means if you put lots of people in a small space, statistically there are going to be a few rotten apples.
Re: (Score:2)
Of course. Security must come in layers, and requires a holistic approach.
But tight computer security can also help keeping the human factor in check. By making sure no unauthorised persons can access the data for starters, particularly related to breaches from the outside. And then making sure there is an audit log of all accesses made to the system, particularly who accessed which piece of sensitive data when. So in case there is a security breach, that there is a way to trace back and know who did it. K
Copyright issues (Score:4, Informative)
We're going to see more of this sort of thing. Almost everyone assumes that all software is copyrighted, or that only the copyright holder can release software as free/libre/open source software (FLOSS). Neither are true!! This matters when the US government gets involved, because its "normal" rules are really different from most organization's.
For example, if a government employee develops software as part of his official duties, then in practically all cases that software is NOT subject to copyright in the US (per US law 17 USC 105). It's not just that the author doesn't have copyright; there IS no copyright in the US. Also, when a contractor writes software, the government often receives all the release rights as if it was the copyright holder yet it is not the copyright holder (these are called "unlimited rights"). In this case, the government can release the software as FLOSS, on its own initiative, even though it is NOT the copyright holder. For more details, see: Publicly Releasing Open Source Software Developed for the U.S. Government [thedacs.com].
The US government spends billions of dollars each year developing software. It's my hope that, over time, it will release more of the software it develops to the people who paid for it.
Re: (Score:2)
So if I understand this correctly, and to say it simple: any software (and for that matter any creative work) created by the US government automatically falls in the public domain?
Is there an OSI license that can work? (Score:2)
That's an interesting problem - most open source licenses depend on copyright for enforcement. If there is no copyright, those licenses can't be used. Is there a way to incorporate?
Interesting modification (Score:2)
It seems that the extra cell-level security is more of a capability, in that you can categorize (or add a label to) a cell and when you query you specify the access level you have...and the result is included or not depending.
I wonder how it deals with "lost security levels?" If you don't know the security level of a cell, you can't ask for it. If everyone forgets, then the data just sits around, waiting to be pruned. How can you tell the difference between a resource leak and unarchived classified document
Re: (Score:2)
You do realize that for an organization like the NSA to trust anybody, even their own employees, would be exceedingly foolish.
As far as I know, the people of the United States have had the right to bear arms for over two centuries. However if you think this means an American can go out on a whim and buy a heavy machine gun then you are mistaken.
Not sure why all of a sudden people are locking their doors in New Zealand, but I would suspect it has more to do with an uptick in local crime than American foreig
Mixing tenses... which is it? (Score:1)
It appears there are some hurdles that must be cleared concerning copyright before the project could be accepted.
Wait... what? I'm not trying to be a grammar nazi, I'm genuinely confused. ARE some hurdles that must be CLEARED (future tense)... COULD be accepted (past test)? Which is it-- do the hurdles still need to be cleared before the project can be accepted, or have the hurdles been cleared and the project accepted?
Re:OK, now try it in English (Score:4, Informative)
You're either trolling terribly or just terribly ignorant. In the hopes of the latter:
The Apache Foundation maintains many open-source software projects, one of which is a popular web server. Another is Hadoop, which is a distributed file system for storing huge amounts of data on a cluster of individual computers, based on Google's Google File System and other similar technologies.. To facilitate access to that data, there are other projects that function as databases, with the actual information stored in Hadoop. One existing project is HBase, which is an implementation of a system (called BigTable) described by Google. Now, the NSA has donated the source code for their own such database (also based on BigTable) to the Apache Foundation.
Now, there are a lot of Apache Foundation projects, and never enough time or people to maintain them all completely. The best projects are considered "mature", and the ones that aren't up to the normal Apache levels of quality and support and considered to be in "incubation". Someday, if enough people like Accumulo and help with it, it will mature.
Re: (Score:1)
If you're an outsider, why do you care about an article that essentially only matters to insiders? And while we're explaining the intricacies of the software industry, I will take the opportunity to introduce you to this wonderful invention. It's called a search engine. When you don't know what something means, you can search for it yourself, therefore avoiding looking both ignorant and lazy.
By the by, this is /. Notice the subtitle: "News for nerds". I think you may be lost. You may feel more comfortable h
Re:OK, now try it in English (Score:4, Insightful)
Re: (Score:2)
Re:OK, now try it in English (Score:4, Funny)
How do you "submit" a databse?
It turns out that if you read sentences all the way to the end, they become a lot more clear.