Forgot your password?
typodupeerror
Cloud Government Security Apache

NSA Makes Contribution To Apache Hadoop Project 102

Posted by samzenpus
from the need-a-hand? dept.
An anonymous reader writes "The National Security Agency has submitted a new database, Accumulo, to the Apache Foundation for incubation. Accumulo is based on the original BigTable paper with some extensions such as the ability to provide cell-level security. It appears there are some hurdles that must be cleared concerning copyright before the project could be accepted."
This discussion has been archived. No new comments can be posted.

NSA Makes Contribution To Apache Hadoop Project

Comments Filter:
  • It's a trap (Score:2, Insightful)

    by hjf (703092)

    It's a trap! It HAS to be. /tinfoil

    • It's a trap! It HAS to be. /tinfoil

      No, no, it's not a trap, not in the slightest. Just insert your penis into this device... I assure you, it's not a meat-grinder, really, it's not! And I didn't have my fingers crossed when I said that, not even a little bit.

      • by wootest (694923)

        Helpful tip: Having your fingers crossed while saying something only means what you think it means (that you're lying) in Swedish, not in English. In English, it means you're hoping for a particular outcome, which could be true in this case too, I guess.

        • Re: (Score:2, Funny)

          by TheRaven64 (641858)
          Depends on where your fingers are. Not sure where it comes from, but when I was growing up (in England, home of English), crossing your fingers behind your back meant that you were lying.
          • Depends on where your fingers are. Not sure where it comes from, but when I was growing up (in England, home of English), crossing your fingers behind your back meant that you were lying.

            That's also the case in America, at least in the places I grew up in - and we moved around a lot when I was a kid.

          • I'm English too and you're right only in as much that hiding ones crossed fingers (can be behind ones back or in ones pocket) is a sign of subterfuge (something intended to misrepresent the true nature of an activity). Showing them openly to be crossed is not a sign of lying or subterfuge and, as per wootest comment, is merely signifying a hope for good luck in pursuit of a particular outcome.
        • by vlm (69642)

          Helpful tip: Having your fingers crossed while saying something only means what you think it means (that you're lying) in Swedish, not in English. In English, it means you're hoping for a particular outcome, which could be true in this case too, I guess.

          Well, OK, whatever, so what does meat-grinder mean in Swedish? Slang for some body orifice, I'm guessing from context.

          • Re: (Score:2, Funny)

            by AliasMarlowe (1042386)

            Well, OK, whatever, so what does meat-grinder mean in Swedish? Slang for some body orifice, I'm guessing from context.

            No. It means a meat-grinder, or köttkvarn - the mechanical device which turns lumps of meat into ground meat or minced meat.
            Pro tip: don't stick your dick into one of these.

            • by wootest (694923)

              So it seems from the other replies that "crossing your fingers" in that way is used by at least some English speakers as well. Neat. Didn't know that, primarily because Swedish English education teaches that it is a false friend since "crossing your fingers" is already something else in English.

              With this in mind, I'm not surprised that AliasMarlowe is Swedish per the above because I've never heard a native English speaker use it in that context.

              • by sumdumass (711423)

                A lot of words and gestures in the US and likely other English speaking nations carry duel meanings. This is probably because of the expansions of the English speaking countries into other nations and territories as well as a one time liberal immigration policy over the years.

                Take shag for instance, in some uses, it means sex, in others it describes the look and feel of something like shag carpet which is a thick loose pile of thread instead of a rug someone had sex on/with. That would be a shag on the carp

                • I just wanted to step in and express my gratitude for how much I have learned in this thread. I now know where not to stick body parts, what crossing my fingers means, and what a meatgrinder does.

                • Huh. I always assumed the etymology was related... As in, a "shag carpet" being thicker and softer than most floor surfaces, it must've seemed like a clever place to practice the various marital arts for quite a few couples....

                • A lot of words and gestures in the US and likely other English speaking nations carry duel meanings.

                  Quite right! Pistols at dawn, then?

                • > like shag carpet which is a thick loose pile of thread instead of a rug someone had sex on/with

                  I dispute the accuracy of that claim, there is no reason that your "instead of" could not be replaced with "as well as". In fact, considering how soft and comfortable shag carpets are compared to other carpets the odds that somebody already shagged on it is much higher than other carpets.
                  Then again not so long ago shag was a popular brand of particularly strong pipe tobacco, Sherlock Holmes had an affinity fo

              • So it seems from the other replies that "crossing your fingers" in that way is used by at least some English speakers as well. Neat. Didn't know that, primarily because Swedish English education teaches that it is a false friend since "crossing your fingers" is already something else in English.

                With this in mind, I'm not surprised that AliasMarlowe is Swedish per the above because I've never heard a native English speaker use it in that context.

                Um, not exactly, but Google Translate [google.com] was helpful. I spent a few months in Sweden back in the 80's, and have made shorter visits since then, but never picked up much of the language. I'm a native English speaker and have spent decades in various places on both sides of the Atlantic, so consider myself fluent in British and American English and familiar with many local variants (Ontario, BC, Alabama, Florida, Maine, Wisconsin, as well as several regions of the British Isles).

                On the "fingers crossed" phras

                • by wootest (694923)

                  Serves me right then, but assuming someone that knows the correct spelling and meaning of "köttkvarn" to be Swedish is generally a low-risk bet.

                  If fingers are crossed in plain view, then it has the connotation of hopeful intent. If they are crossed while concealed - such as behind one's back or under a table - then the implication is that one is lying. The assertion that one's fingers were not crossed would be necessary only if one or both hands were not in plain view, so the association would be an un

            • Heh? They're perfectly safe. Just don't turn the handle.

        • by Moridineas (213502) on Sunday September 04, 2011 @04:02PM (#37303890) Journal

          I agree with everyone else who says you're absolutely wrong. In common (American at least) English usage if you say something like "Here's hoping!" or "Did you get the part? I hope so!" and cross your fingers it means you're hoping for an outcome.

          If you have your fingers crossed for another type of statement (typically obscuring them), it means you're lying. Typically children's usage.

        • I grew up with both.

        • by gatkinso (15975)

          In America it also means you are lying. At least in some places.

      • Finally, the NSA's secret plan to eliminate Julian Assange is revealed!
  • I'm a heavy user of open source and GPL software, but I admit to not knowing the nuances of open source licenses. My question is this: why are corporations donating their code to Apache instead of just releasing them through the GPL and Sourceforge? Oracle recently did this as well with OpenOffice, and I seem to vaguely recall a few others.
    • by Fnord666 (889225) on Sunday September 04, 2011 @03:41PM (#37303784) Journal
      According to TFA:

      Apache Brand

      Our interest in releasing this code as an Apache incubator project is due to its strong relationship with other Apache projects, i.e. Accumulo has dependencies on Hadoop, Zookeeper, and Thrift and has complementary goals to HBase.

    • by joe_kull (238178)

      Works of the US government are public domain, and thus can't be released under the GPL. That's the copyright issue mentioned in the summary.

      (I know people here don't read the articles, but don't they at least read the summaries?)

      • by arose (644256)
        Public domain is perfectly GPL compatible, where did you get the idea that it wasn't?
    • i think you might enjoy the book "Shadow Factory" by James Bamford,
      or maybe you might like the PBS Frontline special about his book, available online at pbs.org (the video is called Spy Factory for some reason)

  • by Animats (122034) on Sunday September 04, 2011 @04:54PM (#37304192) Homepage

    NSA has been trying for decades to get vendors to get serious about security, without much success. One of NSA's units is the Central Security Service, the defensive side, which develops and tests security technologies for Government and military use. They have people testing safes and locks, for example.

    Back in the 1980s, NSA tried applying that approach to computing, with the Trusted Computer System Evaluation Criteria. [wikipedia.org] Systems were classified from A1 down to D. A very few specialized systems made it to an A level, but most commercial systems couldn't come close.

    Manufacturers hated the testing procedure. Software vendors are used to controlling their own Q/A process. The NSA approach came from the test procedures for safes and padlocks - vendors could submit something, and it was tested by NSA personnel against NSA criteria. If it failed, the manufacturer got a list of defects, which was not necessarily complete. The manufacturer could resubmit the product, and NSA would retest it, on a strictly pass/fail basis. No third try was allowed, and failure was publicly announced by NSA.

    After a decade of screaming and foot-dragging by vendors, the "common criteria" [wikipedia.org] security scheme replaced the TCSEC in 2002-2005. This is much more "vendor friendly". The most strict levels of the TCSEC criteria were removed. Security evaluation is mostly done by outside labs, not NSA, and the vendor pays for and controls the process. The vendor can keep trying to pass as many times as they want. Failure is not publicized.

    A reasonable number of systems meet some levels of the common criteria, but nothing below EAL5 really means much. Windows XP made it to EAL4.

    NSA has tried, with NSA Secure Linux, to get people to take mandatory security seriously. NSA Secure Linux has "mandatory security", where there are levels and compartments which create boundaries data is not allowed to cross. Think of everything being in its own sandbox, with limited and tightly controlled intercommunication between sandboxes.

    The point of that is not that NSA Secure Linux is a highly secure implementation of mandatory security. It was to get people to implement, modify, and partition applications so that they could work under a mandatory security model. A web browser, for example, would have to be structured so that the parts which could open local files were completely separated from the parts that communicated with the untrusted outside world. This didn't catch on in the browser world, although finally, a decade or so too late, browsers are starting to to run Flash in sandboxes.

    NSA keeps trying. This new database is one for which fine-grained access control is possible. The challenge is to write apps that can live with such tight controls. They're trying to get people to get serious about security.

    (It's been a long time, but I used to work on this stuff.)

    • by hjf (703092)

      Most competent sysadmins try to do their best to secure their system, and those worth their salt, succeed to do so. SELinux (and Tomoyo) are painful to use, easy to lock yourself out, and cumbersome. But that's the price to pay, I guess. Some admins decide the price is too high.

      • by lennier (44736) on Sunday September 04, 2011 @06:07PM (#37304456) Homepage

        Most competent sysadmins try to do their best to secure their system, and those worth their salt, succeed to do so.

        So, um. What does that make the kernel.org guys? ;)

        Yeah, I thought so.

        • by hjf (703092)

          Yeah that's the other thing too. SELinux doesn't "protect" you against attacks more than mosquito repellant doesn't protect you against mosquito bites... there's always going to be a way. But the more precautions you take, less chances of getting hacked you have.

          Or in more technical terms, SELinux doesn't protect you from a malicious user hacking into your system more than giving him a regular user account instead of root access. There are exploits to gain root access, and I guess SELinux can be exploited t

          • by Errtu76 (776778)

            There are exploits to gain root access, and I guess SELinux can be exploited too.

            Indeed. Google for 'selinux 2.6.30 exploit' and you'll find one by 'cheddar bay' that's targeted explicitly at SELinux.

    • by decora (1710862) on Sunday September 04, 2011 @08:35PM (#37305140) Journal

      You are describing software testing in the 1990s. Thomas Drake was heavily involved in software testing, and worked for NSA contractors until 2001, when he was hired at NSA itself.

      After 9/11, he got disturbed with some of their wasteful practices . . . I am wondering if 'vendor friendly' software testing was one of the practices he might have had a problem with.

      The DoD IG report on Trailblazer is still mostly redacted... the public is left in the dark about these things.

      • ThinThread-LITTLE is listening... Seriously, the ThinThread project was what could have stopped 9/11 but instead the elites wanted to spend mega-bucks on TrailBlazer; you know the dance step. It is rumored that ThinThread code was adapted for the current system.
    • by TheLink (130905)

      NSA has been trying for decades to get vendors to get serious about security, without much success

      Car analogy. Most vendors can barely get their cars to run, so preventing the cars from getting broken into and/or stolen is not the top priority.

      It only becomes a priority in places where legislation requires the vendors to worry about it.

  • by hattable (981637) on Sunday September 04, 2011 @05:47PM (#37304384) Journal
    I know NSA doesn't have the best 'street-cred' but remember that they are the folks that brought up SELinux. When they are working for security they generally know what they are talking about. Has anyone had any experience installing software on a NSA machine? If you have then you know the hurdles and testing that takes place to get something usable. They LOVE security and really just want you to love it as much as they do.
    • by Anonymous Coward

      The NSA's lack of street cred is based on a smear job by the NY Times. What the Times artfully hid is that the agency complied with federal law in the wiretapping. They exploited a loophole in the law to support CALEA efforts. However, what they did was legal, and how the agency did it was legal. The folks requesting the information from the agency broke several laws, but attacking the NSA is a lot safer than attacking federal law enforcement agencies who broke the laws.

      The NSA has two huge problems right n

    • by gatkinso (15975)

      Most of these hurdles and testing are performed by so called "Information Assurance Engineers" who could not hack it as a sys admin. Most of what they do consists of installing from an approved kick-start media, running various scripts to configure and test the machine.

      Dispel yourself of the notion of some super security guru setting up this machine. It is some drone following a checklist he/she doesn't really understand using media given to them, running the STIG scripts, running their verification scrip

  • ...the least our overlords can do is pitch in on building the databases our overlords are going to store all that crap they recorded about us.

  • ... the best security programmed in software can and will be breached by other means. This emphasis on security IMHO is misplaced, if you want something secure you don't hook it up to the outside world.

    • by sumdumass (711423) on Sunday September 04, 2011 @07:57PM (#37304884) Journal

      When i gain employment at your company and you are not looking, the outside world is effectively inside the company.

      Almost everywhere one of these databases will be used will have employees accessing the systems (remember manning?) and there may be a complete need to access the information remotely which even if the internet isn't involved (T1 loop or something) you have the potential of unauthorized access.

      You simply cannot focus on one side of the equation. This focus is for where the other sides can't be effective either.

    • by syousef (465911) on Sunday September 04, 2011 @08:18PM (#37305002) Journal

      ... the best security programmed in software can and will be breached by other means. This emphasis on security IMHO is misplaced, if you want something secure you don't hook it up to the outside world.

      I know my front door can easily be breached by a determined attacker, yet I put a lock on it. Why bother? Insurance requires it for starters. It deters casual thieves for another. Abandoning security altogether is just as stupid as making what you're trying to secure unusable by over securing it. A bit of balance goes a long way.

      • >I know my front door can easily be breached by a determined attacker, yet I put a lock on it

        Of course, the world is full of towns and cities where people do NOT put locks on, or bother to shut the locks that came with the door.
        A lock can reduce casual theft, reducing the casual thieves work better.

        This is no less true of cybersecurity. As long as most cybercriminals get away with it most of the time - we won't see a reduction in exploits.

        • by syousef (465911)

          A lock can reduce casual theft, reducing the casual thieves work better.

          This is no less true of cybersecurity. As long as most cybercriminals get away with it most of the time - we won't see a reduction in exploits.

          Yeah I agree but good luck with that. Dark side of human nature means if you put lots of people in a small space, statistically there are going to be a few rotten apples.

    • by wvmarle (1070040)

      Of course. Security must come in layers, and requires a holistic approach.

      But tight computer security can also help keeping the human factor in check. By making sure no unauthorised persons can access the data for starters, particularly related to breaches from the outside. And then making sure there is an audit log of all accesses made to the system, particularly who accessed which piece of sensitive data when. So in case there is a security breach, that there is a way to trace back and know who did it. K

  • Copyright issues (Score:4, Informative)

    by dwheeler (321049) on Monday September 05, 2011 @12:15AM (#37306166) Homepage Journal

    We're going to see more of this sort of thing. Almost everyone assumes that all software is copyrighted, or that only the copyright holder can release software as free/libre/open source software (FLOSS). Neither are true!! This matters when the US government gets involved, because its "normal" rules are really different from most organization's.

    For example, if a government employee develops software as part of his official duties, then in practically all cases that software is NOT subject to copyright in the US (per US law 17 USC 105). It's not just that the author doesn't have copyright; there IS no copyright in the US. Also, when a contractor writes software, the government often receives all the release rights as if it was the copyright holder yet it is not the copyright holder (these are called "unlimited rights"). In this case, the government can release the software as FLOSS, on its own initiative, even though it is NOT the copyright holder. For more details, see: Publicly Releasing Open Source Software Developed for the U.S. Government [thedacs.com].

    The US government spends billions of dollars each year developing software. It's my hope that, over time, it will release more of the software it develops to the people who paid for it.

    • by wvmarle (1070040)

      So if I understand this correctly, and to say it simple: any software (and for that matter any creative work) created by the US government automatically falls in the public domain?

    • That's an interesting problem - most open source licenses depend on copyright for enforcement. If there is no copyright, those licenses can't be used. Is there a way to incorporate?

  • It seems that the extra cell-level security is more of a capability, in that you can categorize (or add a label to) a cell and when you query you specify the access level you have...and the result is included or not depending.

    I wonder how it deals with "lost security levels?" If you don't know the security level of a cell, you can't ask for it. If everyone forgets, then the data just sits around, waiting to be pruned. How can you tell the difference between a resource leak and unarchived classified document

  • It appears there are some hurdles that must be cleared concerning copyright before the project could be accepted.

    Wait... what? I'm not trying to be a grammar nazi, I'm genuinely confused. ARE some hurdles that must be CLEARED (future tense)... COULD be accepted (past test)? Which is it-- do the hurdles still need to be cleared before the project can be accepted, or have the hurdles been cleared and the project accepted?

Are you having fun yet?

Working...