Lax SSH Key Management A "Big Problem" 212
cstacy writes "Tatu Yionen, inventor of SSH, says he feels
'a moral responsibility' to come out of retirement and warn that a 'little-noticed problem' could jeopardize the security of much of the world's confidential data. He is referring to the management (or lack thereof) of SSH keys (i.e. 'authorized_keys') files. He suggests that most organizations simply allow the SSH key files to be created, copied, accumulated, and abandoned, all over their network, making easy pickings for intruders to gain access. Do you think this is a widespread problem? How does your company manage SSH keys?"
cstacy's summary here is accurate, but as charlesTheLurker notes, the article is a bit over the top: "The Washington Times claims that there's a huge vulnerability in ssh. It turns out that some reporter there has discovered that you can do passwordless login with the software, and has spun this into a story of a dangerous vulnerability. Sigh."
Passwordless login (Score:5, Funny)
I've just noticed a huge vulnerability in keyless entry on cars - you can open the door without a key!
** just so we don't have a story without a car analogy **
Re: (Score:2, Offtopic)
Re: (Score:3)
Or a librarian.
Or even Nicholas Cage.
Re: (Score:2)
I've just noticed a huge vulnerability in keyless entry on cars - you can open the door without a key!
** just so we don't have a story without a car analogy **
You can get in some convertibles without opening the door even if the windows are up.
--
Re: (Score:2)
Yes, that huge gaping hole where the roof normally is...
who is doing this? (Score:3)
Re: (Score:2)
Re:who is doing this? (Score:5, Insightful)
Re: (Score:2)
That is what you should do.
I use rssh for something like this all the time and each process has it own user. That takes planning though and people are lazy.
Re: (Score:2)
Re:who is doing this? (Score:5, Funny)
I always run my backup scripts in chroot environments. I've also found that, as a bonus, they complete many orders of magnitude faster and use significantly less space. Huge wins all around.
Re:who is doing this? (Score:4, Informative)
I think you are trying to be funny and failed (also I think you are mixing up chroot and /dev/null)
If you chroot, you can't see any files if the aren't in the chroot directory or below. So, if you want to back up /etc, your chroot better include /etc (either as the root or as a subdirectory).
This is why the OP said the backup finished faster...because after the chroot, none of the files that needed to be backed up were visible.
Re: (Score:2)
... unless you mount the files/directories you want to backup.
Isn't that what mount --bind/rbind is for?
Re: (Score:2)
Did you read the parent's post? How exactly do you restrict an account used for backups on a Linux system? I'm not saying yours is a bad idea, but this was a poor context to use it in.
Backups are a perfectly valid reason for password-less logins, and a really embarrassingly bad case to use for privilege separation in UNIX.
If you have to use SSH for this kind of job you can create a separate key especially for this job so that it's only valid in the context of making backups. You can also restrict what you can do on the remote endpoint (OpenSSH has support for that) so that it can only be used for the purpose of creating backups. That means that you can have your main ssh keys encrypted, and still use SSH for automatic backups without having to decrypt the key file.
Re: (Score:2)
You have the client you are backing up push the backups to the backup server via SSH, instead of the backup server remoting into you client to pull the files. This way you can deposit the files into a restricted account on the backup server.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
You could copy the files to some unprivileged user home dir and have the servers run a cronjob to mv and chown the files as needed. That way if they get the management server all they can do is bork the config files.
Re:who is doing this? (Score:5, Insightful)
Who exactly is it that isn't password protecting their ssh keys?
Anyone who needs services to launch without manual intervention--which is a whole lot of services and a whole lot of people...
Re: (Score:3)
Which is why we have things like chroot and rssh. Limiting the scope of the damage is a vital practice with those kinds of accounts.
Re: (Score:3)
Which is why we have things like chroot and rssh. Limiting the scope of the damage is a vital practice with those kinds of accounts.
Yep. And more directly related to the article--not being a dipshit idiot about the key files ;-)
Re: (Score:2)
Re: (Score:3)
We just put the password in another file, and use a script to handle it.
Naah just put the password on the command line. What could possibly go wrong?
(Note, just kidding)
At one point I had at work an implementation looking a lot like the Debian development anon-ftp system... Sure, upload whatever you want, but if its more than 5 minutes old and has no valid GPG signature, then it gets wiped. On the other hand if its got a valid GPG sig, trust it and run it. If you're going to put an anon ftp server on the internet you need to partition upload and download so as not to become
Re: (Score:2)
That's a good idea. In that case your trust is in the keysigning. I think this is what some folks are missing - a passwordless ssh-key based automatic system is relying on other means of trust. Whether or not that trust is well placed or constructed is a separate matter. An encrypted key-based connection is still a great improvement over older systems.
Re: (Score:2)
Re: (Score:2)
Who exactly is it that isn't password protecting their ssh keys? I mean if you choose to press enter shame on you.
It appears that you haven't seen any workplace yet.
Re:who is doing this? (Score:5, Informative)
It is not a matter of password protecting ssh keys, it is a matter of deleting obsolete keys from ~/.ssh/authorized_keys when they aren't used anymore. When someone quits or gets fired someone should be going through everything they have access to and removing the keys. They are remarkably easy to forget where they were put as well.
After running my head into this problem one too many times I ended up creating a management system that rebuilds every copy of authorized_keys on each server.
Re: (Score:2)
Who exactly is it that isn't password protecting their ssh keys? I mean if you choose to press enter shame on you.
From the IT policy standpoint, there's no way of requiring that a key has a password. There are lots of people who don't understand (or otherwise care to use) ssh-agent and similar mechanisms, and there are lots of people who assUme that their own systems' security is sufficient and don't realize that it jeopardizes the security of the IT department's systems.
For this reason, there are lots of security-conscious departments that ban SSH key access on any external-facing system.
Re: (Score:2)
For this reason, there are lots of security-conscious departments that ban SSH key access on any external-facing system.
So what your telling me is that they decided that a password that said user probably wrote on a sticky-note attached to their laptop or saved in a plaintext password is more secure than a ssh private key that MIGHT not be password protected?
If a user isn't going to properly secure an ssh private key, there is no way in hell they are going to properly protect a password!
Re: (Score:2)
Be aware that there is a reason that ssh asks for a "passphrase". Encyrption passwords must have much more entropy than login passwords because the only thing restricting the number of attempts the attacker may make is the amount of computing power the attacker has at their disposal.
I bet there are a lot of ssh keys out there with passwords that ammount to nothing more than a false sense of security.
Not sure where you work.. (Score:2)
Unmanaged keys are most certainly a pretty serious security vulnerability, and I agree - in my experience it is endemic.
Key management - signing, validation, revocation - doesn't happen for free. Everything worthwhile requires effort.
Passwords are a worse vulnerability (Score:5, Insightful)
So rather than scare people about poor key management, let's scare people about bad passwords -- which is nearly all passwords.
Re:Passwords are a worse vulnerability (Score:4, Interesting)
So rather than scare people about poor key management, let's scare people about bad passwords -- which is nearly all passwords.
Hey slashdot does anyone have an implementation where the sshd config would look something like:
PubkeyAuthentication yes
PasswordAuthentication no any/any
PasswordAuthentication yes 10.0.0.0/8
And no, last time I checked openssh could not do that. Either yes or no, no src address filtering.
The closest I could come up with is running two SSH servers on different port numbers and filter at the network level which src addrs can talk to which port.
Re: (Score:2)
why would you want passwords in your lan?
when you need passwords sometimes, when you are on the road with your laptop or at some pc, which is not yours, okay there its useful. But in you own lan, you always can access some ssh-key. So your policy isn't very useful that way.
Re: (Score:2)
(years ago) I used to get blasted at home from China IP addrs trying various root passwords. Flooding my logs with failures. Since then I only allow key auth from the untrustable internet. I'd never allow password auth over the internet. Copying a key is not a big deal. Or logging into another machine that I know has a key, etc.
On the LAN, if some clown gets infected and port scanning / password scanning, I can literally walk over and physically take care of it.
Re:Passwords are a worse vulnerability (Score:4, Interesting)
(years ago) I used to get blasted at home from China IP addrs trying various root passwords. Flooding my logs with failures. Since then I only allow key auth from the untrustable internet. I'd never allow password auth over the internet. Copying a key is not a big deal. Or logging into another machine that I know has a key, etc.
On the LAN, if some clown gets infected and port scanning / password scanning, I can literally walk over and physically take care of it.
This may not be of use to you anymore, but I'll toss it out there: fail2ban.
I use it to ban IPs attempting ssh-as-root on first attempt, ssh login password failures on 3rd attempt. Uses iptables to block the malicious addresses. Works like a charm.
Also has "jails" for Apache-based log failures, such as attempts to access PHPMyAdmin (what ever it's called), which is worthwhile to run just for that, and a host of others.
Re: (Score:2)
I believe fail2ban can now ban entire subnets, so if you're being attacked from China, just block China's subnet (they of course have many, but you get the idea). I think you can set it up so the first ban is for an IP, if it bans more than X from a given subnet, the whole subnet gets banned (or if you're really evil/smart, redirects to a black-hole machine with no valid logins).
One thing to remember is that for non-scripted attacks (where someone is specifically going after YOU), the instant you ban them,
Re: (Score:2)
If copying a key is not a big deal why do you want password authentication enabled for lan?
Re: (Score:2)
why would you want passwords in your lan? when you need passwords sometimes, when you are on the road with your laptop or at some pc, which is not yours, okay there its useful. But in you own lan, you always can access some ssh-key. So your policy isn't very useful that way.
When I first started my current job, we could only login to one server by using an SSH Key. However, you were expected to generate your own key and put in on the server. The only method was to use the insecure FTP (where passwords were allowed) to upload a new authorized_keys file.
A couple years later, the hard disk failed and we had to rebuild the server. Needless to say, the SSH Key requirement did not get transferred. (And no, I'm not the admin; if I were It might get reinstated for logins from the ou
Re: (Score:2)
i think (good) password vs. ssh-keys is both okay from the security point of view. But ssh-keys have much more convenience, where you have them (with ssh-agent you type your password only once, agent forwarding, etc.), and passwords are a sane fallback for when you just somewhere and downloaded putty to access your server. another good option for not so trustworthy environments are one-time-passwords. There is a pam module for this, and you just carry a list of 100 OTPs.
Re: (Score:2)
i think (good) password vs. ssh-keys is both okay from the security point of view. But ssh-keys have much more convenience, where you have them (with ssh-agent you type your password only once, agent forwarding, etc.), and passwords are a sane fallback for when you just somewhere and downloaded putty to access your server. another good option for not so trustworthy environments are one-time-passwords. There is a pam module for this, and you just carry a list of 100 OTPs.
Any links for that PAM module? Or how the 100 OTP list would work?
Re: (Score:2)
I'm not sure about that specific module, but I've found using the Google Authenticator [google.com] pam module to be really useful. It ties in easily with SSH and allows users to use time-varying OTPs generated using the TOTP standard to authenticate to the system.
It's worth pointing out that although the module is developed by Google, it does not rely on nor communicate with their servers in any way: it's just an implementation of the TOTP standard. One can use their Google Authenticator client for iOS/Android or any o
Re: (Score:2)
its called libpam-opie. There are debian packages, and googling finds some howtos. I'm not sure which to link, because some apply on freebsd, others are written in german ... so search for yourself which one works for you.
In the end you get prompted with a seed ((almost) constant per account) and a serial number, and can generate the corrsponding OTP (a sequence of some english words) or use a list of precomputed OTPs.
Re: (Score:2)
Re: (Score:2)
Re:Passwords are a worse vulnerability (Score:5, Informative)
And no, last time I checked openssh could not do that
Last I checked, PasswordAuthentication is allowed inside a Match block, so
PubkeyAuthentication yes
PasswordAuthentication no
Match Address 10.0.0.0/8
PasswordAuthentication yes
Re: (Score:2)
Re: (Score:2)
And no, last time I checked openssh could not do that
Last I checked, PasswordAuthentication is allowed inside a Match block, so
PubkeyAuthentication yes
PasswordAuthentication no
Match Address 10.0.0.0/8
PasswordAuthentication yes
With a note that it doesn't always work:
https://bugzilla.redhat.com/show_bug.cgi?id=869903 [redhat.com]
Re: (Score:2)
https://bugzilla.redhat.com/show_bug.cgi?id=869903 [redhat.com]
Weird, missing comments on that thread.
Re: (Score:2)
You should be able to approximate that using Match keywords in sshd_config.
Re: (Score:3)
xinetd can. we launch a separate sshd with separate config file from xinetd with response from only certain ip addresses.
Re:Passwords are a worse vulnerability (Score:4, Informative)
use the Match config file directive:
PubkeyAuthentication yes
PasswordAuthentication no
Match Address 10.0.0.0/8
PasswordAuthentication yes
Re: (Score:2)
Re: (Score:2)
I wish more people thought like you do. I protect all hosts like they were real assets or potential vectors for local 10./ infection.
Because they are. Pubic facing? Secure perimeters? False security.
Your keys, my keys (Score:5, Insightful)
I've worked at a place where someone thought it was a great idea to mass-add a specific entry everyones authorized_keys, "because then $x would just work". No need to tell everyone of course.
It's not enoguh that you know how to manage your keys. The rest of the organization has to know what's OK and not.
Re: (Score:3)
Would have been a lot funnier if they did
cat somekey > ~/.ssh/authorized_keys
rather than
cat somekey >> ~/.ssh/authorized_keys
I suppose that is one way to "solve" the problem.
Re: (Score:2)
Turned out someone had a cron job that checked certain files for modifications, including authorized_keys. So it was detected fairly quickly.
With great power comes great responsibility, or whatever it is that sudo sais the first time you run it.
Re: (Score:2)
just fyi, i found your comment somewhat confusing - probably because of a word you left out :)
i'd guess "entry everyones authorized_keys" should have been something like "entry into everyone's authorized_keys", which then starts to make some sense
Re: (Score:2)
His name is (Score:5, Informative)
Re: (Score:2)
Bloody sans-serif fonts.
Forget SSH (Score:2)
Is he going to run around yelling, "The plane, boss the plane!"
That would be a good reason to come out of retirement.
Robert Paulson (Score:2)
His name is Tatu Ylönen.
His name is Tatu Ylönen...
Inability of server to enforce policy (Score:5, Insightful)
The biggest issue with SSH is the inability of the server to enforce policy on the client keys. If I'm wrong, I'd love to be corrected and learn what I've been overlooking.
As it stands, there's no way for the server to reject a key that has no pass phrase, a poor passphrase, or an old pass phrase. Short of over-the-shoulder random audits of users using their keys, there's no way to enforce a policy that sets minimum standards for pass phrases on SSH keys.
To my way of thinking that is one of the bigger areas of risk with and drawbacks of the use of SSH.
Re: (Score:3)
If you are that worried about it you could disable the use of ssh keys. No one is going to be guessing the keys anyway, but it is a concern if someone copied one.
Re:Inability of server to enforce policy (Score:5, Insightful)
Its a mess. In more detail.
The pass phrase never leaves the client, right, so you'd have to magically protect against trojaned client that lies and pinky swears that it really did check.
Or send the passphrase cleartext to the server for the server to test it. That would be bad. Of course if you had a trustable connection between, then you could safely transfer the passphrase to the server for testing, but then you'd already trust the link so it would be pointless.
The solution is after you set up the trusted link you send the passphrase over the link and have the server verify the phrase and dump the connection if it fails.
There's probably some horrible failure mode where if a server is owned then you're really screwed because now it has your passphrase which is probably your gmail password and xbox password and who knows what else.
Kerberos does it (somewhat) better.
Re: (Score:2)
There are two (very ugly) "secure" solutions to this.
1. Draconian: The IT department requests the private key, tries to brute force access, then deletes it after a certain degree of failure. IIRC, pubkeys can be generated from private keys without passwords, so it's verifiable. Big snag: the user could remove the password later. As long as the private key is safely and securely submitted (say via an SSL form) and safely/securely stored during the brute forcing, this is as secure as your trust in the s
Re: (Score:2)
Kerberos does it (somewhat) better.
But Kerberos management is itself a PITA. I suppose it's not too bad within the context of a single organization, but it can't scale larger than that because of cross-domain trust problems. (Large organizations often internally organize themselves into multiple smaller organizations that are usually at loggerheads with each other over something stupid. Or the budget allocation.)
It's important to remember that the main problem with SSH is that it allows authentication via password by default. That's by far t
Re: (Score:3)
That's because the server has no way of knowing anything about the passphrase unless the client tells it, and no way of knowing if the client is telling the truth, so it would be foolish to include such verification.
Re: (Score:3)
As it stands, there's no way for the server to reject a key that has no pass phrase, a poor passphrase, or an old pass phrase
There is, however, a way for the server to accept only particular client keys -- say, keys that are known to have been generated on a smart card. SSH can be configured to look for authorized keys somewhere that users cannot write to (a basic centralized keystore), and there is some amount of support for a PKI. You cannot force users to have good passphrases, but you can force them to use smartcards (which is probably better than a passphrase anyway).
Re: (Score:2)
How does the server know the key came from a smartcard?
Re: (Score:3)
When the smartcard is given a public/private key pair, it registers the public key with a basic centralized keystore. Then the ssh server starts refusing key authentication with any client not trying to authenticate against one of the public keys generated by the smartcard setup process.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
Re:Inability of server to enforce policy (Score:4, Informative)
Of course there isn't. The server doesn't see the passphrase and there'd be no point in trusting the client to tell you "oh yeah, there was a huge passphrase, I promise".
Re: (Score:2)
That's the client enforcing policy, which is not the server enforcing policy and hence completely irrelevant.
It's not hard (Score:3)
Managing SSH keys for known service accounts is easy with configuration management tools like Puppet, Chef, or Salt.
We've used it for backups (Score:2)
It's not perfect, but - if you restrict where it (ostensibly) can connect from, and you restrict it to a specific command, I wouldn't exactly consider it low-hanging fruit.
Glad I read entire summary (Score:2)
The SSH design is part of the problem (Score:2)
The SSH design of how keys are actually handled is part of the problem. This design makes things harder for key management. The first is that the ssh daemon wants to only look at one file. It should provide for a way to keep the files separate so it is not necessary to rebuild the authorized_keys file. Have a directory where keys can be stored with multiple keys per file. Provide an option to allow the user to configure which key files are active (choose whether not executable or not world readable is
Re: (Score:2)
How would that be any better? It doesn't know how to figure out which keys to get. And it still has to rewrite the whole file.
Kerberos 5 is the solution (Score:2)
Re:By not using SSH (Score:4)
What do you use to log in from one machine into another, telnet? Or instead of scp you ftp stuff?
Also I have several boxes where the only way I can run "X" GUI stuff (at home, my mythtv backends) is via SSH X11 forwarding.
Re: (Score:2)
Either there's a way to get data in and out of these computers -- and therefore the potential for attack -- or the computers are just sitting their twiddling their thumbs. The idea that somehow disconnecting from the big scary Internet will keep you safe is ridiculous; there have been a number of high-profile attacks that did not require Internet access in recent years, and before Internet access was common most viruses spread via shared physical access, not network access.
Re: (Score:2)
What will you do after your office closes due to being less productive than other workplaces?
Re: (Score:2)
What will you do after your office closes due to being less productive than other workplaces?
It's probably government. And, most likely, 90% of the users just use SSH and don't tell anyone.
Re: (Score:2)
If you have to do things between computers in your local network (transfer a file, or execute a process, or whatever), doing it using an unencrypted protocol means that any computer doing sniffing in that network could get both content and access. And a trojan in the desktop of the clueless/weakest link on your network could give from inside access to people from outside.
Don't feel safe because not explicit access from outside, whats inside matters too.
Re: (Score:2)
If you have to do things between computers in your local network (transfer a file, or execute a process, or whatever), doing it using an unencrypted protocol means that any computer doing sniffing in that network could get both content and access.
With switches, you can't see any packets unless they are destined for the Ethernet card in the machine doing the sniffing. So, it means that one of computers involved in the transfer is compromised, in which case there isn't any need to sniff the traffic...you just read the file from the local disk. It could be that your switch is compromised, and has been instructed to copy packets from other ports. In that case, encryption over the network would help.
Re: (Score:2)
And, of course, no one has developed any packet sniffing tools that work in a switched environment......
Re: (Score:2)
With switches, you can't see any packets unless they are destined for the Ethernet card in the machine doing the sniffing. So, it means that one of computers involved in the transfer is compromised, in which case there isn't any need to sniff the traffic...you just read the file from the local disk. It could be that your switch is compromised, and has been instructed to copy packets from other ports. In that case, encryption over the network would help.
You don't know how a switch works. It caches the mac address of the remote machines, and only sends out packets on the interface it received traffic from that mac on. The problem is that they trust any mac addresses they see. If I set my ethernet card's mac address to the same as another machine, I will get that machine's traffic until it sends something to make the mac caching switch back.
A typical impersonation attack on a machine on a switch includes a DoS attack to get it to stop sending packages (n
Re: (Score:2)
ifconfig eth0 hw ether 00:50:56:c0:00:01
There, you just cloned one of my vmware instance's mac.
Trusting your switch is ridiculous. Routing provides true isolation, but still, any real security has to be implemented in several layers. Ignoring the application layer and trusting your network is a recipe for disaster.
Re: (Score:2)
Re: (Score:2)
How quaint...
Re: (Score:2)
Public keys are just that. What good would that really do for you?
Re: (Score:2)
remember the debian ssh key fiasco. How many people still use some of these not-so-random keys? I agree that posting your authorized_keys should not be a problem. But the least information on your setup out there is the better.
Re: (Score:2)
AuthorizedKeysCommand can police this easily (Score:2)
In my opinion, this is the interest of the new authorizedkeyscommand. A sample usage is available at http://www.sysadmin.org.au/index.php/2012/12/authorizedkeyscommand/ [sysadmin.org.au]
Nice! AuthorizedKeysCommand [gossamer-threads.com] (introduced 2012/10/31) can do exactly what we need: Set up a script that (securely) logs key usage and e.g. deny any key that is older than 366 days (by first use or else filesystem timestamp) and hasn't been used in 90 days or for any user whose last login (regardless of which key) was over 60 days ago, with a list of exceptions (by key, not user).
That's still messier than something that can go right into the authorized_keys file as a parameter, but it would do the trick ha
Re: (Score:2)
Maybe i have too much christmas cheer in me still but how on earth would this solve the problem (and not make it actually worse)?