SolarWinds' Former CEO Blames Intern for 'solarwinds123' Password Leak (cnn.com) 172
"Current and former top executives at SolarWinds are blaming a company intern for a critical lapse in password security that apparently went undiagnosed for years," reports CNN.
The password in question, "solarwinds123," was discovered in 2019 on the public internet by an independent security researcher who warned the company that the leak had exposed a SolarWinds file server...
It is still unclear what role, if any, the leaked password may have played in enabling suspected Russian hackers to spy on multiple federal agencies and businesses in one of the most serious security breaches in U.S. history. Stolen credentials are one of three possible avenues of attack SolarWinds is investigating as it tries to uncover how it was first compromised by the hackers, who went on to hide malicious code in software updates that SolarWinds then pushed to some 18,000 customers, including numerous federal agencies. Other theories SolarWinds is exploring, said SolarWinds CEO Sudhakar Ramakrishna, include the brute-force guessing of company passwords, as well as the possibility the hackers could have entered via compromised third-party software.
Confronted by Rep. Rashida Tlaib, former SolarWinds CEO Kevin Thompson said the password issue was "a mistake that an intern made... They violated our password policies and they posted that password on an internal, on their own private Github account," Thompson said. "As soon as it was identified and brought to the attention of my security team, they took that down...."
Ramakrishna later testified that the password had been in use as early as 2017... That timeframe is considerably longer than what had been reported.
The remarks were made at a hearing of a House security committee, where Representative Katie Porter also strongly criticized the company. "I've got a stronger password than 'solarwinds123' to stop my kids from watching too much YouTube on their iPad! You and your company were supposed to be preventing the Russians from reading Defense Department emails!"
CNN also reports that Microsoft (which is leading the forensic investigation into the breach) "later said there is no evidence that the Pentagon was actually affected by the Russian spying campaign."
It is still unclear what role, if any, the leaked password may have played in enabling suspected Russian hackers to spy on multiple federal agencies and businesses in one of the most serious security breaches in U.S. history. Stolen credentials are one of three possible avenues of attack SolarWinds is investigating as it tries to uncover how it was first compromised by the hackers, who went on to hide malicious code in software updates that SolarWinds then pushed to some 18,000 customers, including numerous federal agencies. Other theories SolarWinds is exploring, said SolarWinds CEO Sudhakar Ramakrishna, include the brute-force guessing of company passwords, as well as the possibility the hackers could have entered via compromised third-party software.
Confronted by Rep. Rashida Tlaib, former SolarWinds CEO Kevin Thompson said the password issue was "a mistake that an intern made... They violated our password policies and they posted that password on an internal, on their own private Github account," Thompson said. "As soon as it was identified and brought to the attention of my security team, they took that down...."
Ramakrishna later testified that the password had been in use as early as 2017... That timeframe is considerably longer than what had been reported.
The remarks were made at a hearing of a House security committee, where Representative Katie Porter also strongly criticized the company. "I've got a stronger password than 'solarwinds123' to stop my kids from watching too much YouTube on their iPad! You and your company were supposed to be preventing the Russians from reading Defense Department emails!"
CNN also reports that Microsoft (which is leading the forensic investigation into the breach) "later said there is no evidence that the Pentagon was actually affected by the Russian spying campaign."
Fuck that guy (Score:5, Insightful)
Yeah, fuck that guy (the c-suites, not the intern)
Its not the intern's fault that a SECURITY COMPANY didn't have SECURITY POLICIES in place.
Re: Fuck that guy (Score:5, Insightful)
Re: (Score:2)
As soon as a corporate type gets out of the mail room every other sentence starts with "as a leader...."
There are two ways to spot an actual leader. They never call themselves that, and they never pass off the blame.
Re: (Score:2)
2019 and they didn't have anything in place to detect crap passwords.
Re: (Score:2)
The bigger issue is that there was remote access without MFA
In other news.... (Score:2)
The CEO also ordered a flunky to change the combination on his luggage to something besides "solarwinds123".
Re: (Score:2)
Be serious. Everyone knows all executives have the combination set to "12345". Spaceballs [youtube.com]
--
Re: (Score:2)
Re: Fuck that guy (Score:5, Insightful)
Honestly, I blame the reporter, the CEO, Congress, and their Auditor for gross stupidity and incompetence. To even say such an answer tells of a incompetence at multiple levels of the organization!
Rather than just being another form of media of consumption of the vomit of an answer this was, the reporters and Congressional staff should have responded with: You mean your personal incompetence in your inability to manage a security company and being unable to find a mediocre security and audit group allowed an employee at the lowest levels of your organization to easily violate one of the most widely known and used policies? Yes or No?
And that's before we begin discussions on how that could possibly result in bypassing your CRB, SOD, and production infrastructure protection processes.
I am sorry, but that answer shows not only how incompetent this company was but most likely how many of their customers are. Apologies if there was more details in the article and I totally misunderstood the situation.
Re: (Score:2)
Solarwinds was founded in 1999. In that era, even Linux distros had telnet ports open by default.
The world has since moved on to more secure policies. Solarwinds went to less secure policies.
Re: (Score:2)
Solarwinds had terrible security. But many government and industry organizations were relying on them for critical security without ever asking for a security audit. Even an ISO-9001 audit should have uncovered their lax practices.
This debacle was a failure on many levels.
Re: (Score:2)
Yeah. And many government and industry organizations will continue to use them. As one military IT administrator told me, "everyone is using them."
Re: (Score:2)
As one military IT administrator told me, "everyone is using them."
There is safety in numbers.
Nobody ever got fired for buying Solarwinds.
Re: Fuck that guy (Score:2)
What's CRB ? I could not find any useful info. (Assuming it's nothing on the Wikipedia disambiguation page.)
And SOD is "separation of duties", right? (Makes me think of "GRUB" and "GIMP", in terms of how well the acronym was chosen.)
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
Real Security Expert: "You took the password down from the Repo? Why didn't you remove or change the internal password in the product and notify your customers?"
CEO: "It was all Justin's fault."
Re: (Score:2)
Oh, I'm sure they had security policies in place, but you're right, if the company can't make those policies stick, fuck the CEO.
Re: (Score:2)
Re: (Score:2)
Came here to say that. Left happy that someone first posted it.
Re: (Score:2)
Same here. Happy to pile on.
Re: (Score:2)
The real fault lies with the direct manager of
Re: Fuck that guy (Score:2)
So it's like writing a backup script, but not only not checking if restoring works, but never running nor even hooking the script into the system!
Re: (Score:2)
The other sad thing is that everyone is going to keep using SolarWinds. And the hacking will continue.
Re: Fuck that guy (Score:2)
Speak for yourself!
-- 1337_h4x0r_13
Yep, blame the intern (Score:5, Insightful)
Some really stupid people may even for a second believe that is the person that screwed up. In actual reality, you had no password changes, did not disable the password after the intern left, did give an intern far too much access and failed to notice for a long, long time that people that had no business doing so were in your network. Of course, all that is the fault of said intern too, right?
Smart people, on the other hand, will see that not only is it _you_ that screwed up, but also that you have no honor or integrity, which you nicely demonstrated by blaming things on somebody that likely cannot defend themselves and has the smallest part of the blame in things.
Comment removed (Score:5, Funny)
Re: (Score:2)
Re: Microsoft? (Score:2)
Re: (Score:3)
Is your opinion that Microsoft is infamously insecure based on a 20 year old OS?
No. Forty year history of the company.
Re: Microsoft? (Score:2)
Nah. Yesterday. 9PM.
(Last time I looked at the latest security news and exploits.)
Re: (Score:2)
Micro$oft: "In February 2021, Microsoft President Brad Smith said that it was “the largest and most sophisticated attack the world has ever seen" - WP
H4x0r: solarwinds121? No? solarwinds122? No? solarw...
Re: (Score:2)
Re: (Score:2)
Yes, that's the important stuff, and CSOs and others will see it. Trouble is, the CEOs bought stock in it two years ago when their CTOs brought it into the budget.
So, "smart people...will see", but...
Falsely and deceptively blaming interns? (Score:5, Insightful)
"a mistake that an intern made... They violated our password policies and they posted that password on an internal, on their own private Github account," Thompson said.
The issue sounds to me like Not a leak of password strings if that is indeed what the string was. "solarwinds123" is not a secure string to use as a password, period - It was insecure from the moment it was set, So the idea that someone "broke" a policy by leaking this variant of password123 is absurd, and it's more ridiculous to try and blame a breach on such a leak of solarwinds123 -- It seems to me like they could properly intuit that (common company name)123 is obvious and NOT a secure password, therefore it's not been valued to keep secret, and such (dictionaryword)123 passwords are common and among the world's most commonly used passwords, take password123, for example, so any person making a code project that does anything related to solarwinds could easily be using the same one by coincidence, whether they're affiliated or not with SolarWinds.
The failure would be on SolarWinds part, for (1) Failing to implement technical measures to enforce a secure password policy,, (2) Failing to regularly audit their network using password crackers for password strength and detect such weak passwords, or (3) Failure to ensure whatever affected servers were adequately monitored and covered by such policies and auditing measures.
In addition (4) Critical resources should require Two-Factor Authentication. They are failing to meet
the bare minimum among industry best practices by not ensuring a Security Key or second factor is required for client device user logins to file servers/server systems.
Re: (Score:2)
Indeed. And they failed at all that while being an IT security company!
That said, interns are supposed to screw up from time to time. If an intern screwing up breaks the whole security of your company at a fundamental level, then you have no business being in IT security because you are just completely and utterly incompetent.
Re:Falsely and deceptively blaming interns? (Score:5, Funny)
The issue sounds to me like Not a leak of password strings if that is indeed what the string was. "solarwinds123" is not a secure string to use as a password, period ...
Ya, but if they had just switched the "1" and "i" ... :-)
Re: (Score:2)
Re: (Score:3)
Re:Falsely and deceptively blaming interns? (Score:5, Insightful)
Agreed.
With a password like 'solarwinds123' there's a good possibility that the hackers found the password not via leak, but with a straightforward dictionary attack.
Re: (Score:2)
Re: Falsely and deceptively blaming interns? (Score:2)
probably "root" or "Administrator" or "solarwinds".
Fun fact: 25% of all banks in Luxemburg have the password "Administrator" or "Telindus". The latter is the company setting up half the systems there, and my sources told me, 50% never change the pasword after they leave. At least that was the case, a decade or so ago.
Re: (Score:2)
Re: (Score:3)
I had worked in a company where (company name)123 was widely used.
It is specifically used to send documents through email, between people within the company, in order to prevent outlook filtering it. Put it in a zip and password it with the company wide known password. the outlook is too stupid to see what's in there and so prevent you sending word documents or python scripts or similar things.
It is not a security thing at all. It's a workaround for outlook policies that prevent you getting work done.
If act
Re: (Score:2)
...the CEO should be replaced by the intern, who by now has probably learned a lesson, unlike the CEO.
QFL (Quoted for Lulz)
Yeah, it's always a balancing act between risk, security and usability.
Re: (Score:2)
A lot of time people lack the understanding of how big organizations work.
It is entirely possible the company had a wonderful password policy.
It's very possible, someone wrote entire policy documents on the password policy and how often it needs to be changed.
Given how prevalent SolarWinds is, I'd venture to bet they have all kinds of amazing documents detailing their security policies. They'd definitely need it to get into many organizations.
Yet, there is such a thing as making sure your policy is actually
Why did that password exist? (Score:3)
Was it leaked as "You wouldn't BELIEVE how stupid their passwords are around here!"?
To quote Shaggy, "It wasn't me." (Score:3)
I see how it is, one of the biggest technology blunders in recent times and you blame the intern.
If any interns at Solar Winds are reading this, if you could embed this video on the main website of SolarWinds that would be great. I am sure you have the password, try "Solarwinds1234" if you have to guess. They probably changed the password rules to have one upper case and 4 numbers now.
https://youtu.be/T_x6QmuJdms [youtu.be]
Re: (Score:2)
Ha, I've run into password rules on web sites that disallowed uppercase and some special characters.
Probably the special characters because they just append the password to an SQL query, but I never understood why upper case would be disallowed.
Re: (Score:3)
Re: (Score:2)
Old mainframe backend and database. They simply can't do it because the field in the database is a crusty sixty year old relic that they insist on reusing and which cannot hold anything approaching the length of a properly hashed password value. So they limit your password to whatever the old mainframe can handle.
That would be my bank.
Re: (Score:2)
Intern? (Score:5, Insightful)
If the actions of a single intern can compromise your network, then it's not the intern's fault; it's yours. The appropriate response is to thank the intern for highlighting the flaw and fix it.
Re: (Score:2)
If the actions of a single intern can compromise your network, then it's not the intern's fault; it's yours. The appropriate response is to thank the intern for highlighting the flaw and fix it.
Very much so, yes. Interns make mistakes. Not expecting that disqualifies this company from doing anything in IT, let alone IT security.
Re: (Score:2)
Not expecting that history will repeat itself [youtu.be], similarly.
Re: (Score:2)
Ho Lee Fook. I was *so* looking forward to her pronouncing the short "u".
Re:Intern? (Score:5, Interesting)
A company that used to be next to us went quickly out of business because they stored all the customers passwords in plain text in a file that could be accessed remotely. This was the CEOs decision because it was a small startup and apparently good security is too expensive. We expanded into their space once they left.
I mean, nobody with a brain has used plain text passwords since the 70s, and any decent system will not even see the plaintext password. And yet once I got an email from a third party trainer who said I had a new account for trainng and here is the password to use to log in, and the password was my work password in plain text. So something on our system had the passwords, maybe some other internal web site, maybe there was a keylogger, maybe whatever active directory does stores the plain text, but it was pretty unnerving to see. No one really seemed that concerned about it, which was even more concerning.
Good security is often treated as an afterthought, something you do after you get hacked. Security is by necessity, inconvenient and expensive, and thus it gets bypassed a lot.
Re: (Score:3)
Ahh, Katie Porter (Score:2)
The remarks were made at a hearing of a House security committee, where Representative Katie Porter also strongly criticized the company. "I've got a stronger password than 'solarwinds123' to stop my kids from watching too much YouTube on their iPad! You and your company were supposed to be preventing the Russians from reading Defense Department emails!"
I so desperately need to see video [youtu.be] of this.
Re: (Score:2)
I so desperately need to see video [youtu.be] of this.
Now I wonder what this screw-up that pretended to be a CEO has as combination on his luggage....
Re: (Score:3)
Yes, found it [twitter.com]!
Re: (Score:2)
Got me! I clicked, expecting to see a CongressCritter berating an executive about stupid passwords. Well played!
Though I think I'll look for a vid about it. Watching bullshitters wallow in their, um, product, is fun.
If the intern's responsible, he should get the pay (Score:5, Insightful)
Clearly the CEO isn't in charge. Give his pay to the intern, the guy who makes security decisions. Huge responsibility. Isn't that why the people up top allegedly make the big bucks?
SW Stops Russians from Reading DOD Emails? (Score:2)
You're doing something seriously wrong if Solarwinds is used to protect your email.
Re: (Score:2)
Well, there is a reason this former "CEO" thinks this ludicrous excuse may work: He knows his customers are even more stupid than he is.
Re: (Score:2)
Re: (Score:2)
Yeah, absolutely right. But great line, anyways.
Intern had access to production's servers?! (Score:4)
Really?!
Re: (Score:2)
Are they stupid (Score:2)
Literally, every hacker knows to try 123 .. even before trying "password" you don't need an intern to tell you that. Almost everyone in the world knows many companies use that as a password, it's common knowledge.
View from under a bus. (Score:3)
You hear that? That's the sound an intern makes when thrown under a bus.
who wants to bet? (Score:2)
It was an engineering intern.... So someone with privileges and rights as an engineer did it.
And of course they aren't around anymore to be questioned about how it happened.
How CONVENIENT! -- Charlie did it but he's dead now. Oh Well.
Asok did it (Score:2)
Isn't that one of the roles of interns?
Hacking like in the hollywood movies .. (Score:2)
Thank you solarwind after explaining to my normal friends and relatives, that normally portrait hacking as seen in movies is just not real .. thank you for proving me wrong.
Hacker1: "You cannot break into that computer network, it's a high security service company" .." .. password?
female Hacker to Hacker2(one on the keyboard): "Start with the basics try solarwinds123."
Hacker1: "Yeah right, nobody with a sane mind would use that
Hacker2: "I'm in!"
Hacker1:
conclusion:
But don't worry it really was the interns fa
Very funny (Score:2)
"solarwinds123" is a password? When used for Solarwinds?
Ridiculous.
Re: (Score:2)
I could beat those Russians. I'd use "lunarwaves321".
Re: (Score:2)
"1unarwaves32l". n00b.
Poor Intern (Score:2)
I used to "google" password a few times a month to see which idiots that I was providing services to were "helping" our business partners.
I caught about 10 or 15 internal clients that I promptly referred to corporate IT security.
Some of them got their hands slapped, some of them had their system account privileges revoked.
Begs the question (Score:2)
Who in their right mind would hire a security company whose file server password is "solarwinds123"?
Answer: Probably a president who uses "12345" for the combination to his luggage.
Uhhh (Score:2)
But not really surprising nevertheless. I worked for a huge tech company who had a big presence in network security and discovered that all of the offices and small engineering data centers had APC UPSes which were all connected to the network. I enquired about getting power utilization statistics for an upcoming deployment and the facilities team had no clue how to give me the data... so
If the committee buys this excuse (Score:2)
If the House security committee buys this excuse and absolves SolarWinds of the responsibility, US cybersecurity is going to be a laughing stock of the world. If this story is even remotely true, SolarWinds should not be trusted with anything requiring security beyond passing messages in a high school classroom of who likes whom. If the story is a complete fabrication and the committee buys it, then we have a completely incompetent government.
Brute force (Score:2)
>"Other theories SolarWinds is exploring, said SolarWinds CEO Sudhakar Ramakrishna, include the brute-force guessing of company passwords"
Any system that ALLOWS brute-force guessing of passwords is broken. Full-stop. It is not a problem with passwords. Brute force is easily stopped with trial delays, filters, number of trial limits, and auto account locking (even if just temporary).
Re: (Score:2)
Point of order, with the password being solarwinds123 that would have been, at most, the 3rd password tried. So all those measures could have been in place, and with the password used it wouldn't have mattered.
Re: (Score:2)
>"Point of order, with the password being solarwinds123 that would have been, at most, the 3rd password tried. "
Are you sure?
solarwinds
Solarwinds
solarwinds1
solarwinds[2-9]
[etc]
Had it been solarwinds1, I would probably agree with you, since most systems require 1 number. Yes, it was a pretty obvious password, but people tend to think of "brute force" as throwing thousands and millions of tries at something.
Just like Boeing.. (Score:2)
.. blaming maintainers for their dodgy AF gaming the regulations that made a critical system non-redundant.
So, we still have no proof Russia had anything to do with it.... yet more deflection?
Re: (Score:2)
At this point I'd assume everybody except maybe the Nigerians but including the Russians were up in Solarwind's servers, all routing their traffic through Russian VPNs.
WHY ?? (Score:2)
Blames intern for stupid passwording. (Score:2)
No. This is the kind of shit that rolls UPHILL.
This password should NEVER have existed.
Other passwords to try. (Score:2)
Its not getting better.... (Score:2)
If an intern can fuck up hundreds of prominent customer systems completely, then the correct statement is "we had problems with out internal review and quality assurance processes".
Which is worse? (Score:3)
Which is worse?
A) An intern set a dumb password
B) A security company created a system allowing any old intern to set a dumb password.
Re: (Score:3)
Although not the strongest, "solarwinds123" is at least fairly long, and contains some numbers... it could have been worse.
You are seriously going to claim that? This password is among the first 10 things a halfway competent pen-tester would try just to check for extreme stupidity.
Re: (Score:2)
Re: (Score:2)
This password is among the first 10 things a halfway competent pen-tester would try
Yes but we are talking about criminals here, not competent pen-testers.
Why would "criminals" who earn a living from hacking be less competent than a mediocre pen-tester?
Re: Read my post again, my claim holds. (Score:2)
I agree. I'd say it would require a seeded dictionary with some additional simple brute-forcing. So harder than it looks. Not hard, but, not cracked by a lazy stupid person either.
Re: interns eh.. well (Score:3, Funny)
Re: (Score:3)
Doing work without pay is still not a crime yet in the US?
Unpaid internships are illegal in America.
They are only allowed if the position is purely for learning without any actual work. But in that case, it isn't really an internship.
Re: (Score:2)
Unpaid internships are illegal in America.
No, they are not. The number of situations where the intern is the "primary beneficiary" is small, but they are legal and in use in certain circumstances. It doesn't take much effort to google "are unpaid internships legal" to see the criteria and reasons they might benefit the intern.
--
Re: (Score:2)
Re: (Score:2)