




T-Mobile Screwups Caused Nationwide Outage But FCC Isn't Punishing Carrier (arstechnica.com) 38
An anonymous reader quotes a report from Ars Technica: The Federal Communications Commission has finished investigating T-Mobile for a network outage that Chairman Ajit Pai called "unacceptable." But instead of punishing the mobile carrier, the FCC is merely issuing a public notice to "remind" phone companies of "industry-accepted best practices" that could have prevented the T-Mobile outage. After the 12-hour nationwide outage on June 15 disrupted texting and calling services, including 911 emergency calls, Pai wrote that "The T-Mobile network outage is unacceptable" and that "the FCC is launching an investigation. We're demanding answers -- and so are American consumers."
Pai has a history of talking tough with carriers and not following up with punishments that might have a greater deterrence effect than sternly worded warnings. That appears to be what happened again yesterday when the FCC announced the findings from its investigation into T-Mobile. Pai said that "T-Mobile's outage was a failure" because the carrier didn't follow best practices that could have prevented or minimized it, but he announced no punishment. The matter appears to be closed based on yesterday's announcement, but we contacted Chairman Pai's office today to ask if any punishment of T-Mobile is forthcoming.
Pai has a history of talking tough with carriers and not following up with punishments that might have a greater deterrence effect than sternly worded warnings. That appears to be what happened again yesterday when the FCC announced the findings from its investigation into T-Mobile. Pai said that "T-Mobile's outage was a failure" because the carrier didn't follow best practices that could have prevented or minimized it, but he announced no punishment. The matter appears to be closed based on yesterday's announcement, but we contacted Chairman Pai's office today to ask if any punishment of T-Mobile is forthcoming.
So how much (Score:2)
Re: (Score:2)
Who knows, but as an American, I demand answers! Specifically, to the question of why he needs a mug that big.
Re:So how much (Score:4)
RTFA (Score:3)
If you quickly glanced at the article you would find this PDF
https://docs.fcc.gov/public/at... [fcc.gov]
which explains the whole thing quite nicely. Human error compounded by system complexity were the root issues.
Re: (Score:1)
Re: (Score:2)
How much do you think Pai got for this? $500000? $1000000?
Nothing. Not directly now. He got his job because they already tested him and knew he was loyal from what he did before. When he leaves his current job he will be on directorships for life. His every wish will be catered to. It will, however be perfectly "legal" unless you can see inside his head and know what motivated him to sell out. Trust me, when you see the value of those directorships, 1 million will be nothing. More like half a million a year for the rest of his life.
Re: (Score:2)
Michael Powell is doing well for himself, I certainly don't expect Pai to miss the opportunity.
Re: (Score:2)
Re:So how much (Score:4, Informative)
What happened is is explained below: https://yro.slashdot.org/comme... [slashdot.org]
Essentially an unbelievable level of incompetence combined with the stupidity of not having a fall back plan or even the most basic testing of a change in the network configuration. Then it took them 12 hours to figure out what went wrong. Yikes.
Re: (Score:2)
Re: (Score:2)
I live within view of the T-Mobile complex in Washington. I've noticed that the last several years they're occupying less and less of the space. AAA Washington took over an entire building a couple years ago, and other smaller renters have moved into other spaces.
Re: (Score:2)
Re: (Score:2)
Sternly worded! (Score:3)
Did he wag his finger at them when he gave the sternly wording warning?
Re: (Score:2)
Did he wag his finger at them when he gave the sternly wording warning?
"I hope you know that this will go down on your permanent record!" -- Violent Femmes
To Be Fair... (Score:5, Insightful)
Re: (Score:1)
you can't really expect him to do his job
On the contrary, he is doing exactly as he is told. He follows direction perfectly, or he wouldn't have this job
Outages on T-Mobile? I didn't notice (Score:2)
Re: (Score:2)
s/T-Mobile/cellular/g;
prosperity gospel blasphemy (Score:2)
https://www.theverge.com/2017/... [theverge.com]
If your pious enough, wealth comes to you, https://en.wikipedia.org/wiki/... [wikipedia.org]
No need for silly, job killing regulations....
I'm on T-mobile... (Score:2)
and I forgot that even happened. Really not that big of a deal. I guess it could be but then I'd just change carries and T mobile would eventually die. What's the issue here, exactly? Technology is faulty. Life goes on.
How about we turn the Internet off for a full 24 hours. Let people know in advance that this would be an Internet free day. Enjoy it.When we turn it back on, we can talk about it for the next week. We could even make it a national holiday. 3rd Tuesday of every October.
Re: (Score:1)
Comment removed (Score:4, Interesting)
Re: (Score:2)
" how you deal with companies that fuck over their customers."
You prohibit them from paying dividends to shareholders until they have paid out for their fines or to their creditors.
Root Cause - poor change control + poor failover (Score:2, Informative)
On June 15, 2020, T-Mobile was midway through the process of installing routers in the southeast region of its network. Once the first of the two routers was installed, active, and handling customer traffic, T-Mobile planned to slowly integrate the second new router into its network. T-Mobile had planned to configure the second new router so that it was passive, connected to the network but receiving network traffic if another router or link between routers failed. To do this, T-Mobile deliberately configured the links connected to the new passive router to have high Open Shortest Path First weights. In the course of that configuration, T-Mobile misconfigured the weights of the links of another router that was already active in the network segment but was not designed to process call signaling traffic. T-Mobile did not have a fail-safe process in place to prevent or provide notice of this misconfiguration. In the event of a router or link failure, the low Open Shortest Path First weights to this router would cause it to receive a large percentage of call signaling traffic, which it could not pass.
Less government is good (Score:2)
Why would you want the government punishing private companies for making an internal company mistake? The government governs the way that carriers are allowed to perform their service, but they don't govern or manage the companies themselves. It's up to T-Mobile to compensate their users, and it's up to the users to go after T-Mobile if the disruption was a measurable interruption. How is fining T-Mobile going to make ME whole, and what would stop T-Mobile from simply raising prices to subsidize the cost of
Re: (Score:1)
I've got no karma points, otherwise I'd mod the parent up,
Re: (Score:2)
T-Mobile has gotten paid to provide emergency services. I was pretty "meh" about it until I read that the 911 services were also affected. It's not okay to make mistakes of that magnitude that last for that long.
Re: (Score:1)
I still don't think the government should be able to fine you for that mistake. Somebody literally had a bad configuration in a router. I could argue that people should have been less reliant on their phone, because outages are going to happen.
Re: (Score:2)
If you want to be a telco, you have to carry 911 calls.
If you don't want to carry 911 calls, don't be a telco.
Re: (Score:2)
> Why would you want the government punishing private companies for making an internal company mistake?
Because they messed up 911 service. At that point, yes, they need to be explicitly punished severely enough that every other company immediately goes and checks their setup and verifies that they are in compliance with best practices.
How fun..... (Score:2)
DURING the T-Mobile outage there was ALSO an outage with Comcast where I live. We were Internet-free for hours. So much so I had assumed that the backhaul for T-Mobile must have been a fiber link that Comcast shared and was taken out by and excavator or something.