Massive worldwide IT outage, hitting banks, airlines, supermarkets, broadcasters, etc. [19th July 2024]

[moochedit]

I suspect when this started and it wasn't clear what the cause was but it was all windows systems, so microsoft said they were investigating and so the media focused on them. It is now apparent it's crowdstrike thats the problem.

 

[Serge Forward]

My org is currently fine btw.

You're just showing off now.

 

[Silas Loom]

Although the CS fuckup isn’t anything to do with external mischief makers, a load of organisations switching off their endpoint security as a temporary fix will be a brilliant opportunity to deploy malware on a vast scale. So in the medium term things could be bleak for Gails’ Nairobi customers.

 

[Buddy Bradley]

BBC still feeling the need to mention the chance of it being a cyber-attack, just in case they can scare a few more people.

 

[Boris Sprinkler]

Wow. I’m just thinking about this. The root cause. And all the organizations affected who claim to be compliant with various stringent methodologies and have paid their auditors to say so. But if they themselves did not test the security update from crowdstrike before deployment into mission critical environments, then its almost as if those certifications are meaningless.

 

[teuchter]

Is that happening?

Two hospitals in Germany have cancelled non-urgent operations.

It's utter global chaos.

I just checked the trains at my local station and one of them has a four minute delay.

 

[gentlegreen]

 

[Buddy Bradley]

View attachment 433956

NYSE is closed at the moment. Wouldn't be surprised if they suspended trading of CrowdStrike stock though.

 

[moochedit]

View attachment 433956

I doubt the us markets are open yet so any crash will be this afternoon our time.

 

[Artaxerxes]

NYSE is closed at the moment. Wouldn't be surprised if they suspended trading of CrowdStrike stock though.

The NYSE might not even be able to open tbf

 

[cupid_stunt]

I just checked the trains at my local station and one of them has a four minute delay.

Thanks for your update.

Meanwhile in other people's world...

More than 1,000 flights have been cancelled around the world today so far, according to aviation analytics firm Cirium.

This figure - currently 1,078 - will only get bigger as the knock-on impacts grow.

 

[magneze]

Working from home will have exacerbated all this. IT can't run around with a USB.

 

[Buddy Bradley]

Global Windows outage leaves Mac user even more smug than usual – The Chaser

Microsoft has put out a statement saying they are doing everything they can to get their systems back up and running, hoping to harness the smugness energy of Mac users for the reboot.

chaser.com.au

 

[gentlegreen]

 

[tommers]

Lots of employees seeing their options going up in smoke

 

[planetgeli]

My stepson is Director of IT at one of the big electric companies. The Director. His words?

"Bloody nightmare"

 

[Buddy Bradley]

Working from home will have exacerbated all this. IT can't run around with a USB.

Maybe for some smaller businesses, but more than a couple of hundred users you're going to be spread over multiple sites anyway, possibly worldwide (we have offices affected globally).

 

[Boris Sprinkler]

I doubt the us markets are open yet so any crash will be this afternoon our time.

 

[DaveCinzano]

IT can't run around

Typical slur

 

[Boris Sprinkler]

It actually started affecting systems yesterday before close of trading in the US

 

[Shippou-Sensei]

As long as Shippou-Sensei hasn't thrown away any cables I'm sure this'll be straightforward to sort out.

It's in a box somewhere......

 

[existentialist]

Although the fix mentioned in The Register article is pretty simple; removing the CrowdStrike agent file, for most corporate Windows machines which are locked down tightly with Bitlocker etc, this is pretty much impossible to implement - users can't do it as they can't boot to safe mode, they can't boot from another device, and the update can't be pushed out from the central service because the machines won't boot.

There's going to be quite a lot of IT overtime worked...

 

[teuchter]

Thanks for your update.

Meanwhile in other people's world...

So far about 1% of flights worldwide. Which is in fact quite normal.

It'll go a bit above normal as the day goes on, probably. No big drama in the end though.

 

[Callie]

Working from home will have exacerbated all this. IT can't run around with a USB.

((((Floorwalkers))))

 

[Shippou-Sensei]

Typical slur

whITe men can't jump

 

[Boris Sprinkler]

The problem will be made worse, by Downstream vendors using crowdstrike. So for example you outsource your hospitals rostering system to the best vendor that specializes in rostering systems. And you opt for cloud based vms at their collocation. All those vms are using crowdstrike because “it’s best in house and all these major firms use it, so yes we are secure” and it’s gets signed off by all appropriate management.
Amd then a fuck up happens and you can no longer roster because the disaster recovery plan has not been updated due to a change of staff but we are getting a risk manager in and that will be one of their first tasks.

 

[Lord Camomile]

It's in a box somewhere......

This is what I was muttering to myself last night while looking for a power cable for a Sony Vaio. Because APPARENTLY I didn't think to put it in the box the bloody laptop was in

 

[Rimbaud]

Basically everything is on 1 of 3 cloud providers, Microsoft Azure, Google Cloud, and Amazon Web Services. This gives an idea of the extent to which we've put our eggs in 1 (well, 3) baskets.

This appears not to have been a cyber attack, but imagine the chaos if there was a coordinated attack on not just Azure but AWS and Google Cloud? Basically nothing would work, there'd be a total social breakdown within a day.

 

[PR1Berske]

Basically everything is on 1 of 3 cloud providers, Microsoft Azure, Google Cloud, and Amazon Web Services. This gives an idea of the extent to which we've put our eggs in 1 (well, 3) baskets.

This appears not to have been a cyber attack, but imagine the chaos if there was a coordinated attack on not just Azure but AWS and Google Cloud? Basically nothing would work, there'd be a total social breakdown within a day.

Without a shot being fired, n' all.

 

[vinba]

We aren't affected directly but our suppliers are as we use Just-in-time, we have 2,000 people just standing around until they resolve their IT issues and start supplying again.