SEA telcos report minimal impact from CrowdStrike software fiasco

Telcos across Southeast Asia said that they were mostly unaffected by a global internet disruption on Friday caused by a flawed software update from endpoint security firm CrowdStrike that crashed millions of Windows-based computers worldwide.

Airports, banks, health care organizations and television stations across the globe were among many organisations impacted by the disruption. At the root of the problem was an automated software update to CrowdStrike’s EDR monitoring product Falcon, which runs on endpoints like laptops, servers, and routers with deep system access.

According to a CrowdStrike blog post, the problem was caused by a single configuration file (a.k.a. channel file) in the update that changes how Falcon inspects “named pipes” in Windows. Named pipes are what enable interprocess or intersystem communications in Windows.

CrowdStrike said the file was “designed to target newly observed, malicious named pipes being used by common C2 frameworks in cyberattacks”. However, a bug in the file triggered a logic error that essentially caused the infamous “blue screen of death” and sending Windows devices into a fatal reboot death spiral.

CrowdStrike said systems running Linux or macOS were unaffected by the update, as they don’t use that particular channel file. Microsoft said the update impacted 8.5 million Windows devices.

While an initial report from Reuters listed telecoms as one of the sectors impacted by the CrowdStrike update, several telcos around southeast Asia have issued statements saying that the impact was either minimal or non-existent.

In Malaysia, CelcomDigi said in a statement on Friday that it experienced “some disruption … on its support services, such as reload capabilities, for which service recovery teams were immediately deployed to setup workarounds, offering customers alternatives to continue reloading during this period.” Otherwise, network services were unaffected, it said.

Telekom Malaysia said on Friday that “there is no disruption to our business operations and services” in relation to the incident.

In the Philippines, Globe Telecom said in a statement that core services were unaffected, with impact limited to “a few Windows-based workstations that our employees use for work for the most part and some servers that do not materially affect our core services.”

DITO Telecommunity and PLDT-Smart also said the CrowdStrike update had no impact on their operations, with PLDT-Smart adding that it “does not use any software from CrowdStrike in its ecosystem.”

In Thailand, Digital Economy and Society (DES) Minister Prasert Chantararuangthong said that the outage had no impact on the country’s telecoms networks.

In the meantime, CrowdStrike has also issued an update to fix its software, although not all Windows machines can receive it automatically. According to media reports, some IT admins have had to reboot machines several times to get the update, while others have had to boot machines in Safe Mode to manually delete the faulty channel file.

Microsoft has released a recovery tool that creates a bootable USB drive and enables IT admins to delete the file automatically. Microsoft also said it is “collaborating with other cloud providers and stakeholders, including Google Cloud Platform and Amazon Web Services, to share awareness on the state of impact we are each seeing across the industry and inform ongoing conversations with CrowdStrike and customers.”

CrowdStrike also said it is reviewing its workflow processes to determine not only how the bug happened, but how the bad channel file made it into the update in the first place.

Many customers opt for automated updates, which is why so many machines were affected at once. However, according to The Verge, security researcher and Objective See founder Patrick Wardle noted that the upgrade was pushed even to CrowdStrike customers that did not opt for automated updates.

MORE ARTICLES YOU MAY BE INTERESTED IN...


Sign-up to our weekly newsletter

Keep up-to-date with all the latest news, articles, event and product updates posted on Developing Telecoms.
Subscribe to our FREE weekly email newsletters for the latest telecom info in developing and emerging markets globally.
Sending occasional e-mail from 3rd parties about industry white papers, online and live events relevant to subscribers helps us fund this website and free weekly newsletter. We never sell your personal data. Click here to view our privacy policy.