How a Faulty CrowdStrike Update Caused a Global IT Outage by Affecting 8.5 Million Windows Devices

On Friday, July 19, a faulty update from CrowdStrike caused a global IT outage, impacting 8.5 million Windows devices. Microsoft stated this affected "less than one percent of all Windows machines," but it still disrupted numerous sectors including retail, banking, and aviation.

CrowdStrike's technical breakdown released on the same day provided more details about the incident. The issue stemmed from a sensor configuration update to their Falcon platform, which led to a Windows logic error.

Configuration File Issues

The problematic configuration files, known as "Channel Files," are integral to the Falcon sensor's behavioural protection mechanisms. These files are updated multiple times daily in response to new tactics and techniques identified by CrowdStrike. This process has been part of Falcon's architecture since its inception.

Patrick Wardle, a security researcher and founder of Objective See, confirmed that the explanation aligns with earlier analyses. He noted that the file "C-00000291-" caused a logic error leading to an OS crash via CSAgent.sys.

Timeline of Events

On July 19, 2024, at 04:09 UTC, CrowdStrike released a sensor configuration update for Windows systems. This update inadvertently triggered a logic error resulting in system crashes and blue screens (BSOD) on affected devices.

The systems impacted were those running Falcon sensor for Windows version 7.11 and above that downloaded the update between 04:09 UTC and 05:27 UTC. Despite settings meant to prevent automatic updates, these channel file updates were pushed to computers regardless.

Impact on Various Sectors

This widespread issue affected many industries reliant on Windows systems. Retailers, banks, airlines, and other sectors experienced significant disruptions due to the system crashes caused by the faulty update.

CrowdStrike clarified that the configuration file responsible is not a kernel driver but determines how Falcon evaluates named pipe execution on Windows systems. This distinction is crucial in understanding why such an error could cause widespread system failures.

The incident underscores the importance of rigorous testing before deploying updates across critical systems. It also highlights the interconnected nature of modern IT infrastructure where an issue in one component can have far consequences.

This event serves as a reminder for organisations to review their update management processes and ensure robust safeguards are in place to mitigate similar risks in the future.