Backdoored xz Package: Impact on sshd and Mitigation Strategies (CVE-2024-3094)

• April 1, 2024
• By Cyberarch Admin

In the fast-paced realm of modern computing, efficiency and security are non-negotiables. Software compression tools like XZ have long been trusted for their reliability in handling data effectively. However, recent discoveries have shed light on a troubling development within the XZ ecosystem.

What started as subtle anomalies on Debian Sid installations quickly escalated into a full-blown security concern. High CPU usage during SSH logins and sporadic Valgrind errors hinted at a deeper issue lurking within the liblzma component of the XZ package. Upon thorough investigation, it was uncovered that the very core of XZ, including its upstream repository and tarballs, had been compromised.Versions 5.6.0 and 5.6.1 of the XZ tarballs were found to harbor a clandestine backdoor, injecting an obfuscated script during the configuration process. This revelation shattered initial assumptions of a localized compromise, exposing a systemic vulnerability within the broader XZ ecosystem.The consequences of this revelation are far-reaching. Beyond mere inconvenience, the compromised liblzma component poses a significant threat to critical system functionalities. The noticeable degradation in SSH login performance is just the tip of the iceberg, raising serious concerns about potential security risks lurking within compromised systems.

CYBERARCH Warriors!! Get ready to unravel the layers of this intricate security breach, providing insights into detection methods and mitigation strategies to safeguard vulnerable systems.

Let’s unpack the situation:

• XZ and its Role: XZ is a widely used data compression library employed in various Linux distributions. It plays a crucial role in optimizing storage space and improving transmission efficiency.
• Backdoor Discovery: A backdoor was identified lurking within the aforementioned xz releases. This malicious code injects itself into the xz source code, raising serious security concerns.
• Impact on ssh Logins: The presence of this backdoor significantly impacts the performance of ssh login attempts. Users may experience noticeable slowdowns during login processes.
• Potential Security Risks: Beyond the performance degradation, the backdoor poses a more significant threat. It has the potential to compromise the security of ssh connections, potentially leading to unauthorized access or remote code execution on vulnerable systems.

Concealment within XZ Source Code:

The backdoor’s concealment within the XZ source code underscores the sophistication of its design. Initially masquerading as innocuous additions to the distributed tarballs, the backdoor evaded routine scrutiny. Notably, a snippet of obfuscated script was surreptitiously injected into the configuration process, camouflaged amidst legitimate build instructions. This obfuscation served as a veil, obscuring the true intent of the injected code and eluding detection during casual inspection.

The backdoor employed a clever two-pronged approach to remain undetected within the xz source code:

Distributed Tarball Injection: A malicious script resided solely within the distributed xz tarballs (archive files) for versions 5.6.0 and 5.6.1. This script wasn’t present in the official Git repository, making it invisible to casual inspection.

Here’s a breakdown of the script’s location:

• Deceptive Location: The script wasn’t directly included in the source code files. Instead, it was hidden within a seemingly innocuous preprocessor macro named build-to-host.m4. This macro wouldn’t raise red flags for those unfamiliar with its intended purpose.
• Obfuscation for Evasion:  The script itself was heavily obfuscated, making it difficult to understand its functionality at first glance. Obfuscation techniques often involve data manipulation and complex operations, disguising the script’s true purpose.

Compromised Repository Commits: While the malicious script resided in the tarballs, the bulk of the exploit code was committed directly into the upstream xz repository. However, these files were cleverly disguised:

• Obscured Code: The exploit code within the repository wasn’t readily apparent. It was hidden in seemingly innocuous test files named bad-3-corrupt_lzma2.xz and good-large_compressed.lzma. These filenames wouldn’t raise suspicion as they appear relevant for testing purposes.
• Deception Through Obfuscation: Similar to the script in the tarball, the exploit code within the repository was also heavily obfuscated. This complex data manipulation further concealed its malicious nature from casual inspection.

Triggering the Backdoor: A Conditional Approach

The activation of the backdoor during SSHD execution is contingent upon a set of precise conditions meticulously engineered to evade detection while maximizing efficacy. Notably, the backdoor exhibits selective activation criteria, including environmental variables, program arguments, and system configurations. The absence of specific environment settings, such as unset TERM variables or debug flags, serves as a prerequisite for triggering the backdoor. Furthermore, the binary’s execution context, particularly when invoked as /usr/sbin/sshd, plays a pivotal role in determining the backdoor’s activation.

The backdoor didn’t blindly activate on every system. Instead, it included specific conditions that needed to be met for it to inject itself into the build process:

• Targeted Architecture: The backdoor primarily focused on x86_64 Linux systems. Systems with different architectures like ARM or PowerPC were less likely to be affected.
• Compiler and Linker Requirements: The script checked if the system was using GCC (GNU Compiler Collection) and the GNU linker. If a different compiler or linker combination was used, the backdoor wouldn’t activate.
• Environment Checks: The script also performed environment checks, likely to avoid detection during analysis. If specific environment variables were set (like LD_DEBUG or LD_PROFILE), the backdoor wouldn’t trigger.

Code Injection: A Multi-Step Process

The code injection technique employed by the backdoor leverages the intricacies of the XZ build process to surreptitiously embed malicious instructions within the generated binaries. Upon successful activation, the backdoor dynamically modifies the Makefile within the liblzma component, injecting a series of obfuscated commands into the build pipeline. This injection technique capitalizes on the inherent trustworthiness of the build process, exploiting its privileged access to system resources to clandestinely implant malicious code fragments.

The code injection technique employed by the backdoor was a multi-step process that can be summarized as follows:

• Script Execution: During the initial stages of the configure script execution, the hidden script within the tarball is triggered.
• Conditional Modification:  Based on the aforementioned trigger conditions, the script decides whether to modify the build process. If the conditions are met, the script injects malicious code.
• Makefile Manipulation:  The script alters the liblzma/Makefile file within the source code. This modification inserts instructions to use specific test files containing the obfuscated exploit code.
• De-obfuscation and Execution: During the build process, the injected instructions extract and de-obfuscate the exploit code from the test files.
• Function Hooking: The de-obfuscated exploit code utilizes a technique called function hooking. It replaces crucial functions within the liblzma library (like crc32_resolve and crc64_resolve) with its own malicious code. These functions play a role in checksum calculations, making them a prime target for manipulation.
• Conditional Activation:  The injected code further refines its activation based on additional checks. These checks might involve environment variables specific to certain environments like systemd or debugging tools.
• Symbol Table Parsing (Performance Impact): If all conditions are met, the injected code performs a resource-intensive process – parsing the symbol tables in memory. This parsing is likely used to identify and potentially manipulate specific functions or symbols within the code.

Now, let’s shift gears and explore the impact this backdoor has on the functionality of sshd (Secure Shell Daemon), the program responsible for secure remote logins. We’ll delve into the observed performance slowdown and uncover the potential security nightmares this backdoor could unleash.

The Performance Puzzle: Why Logins Became Sluggish

The presence of the backdoor significantly impacted the performance of ssh login attempts. Users might have noticed a noticeable lag or delay during the login process. This slowdown stemmed from the backdoor’s resource-intensive activities:

• Hijacked Functions: Remember how the backdoor hooked crucial functions within liblzma (like crc32_resolve and crc64_resolve)? These functions are typically very fast, but the backdoor’s injected code significantly slowed them down.  The injected code likely contained additional malicious checks and manipulations, adding processing overhead and delaying the normal execution flow.
• Symbol Table Parsing Odyssey: As discussed earlier, the injected code embarked on a time-consuming journey – parsing the symbol tables in memory. This process involves analyzing the symbol tables of loaded libraries to identify specific functions or symbols. This parsing is computationally expensive, especially for large libraries, and contributed heavily to the login slowdown.  An analogy: Imagine a thief meticulously searching every room in a house (symbol tables) looking for valuables (specific functions). This extensive search would significantly slow down the burglary (login process).

Beyond Slowdowns: The Looming Security Threat

The performance degradation was just the tip of the iceberg. The true concern lies in the potential security implications of this backdoor:

• Hijacking Authentication: The Man-in-the-Middle Scheme: The backdoor specifically targeted the RSA_public_decrypt function, a critical component of ssh public key authentication. Here’s a possible scenario:
  • User Initiates Login: A user attempts to log in to a vulnerable server using a public key.
  • Backdoor Intervenes: The backdoor intercepts the call to RSA_public_decrypt.
  • Malicious Code Injection: The backdoor injects its own code in place of the legitimate RSA_public_decrypt function.
  • Potential for Man-in-the-Middle Attack: This injected code could potentially manipulate the decryption process, allowing an attacker to impersonate a legitimate user and gain unauthorized access to the system. Imagine a forger replacing a house key with a look-alike that unlocks the door for them.
• Remote Code Execution: A Backdoor to Chaos: Perhaps the most alarming consequence of the backdoor’s presence is the looming specter of remote code execution (RCE), wherein attackers leverage compromised systems as springboards for launching malicious payloads. Through carefully crafted exploits, adversaries may exploit vulnerabilities exposed by the backdoor to execute arbitrary code on remote servers, thereby assuming complete control over compromised systems. This scenario poses catastrophic ramifications for both individual users and organizations, exposing them to data breaches, service disruptions, and reputational damage.

Upgrading Away from the Threat: A Clear Path to Safety

The most critical step to mitigate this vulnerability is to upgrade to xz versions that are not affected. Here’s what you should do:

• Identify Your Version: Check the installed version of xz on your system. You can typically achieve this by running the following command in your terminal: xz –version
• Upgrade to a Safe Version: Update your xz package to a version not containing the backdoor. Consult your specific Linux distribution’s documentation or package manager for upgrade instructions. Generally, any version above 5.6.1 is considered safe. By leveraging the latest stable releases of XZ, users can safeguard their systems against potential exploits and ensure optimal performance and security.

In addition to upgrading XZ versions and deploying detection scripts, organizations are advised to bolster the security posture of their SSH servers by implementing supplementary measures to mitigate potential risks. This includes:

• Disabling Unnecessary Services: Review and disable any unnecessary services running on SSH servers to minimize the attack surface and mitigate the risk of exploitation. By limiting the exposure of vulnerable services, organizations can reduce the likelihood of successful incursions and fortify their defenses against potential threats.
• Enforcing Strong Authentication: Strengthen authentication mechanisms by enforcing the use of strong passwords, multi-factor authentication (MFA), or public key cryptography. By augmenting authentication protocols with robust security measures, organizations can enhance the resilience of their SSH servers and thwart unauthorized access attempts.
• Regular Security Audits: Conduct regular security audits and vulnerability assessments to identify and remediate potential security weaknesses within SSH server configurations. By proactively assessing system vulnerabilities and implementing corrective actions, organizations can fortify their defenses and preemptively mitigate the risks posed by emerging threats.

The Patchwork of Security: Why Updates Matter

This incident serves as a stark reminder of the importance of staying updated with security patches. Software vulnerabilities are constantly discovered, and patch updates are released to address these security gaps.  By promptly installing security patches, you significantly reduce the risk of exploitation by malicious actors.

By promptly upgrading to unaffected XZ versions, leveraging detection scripts, and implementing additional security measures, organizations can fortify their defenses against potential exploits and bolster their resilience in the face of evolving cyber threats. Florian Weimer’s diligent analysis and Vegard Nossum’s invaluable contribution in developing detection scripts underscore the collaborative effort required to address security vulnerabilities and safeguard critical infrastructure.

As we navigate the complexities of modern cybersecurity threats, it’s imperative for users and organizations to remain vigilant, share information, and report any suspicious activity promptly. By fostering a culture of transparency, collaboration, and collective responsibility, we can collectively enhance the resilience of our digital ecosystems and mitigate the risks posed by malicious actors.

Together, let’s remain steadfast in our commitment to cybersecurity, proactively addressing vulnerabilities, and fortifying our defenses to ensure a safer and more secure digital future for all.

Backdoored xz Package: Frequently Asked Questions (FAQs)

This FAQ section addresses common questions regarding the recent backdoor discovered in the xz package, brought to you by Cyberarch, your trusted partner in cybersecurity.

Q: What services does Cyberarch offer?

Cyberarch specializes in cyber security consulting, offering a wide range of services including information security, information forensics, risk management, and tailored solutions to meet client requirements.

Q: Why choose Cyberarch for information security consulting?

Cyberarch stands out in the industry as an independent and in-depth risk management consulting firm. With over two decades of hands-on experience and a track record of delivering unparalleled quality services, we have earned the trust of our clients and established ourselves as a trusted partner in information security.

Q:How does Cyberarch tailor its services to client requirements?

At Cyberarch, we understand that each client’s needs are unique. Therefore, we work closely with our clients to assess their specific requirements and customize our services accordingly. This approach ensures that our solutions are aligned with our clients’ business objectives and address their specific challenges effectively.

1. How can I partner with Cyberarch?

If you’re looking to partner with Cyberarch for your information security needs, simply reach out to us through our website or contact us directly. Our teams are ready to join forces with yours and provide tailored solutions to meet your organization’s unique requirements.

Q: What are the risks of the backdoor?

A: The backdoor could potentially slow down ssh logins and, in a worst-case scenario, allow unauthorized access or remote code execution on affected systems.

Q: How do I fix this vulnerability?

A: The most critical step is to upgrade your xz package to a version not containing the backdoor. Consult your Linux distribution’s documentation or package manager for specific upgrade instructions. Generally, any version above 5.6.1 is considered safe.

Q: Are there any additional security measures I can take?

A: Yes. Here are some recommendations:

• Disable unnecessary services on your system to reduce the attack surface.
• Enforce strong passwords and consider using key-based authentication for ssh logins.
• Conduct regular security audits to identify and address potential vulnerabilities.
• Stay informed about security threats by subscribing to advisories from your Linux distribution or trusted security sources.

We encourage you to share this information with others and report any suspicious activity to the appropriate authorities.

Remember, Cyberarch is here to help!  

Contact Us on: or +372 5912 3819. Our team of experts can provide additional guidance and support to ensure your systems are secure. 

By working together, we can create a more secure digital landscape.