Satya Gupta, Founder and CTO, Virsec
Note: Given the severity of the ongoing SolarWinds attack, Virsec will be providing a range of expert analysis, technical deep dives, demos, and recommendations for enterprises trying to adapt to this new threat environment. As new information becomes available, we will continue to update our analysis and recommendations.
Download our Technical Brief: Taxonomy of The Attack on SolarWinds and Its Supply Chain
We are witnessing a significant uptick in a new class of attacks that exploit vulnerabilities in code used in high-value workloads, in Fortune 500 companies and government organizations. Conventional host-based security controls cannot effectively protect against these Remote Code Execution (RCE) vulnerabilities. This enables attackers to dwell in a victim’s infrastructure for long periods of time, increasing the risk of extensive damage.
What began on December 7th, 2020, as a notable but seemingly isolated attack on FireEye, has quickly snowballed into one of the most serious waves of breaches to date. Although the FBI is still investigating, it appears that a concerted campaign by sophisticated Russian bad actors throughout 2020 has impacted FireEye (leaking sensitive security research tools), the US Treasury and Commerce Departments, nuclear testing labs, and a wide range of Fortune 500 companies. As a blunt New York Times headlines summarized: “Billions Spent on U.S. Defenses Failed to Detect Giant Russian Hack.”
While most of the security news this year has focused on ransomware and the elections, these RCE attacks have been more subtle and insidious, sneaking into networks, burrowing into monitoring tools like SolarWinds, and dwelling for months aiming at stealing sensitive information.
Following are statements and analysis from some of the incident players including SolarWinds, FireEye, DHS, NSA and other players.
An advisory on December 16th, 2020 from SolarWinds set the context:
“We have been advised this attack was likely conducted by an outside nation state and intended to be a narrow, extremely targeted, and manually executed attack, as opposed to a broad, system-wide attack.”
Typically, this means that a specific person within a victim organization was spear phished. Another statement from the same Solar Winds advisory states:
“SolarWinds has just been made aware our systems experienced a highly sophisticated, manual supply chain attack on SolarWinds® Orion® Platform software builds for versions 2019.4 HF 5 and 2020.2 with no hotfix or 2020.2 HF 1.”
This implies that a specific SolarWinds developer who worked on a specific piece of code was targeted. Clearly, the attackers wanted to add a secret backdoor into source code, that could go unnoticed and persist for an extended period. The surgical precision with which the attackers went after their victim suggests they had become very comfortable with the SolarWinds Orion code base and their build processes.
Earlier, on December 13th, 2020, a DHS advisory stated that the backdoor needed two libraries as follows:
1. Orion.Core.BusinessLayer.ddl with a file hash of [b91ce2fa41029f6955bff20079468448]
2. C:\WINDOWS\SysWOW64\netsetupsvc.dll - a file not ordinarily found on a Windows system
A blog on December 13th, 2020 from FireEye stated:
“SolarWinds.Orion.Core.BusinessLayer.dll is a digitally signed component of the Orion Software Framework by SolarWinds (WOW!!). Multiple Trojanized updates were digitally signed from March - May 2020 and posted to the SolarWinds updates website, including this malware URL.
The Trojanized update file is a standard Windows Installer Patch file that includes compressed resources associated with the update, including the Trojanized SolarWinds.Orion.Core.BusinessLayer.dll component. Once the update is installed, the malicious DLL will be loaded by the legitimate SolarWinds.BusinessLayerHost.exe or SolarWinds.BusinessLayerHostx64.exe depending on system configuration).
The backdoor communicates with third-party servers via HTTP. After an initial dormancy period of up to two weeks, it retrieves and executes commands called “jobs,” which include the ability to transfer files, execute files, profile the system, reboot the machine, and disable system services. The malware disguises its network traffic as the Orion Improvement Program (OIP) protocol and stores reconnaissance results inside legitimate plugin configuration files. This allows it to blend in with legitimate SolarWinds activity to avoid detection.”
These highlight both the seriousness of the attack, as well as the high level of sophistication required by the attackers to blend in their backdoor code into legitimate SolarWinds code and yet remain stealthy for long periods.
In their blog published on Dec 14th, Volexity stated that they had previous experience with the attackers associated with the FireEye incident, and were very familiar with the TTP used on other victims.
The blog explained that in one incident they investigated, the same attackers that had targeted FireEye had infiltrated their client by exploiting a Microsoft Exchange Server vulnerability (CVE-2020-0688). Having gained persistent control on the vulnerable Exchange Server, the attackers went on to download several additional exploit tools. Some of these tools facilitated querying the Active Directory server for additional reconnaissance. With command line access to the Exchange Server, the attackers executed administrative commands on the server to gather more information about the environment. They also used PowerShell commands to perform lateral movement. One very interesting technique used by the attackers was to disable a two-factor authentication setting on the email account of targeted users when accessed via the Outlook Web Anywhere (OWA) client. This allowed them to login with just stolen user credentials alone.
The Volexity blog went on to explain that in another incident they had investigated, the same attackers had successfully exported email of targeted users using the OWA Client that front ends the Exchange Server. One exploitation tool that the attackers leveraged on the Exchange Server was disguised as a SQL Server telemetry client (sqlceip.exe). Under the hood this executable was actually the tool AdFind - a command line tool used to query and extract data from the organization’s Active Directory Server.
In both incidents that Volexity investigated, the attackers were able to access and export emails from targeted users in the organization. Armed with very credible details disclosed in these emails, the attackers composed a set of spear phishing emails to several developers. These emails had infected PDFs and EXEs whose icons looked like the MS Word icon. One or more developer fell for that subterfuge and the initial infiltration malware got planted on those developers’ machines. This initial malware established a reverse communication path between the attacker’s C&C server and the developer’s compute instance. Next, the attackers studied the source code and adroitly merged their backdoor source code into the developer source.
The attackers also used lateral movement to infect other victims in the organization. The malicious link in an email triggers a TCP. To resolve the hostname, the target triggers a DNS query to the domain avsvmcloud[.]com. The DNS response included several CNAMES. These additional hostnames resolved to IPs that really pointed to alternate C&C Servers for the attackers. While several of the IPs ended up being designated as invalid domains, some domains in the CNAME response boasted of a long history for a reason. This was because the attackers bought these domains through auctions or previous registrants before the domain expired but before they were deleted by Domain Registrars like GoDaddy. This subterfuge was necessary because many IPs use the length of the domain history to determine their authenticity. In the end, these domains ended up downloading the initial infection malware on the new victim that had triggered the query.
In their alert published on Dec 17th, NSA provided an example of how a nation state actor could abuse the authentication service by leveraging a recently disclosed Command Injection vulnerability (CVE-2020-4006) in VMware Workspace One Access, Access Connector, Identity Manager, and Identity Manager Connector. A statement below in the alert is noteworthy:
While these TTPs require the actors to already have privileged access in an on-premises environment, they are still dangerous as they can be combined with other vulnerabilities to gain initial access, then undermine trust, security, and authentication. Initial access can be established through a number of means, including known and unknown vulnerabilities.
The NSA alert goes on to lay out mitigations actions including hardening and monitoring systems that run local identity and federation services, locking down tenant single-sign-on (SSO) configuration in the cloud and monitoring for indicators of compromise. A very detailed and helpful actionable remediation plan has been published by the NSA.