PBR and Kittens: A Case Involving APT 35 Presented @ CactusCon 11

Uncategorized Leave a comment

At the end of January 2023, James Navarro and Jacob Wellnitz from Kudelski Security’s US Incident Response team spoke at CactusCon 11 in Mesa, Arizona.

This presentation was the culmination of an almost year-long investigation and response into a cyber-attack against a client’s newly acquired subsidiary. A recording of the in-person presentation that accompanies this article can be found on YouTube. This attack is believed to have been carried out by nation state-sponsored threat actors, known as Charming Kitten and Nemesis Kitten. The Threat Actor is also known as Phosphorus, Magic Hound, Newscaster, and APT 35 among others. This Threat Actor is known to be focused on long-term, resource-intensive cyber espionage activities. MITRE provides additional information about this group on the ATT&CK project website here.

Background

Kudelski Security was contacted by an existing client after their internal security tool detected suspicious internal port scanning activity. The activity was targeting the operational technology network within an oil and gas refinery of a company they had recently acquired. The newly acquired company operates several plants, including petroleum refineries.

Kudelski Security’s client inherited all the IT infrastructure of the acquired company, including the vulnerabilities, missing patches, and compromised systems. If the acquired company had performed the SANS Security Awareness Maturity Model exercise, they would likely have fallen as a 1 on the 1 to 5 scale.

1. SANS Security Awareness Maturity Model. Maturity Model | SANS Security Awareness

Preparation

Unfortunately, the acquired company had no incident response plans or playbooks in place. The engagement, therefore, had to begin with extensive discovery.

Visit the Kudelski Security website to find out how we help clients prepare these critical documents that support incident preparedness.

Detection and Analysis

Threat Intelligence

From May 2021 onwards, the FBI and CISA released multiple advisories that specified the Tactics, Techniques, and Procedures leveraged by Advanced Persistent Threats, they attributed to Iran nexus actors, among others. While these advisories were centered on Fortinet devices and vulnerabilities, the post-exploitation activities listed were consistent with what we saw leveraged against our client. Though Fortinet devices were not used by the organization, associated IoCs in the advisories would have been useful for the organization to look for. In fact, in this instance, the IoCs would have enabled the acquired company to carry out threat hunting that would have revealed exploitation.

Kudelski Security ingests FBI and CISA alerts as they come out and uses the information to support and enrich our Managed Detection and Response services.

It was later discovered after speaking with on-site staff and over the course of the investigation by our Incident Response team that an FBI special agent had already reached out to the acquired company in April 2022 about suspicious activity specific to the organization’s domain.

Detection

The initial detection that spurred Kudelski Security Incident Response team’s engagement was an alert from Palo Alto Cortex XDR on April 18, 2022. The acquired company only noticed the alert thanks to an information technology worker logging into the Cortex console. This alert referenced potential port scanning activity related to a refinery OT network. Investigation by the Incident Response team confirmed that Cortex alerting had been disabled, likely by the threat actor, in January 2022.

2. Portion of de-obfuscated code that triggered the Cortex alert.

CrowdStrike Detections

Kudelski Security partners with CrowdStrike as our one of our preferred tools for rapid incident response services. As such, once CrowdStrike was deployed, we were able to see several detections that matched CrowdStrike’s existing Falcon Intelligence and Machine Learning models for malicious activity.

While CrowdStrike had detections that were readily available this threat actor had more tools deployed in the environment that needed to be discovered by Threat Hunting.

Analysis

The Incident Response team correlated results from threat hunting and digital forensic artifact analysis to identify compromised machines. This led to additional IoCs and more machines to investigate. Based on IoCs from the FBI, we were able to identify initial access as a Log4Shell exploitation of the network’s VMWare Horizon environment on January 8, 2022. Additional investigation showed that the threat actor stood up their command and control (C2) infrastructure on December 26, 2021. This was only a month after the discovery of Log4Shell on November 24, 2021.

Analysis of attacker activities and reverse engineering of the binaries utilized by the threat actor tracked to IoCs for a Log4Shell exploit via VMware Horizon. VMSA-2021-0028 from December 10th, 2021 shows this software as vulnerable. Forensic artifact analysis shows the attacker then compromised the VMware Identity Manager platform within the network to deploy backdoor users, escalate privileges, and enable lateral movement. Attacker outputs were then sent back via webhook.

3. Incident timeline.

Additional DotNET binaries found, and reverse engineered appeared to be of the same strain of malware used by MuddyWater. As of the time of publication only one security vendor on VirusTotal flags the C2 domain used by the malware as malicious.

Attribution

Kudelski Security Incident Response Team cyber threat intelligence correlates this attack to threat actors that, according to CISA, operate under Iranian government sponsorship. Several CISA alerts such as AA21-321A, AA22-055A, AA22-138B, AA22-174A, AA22-257A, and AA22-320A match with TTPs seen in this engagement. It should be noted that CISA’s advisory on VMWare vulnerabilities (AA22-138B) was not released until May 18, 2022, about a month after Kudelski Security Incident Response was engaged and five months after the network was exploited. There were also DotNET binaries found that match those utilized by MuddyWater, another known threat actor believed to be from the same region. This may show collaboration or shared toolsets between actors.

Compromised Systems

The Kudelski Security Incident Response team identified well over a dozen systems that were compromised by the threat actor. We define ‘compromised’ as there being evidence of malicious code executed on the system. Notable examples of systems compromised include Domain Controllers, SQL servers, Microsoft Exchange, user Virtual Desktop Infrastructure (VDI) machines, VMWare Horizon components, and the VMWare Identity Manager appliance. Many of these systems had multiple backdoor & C2 methods discovered such as ngrok tunnels, malicious webhooks, web shells, and dropped malware.

Accessed Systems

Additional systems were accessed as well. We define ‘accessed systems’ as those that show evidence of login activity by a threat actor. Several dozen systems were surreptitiously accessed such as other Domain Controllers, additional SQL servers, user VDI VMs, and file servers.

Threat Hunting

The team threat hunted across our client’s environment for many different IoCs and TTPs based on both advisories and discoveries from forensic artifact collections.

James Navarro, our Lead Threat Hunter and Detection Engineer for the US incident response team, has provided some example CrowdStrike queries that may help organizations hunt for similar activity by this threat actor.

Kill Chain in MITRE ATT&CK

TA0001 – Initial Access

The threat actor utilized the infamous Log4Shell vulnerability against the organization’s VMWare Horizon environment. Here is an example of a CrowdStrike event query that can be modified based on environment.

Log4Shell
"(event_simpleName IN (""ProcessRollup2"", ""SyntheticProcessRollup2"") AND (GrandParentBaseFileName=""java*"" OR ParentBaseFileName=""java*"" OR ImageFileName=""*java*"")) OR (event_simpleName=""Network*"" AND RPort IN (""389"", ""1389"", ""636"", ""3269"", ""53"", ""5353"", ""1099"", ""11164"", ""10164"", ""2481"", ""2482"", ""1521"", ""3700"", ""6485"", ""6486"") AND NOT RemoteIP IN (""10.0.0.0/8"", ""172.16.0.0/12"", ""192.168.0.0/16"", ""127.0.0.1"")) OR (event_simpleName=""DnsRequest*"")
| eval processId = coalesce(ContextProcessId_decimal,TargetProcessId_decimal,SourceProcessId_decimal,ParentProcessId_decimal)
| eval temp_resolvedIps=split(IP4Records,"";"")
| eval temp_cname=split(CNAMERecords,"";"")
| eval temp_remoteIp=coalesce(FirstIP4Record,RemoteIP,temp_resolvedIps)
| bucket _time span=30m
| stats values(event_simpleName) as eventNames, values(GrandParentBaseFileName) as grandParentProcessNames, values(ParentBaseFileName) as parentProcessNames, values(ImageFileName) as processPaths, values(FileName) as processNames, values(DomainName) as domainNames, values(RemoteIP) as networkRemoteIps, values(temp_remoteIp) as coalescedRemoteIps, values(RPort) as networkRemotePorts, values(temp_cname) as dnsCNAMERecords, values(FirstIP4Record) as dnsFirstIpRecords, values(RespondingDnsServer) as dnsRespondingServers, values(CommandLine) as commandLines by processId, ComputerName, _time
| convert ctime(_time)
| search (grandParentProcessNames=""*java*"" OR parentProcessNames=""*java*"" OR processPaths=""*java*"") AND networkRemotePorts=""*""
| sort eventTimes desc"

TA0002 – Execution

The execution stage in this case focused on leveraging PowerShell to stage the threat actor’s malware and Command and Control infrastructure.

PowerShell Download
((CommandLine="*.DownloadString(*" OR CommandLine="*.DownloadFile(*") OR (CommandHistory="*.DownloadString(*" OR CommandHistory="*.DownloadFile(*")) 
| table _time ComputerName UserName FileName FilePath CommmandLine  SHA256HashData

Malicious PowerShell Process - Connect To Internet With Hidden Window
"TERM(""powershell"") ImageFileName=""*powershell.exe"" AND CommandLine IN (""* -Ex*"", ""*IEX*"") AND CommandLine=""*Net.WebClient*"" AND CommandLine=""*New-Object *"" AND CommandLine=""* -W*"" AND CommandLine=""* h*""
| stats min(_time) as firstTime, max(_time) as lastTime count, values(CommandLine) as commandLines by ComputerName, ImageFileName
| convert ctime(*Time)"

Powershell Reverse Shell Connection
(ImageFileName="*\\powershell.exe" AND (CommandLine="*new-object system.net.sockets.tcpclient*" OR CommandHistory="*new-object system.net.sockets.tcpclient*")) | table CommandLine,CommandHistory


PowerShell Pastebin Download
"FileName=""powershell.exe"" CommandLine=""*http*"" CommandLine IN (""*pastebin*"", ""*github*"", ""*ghostbin*"", ""*0bin*"", ""*zerobin*"", ""*privatebin*"", ""*klgrth*"", ""*.onion*"", ""*termbin*"", ""*hatebin*"", ""*hastebin*"", ""*paste.*"", ""*dumpz*"")
| stats values(_time) as eventTimes, values(ParentBaseFileName) as ParentProcesses, values(CommandLine) as commandLines count by ComputerName, ImageFileName
| convert ctime(eventTimes)"

TA0003 – Persistence

Multiple persistence mechanisms were found such as reverse shells, webhooks, SSH tunnels, ngrok tunnels, BackRecover.exe, CharlesBokowski.exe, and Interop.exe.

NGROK Tunnel
((CommandLine="* tcp 139*" OR CommandLine="* tcp 445*" OR CommandLine="* tcp 3389*" OR CommandLine="* tcp 5985*" OR CommandLine="* tcp 5986*") AND (CommandLine="* start *" AND CommandLine="*--all*" AND CommandLine="*--config*" AND CommandLine="*.yml*") AND ((ImageFileName="*ngrok.exe") AND (CommandLine="* tcp *" OR CommandLine="* http *" OR CommandLine="* authtoken *")))

DNS Tunnel Technique
(ImageFileName="*\\powershell.exe" AND ParentBaseFileName="*\\excel.exe" AND (CommandLine="*DataExchange.dll*" OR CommandHistory="*DataExchange.dll*"))

Transfer.io
Commandline=IEX(New-Object Net.WebClient).downloadString('http:*//transfer.sh) OR
DomainName IN (http://transfer.sh/get/ejJVyh/task.ps1, http://transfer.sh/get/blWdQM/a.ps1, http://transfer.sh/get/ejJVyh/task.ps1,  http://transfer.sh/get/1rzRLy/a.zip, http://transfer.sh/get/Y2DXfc/task.ps1,  https://webhook.site/945948d3-b94a-4a1b-923b-f8ad583c9b2e, https://webhook.site/f8a54c75-5e5c-4fb5-9115-57f9204b8dda) | table _time ComputerName UserName FileName DomainName RemoteAddressIP4 RPort


WebShells
ImageFileName IN (aspx_okqmeibjplh.aspx,aspx_[a-z]{13}\.aspx,*\System32\Wininet.xml,dhvqx.aspx,aspx_dyukbdcxjfi.aspx) OR CommandLine In (*\Windows\Wininet.bat,*\Windows\dllhost.exe) OR FileName IN (user.exe,MicrosoftOutLookUpdater.exe,MicrosoftOutlookUpdater.bat,MicrosoftOutlookUpdater.xml,GoogleChangeManagement.xml,Connector3.exe)

Backdoors, WebShell, BackRecover.exe, CharlesBokowski.exe, Interop.exe
event_simpleName IN ("ProcessRollUp2","SyntheticProcessRollUp2","DnsRequest","DomainName")  
FileName IN (ECB64Power.exe, impact.zip, CharlesBokowski.zip, CharlesBokowski.exe, Interop.exe, BackRecover.exe, HpDriverUpdate.exe, Details-of-Complaint.docx, Arabic.dotm, Taliban%20relations.docx, NY.docx) OR
CommandLine IN (C:\CharlesBokowski.exe, get-displayname interop) OR 
SHA256HashData IN ("7cb14b58f35a4e3e13903d3237c28bb386d5a56fea88cda16ce01cbf0e5ad8e",
 "Ea127fbcbc184d751cc225e2e87149708ed93df1f37a526d06d0e48b92d48a7e",
"3418b564f18ca0f4f162945fca2922d3d20e669b0242017701e59708c5fce582",
"3355c82f26acff64860005ba137c267bb07c426ac3a4ac4dd6fe1cab50ab36e2a",
"8a286eb052bb77061d9e947b5d3f41f1ee469ace8cf7437a890b2365992b2ac0",
"c40923c35aed9830a3c295894663cb8bfd331640f5593f0d4da729accb22c4bb",
"7f680efadef8c0b3a192b2814077b7b5d8543d20dd24b1d8939f3fec013059a3",
"b5cbce4831a0fd36c728a5c3408a341df41f2d58f618a70b47cac13c7b351ff4", 
"83cb42558f9bbaea5a19240d06149cf4994c94bc3a64485d6f11bd23e6e05fb1",
"a913a35858f873ba7169a2a335d7efa185f186366d7b10fa325fc39d233b9b7f ", 
"a8c062846411d3fb8ceb0b2fe34389c4910a4887cd39552d30e6a03a02f4cc78",
"28DE2CCFF30A4F198670B66B6F9A0CE5F5F9B7F889C2F5E6A4E365DEA1C89D53",
"01CA3F6DC5DA4B98915DD8D6C19289DCB21B0691DF1BB320650C3EB0DB3F214C",
"7CC5E44FD84D98942C45799F367DB78ADC36A5424B7F8D9319346F945F64A72") 
|  table _time,ComputerName,FileName,UserName, CommandLine, SHA256HashData

TA0004 – Privilege Escalation

Harvesting credentials, Dumped LSASS, Password Guessing/Cracking. This threat actor harvested credentials, dumped LSASS and utilized password attacks. Forensic artifact collection analysis showed that tools used included BloodHound, SharpHound, and MimiKatz.

BloodHound and SharpHound Hack Tool
((ImageFileName="*\\Bloodhound.exe*" OR ImageFileName="*\\SharpHound.exe*") OR ((CommandLine="* -CollectionMethod All *" OR CommandLine="*.exe -c All -d *" OR CommandLine="*Invoke-Bloodhound*" OR CommandLine="*Get-BloodHoundData*") OR (CommandHistory="* -CollectionMethod All *" OR CommandHistory="*.exe -c All -d *" OR CommandHistory="*Invoke-Bloodhound*" OR CommandHistory="*Get-BloodHoundData*")) OR ((CommandLine="* -JsonFolder *" OR CommandHistory="* -JsonFolder *") AND (CommandLine="* -ZipFileName *" OR CommandHistory="* -ZipFileName *")) OR ((CommandLine="* DCOnly *" OR CommandHistory="* DCOnly *") AND (CommandLine="* --NoSaveCache *" OR CommandHistory="* --NoSaveCache *"))) || table _time ComputerName UserName ImageFileName CommandLine CommandHistory ShaHashData256

Mimikatz

"ImageFileName IN (""sekurlsa::logonpasswords"", ""lsadump::dcsync"", ""lsadump::backupkeys + dpapi::chrome"", ""misc::memssp"") OR TargetFileName IN ("*mimilsa.log", "*.kirbi") OR TemporaryFileName IN (“mimilsa.log”, "*.kirbi")
| stats  values(_time) as eventTimes, values(ImageFileName) as processPaths, values(ParentBaseFileName) as parentProcessNames, values(CommandLine) as commandLines count by ComputerName
| convert ctime(eventTimes)"""

LSASS Process Memory Dump Files
(((TargetFileName="*\\lsass.dmp" OR TargetFileName="*\\lsass.zip" OR TargetFileName="*\\lsass.rar" OR TargetFileName="*\\Temp\\dumpert.dmp" OR TargetFileName="*\\Andrew.dmp" OR TargetFileName="*\\Coredump.dmp") OR (TemporaryFileName="*\\lsass.dmp" OR TemporaryFileName="*\\lsass.zip" OR TemporaryFileName="*\\lsass.rar" OR TemporaryFileName="*\\Temp\\dumpert.dmp" OR TemporaryFileName="*\\Andrew.dmp" OR TemporaryFileName="*\\Coredump.dmp")) OR ((TargetFileName="*\\lsass_2*" OR TargetFileName="*\\lsassdump*" OR TargetFileName="*\\lsassdmp*") OR (TemporaryFileName="*\\lsass_2*" OR TemporaryFileName="*\\lsassdump*" OR TemporaryFileName="*\\lsassdmp*")) OR (((TargetFileName="*\\lsass*") OR (TemporaryFileName="*\\lsass*")) AND ((TargetFileName="*.dmp*") OR (TemporaryFileName="*.dmp*"))) OR ((TargetFileName="*SQLDmpr*" OR TemporaryFileName="*SQLDmpr*") AND (TargetFileName="*.mdmp" OR TemporaryFileName="*.mdmp")) OR ((TargetFileName="nanodump*" OR TemporaryFileName="nanodump*") AND (TargetFileName="*.dmp" OR TemporaryFileName="*.dmp")))

Password Cracking with Hashcat
(ImageFileName="*\\hashcat.exe" OR ((CommandLine="*-a *" OR CommandHistory="*-a *") AND (CommandLine="*-m 1000 *" OR CommandHistory="*-m 1000 *") AND (CommandLine="*-r *" OR CommandHistory="*-r *")))

Hydra Password Guessing Hack Tool
(((CommandLine="*-u *" OR CommandHistory="*-u *") AND (CommandLine="*-p *" OR CommandHistory="*-p *")) AND ((CommandLine="*^USER^*" OR CommandLine="*^PASS^*") OR (CommandHistory="*^USER^*" OR CommandHistory="*^PASS^*")))

CrackMapExec Command Execution
(((CommandLine="*cmd.exe /Q /c * 1> \\\\*\\*\\* 2>&1" OR CommandLine="*cmd.exe /C * > \\\\*\\*\\* 2>&1" OR CommandLine="*cmd.exe /C * > *\\Temp\\* 2>&1") OR (CommandHistory="*cmd.exe /Q /c * 1> \\\\*\\*\\* 2>&1" OR CommandHistory="*cmd.exe /C * > \\\\*\\*\\* 2>&1" OR CommandHistory="*cmd.exe /C * > *\\Temp\\* 2>&1")) AND ((CommandLine="*powershell.exe -exec bypass -noni -nop -w 1 -C \"*" OR CommandLine="*powershell.exe -noni -nop -w 1 -enc *") OR (CommandHistory="*powershell.exe -exec bypass -noni -nop -w 1 -C \"*" OR CommandHistory="*powershell.exe -noni -nop -w 1 -enc *"))) | table CommandLine,CommandHistory

RomCom RAT & Privilege Escalation Tool
((((ParentBaseFileName="*\\cmd.exe") AND ((CommandLine="*ApcHelper.sys*" OR CommandLine="*/c*" OR CommandLine="*del*" OR CommandLine="*system*" OR CommandLine="*sc*" OR CommandLine="*create*" OR CommandLine="*ApcHelper*" OR CommandLine="*kernal*") OR (CommandHistory="*ApcHelper.sys*" OR CommandHistory="*/c*" OR CommandHistory="*del*" OR CommandHistory="*system*" OR CommandHistory="*sc*" OR CommandHistory="*create*" OR CommandHistory="*ApcHelper*" OR CommandHistory="*kernal*"))) OR (ImageFileName="*\\powershell.exe" AND ((CommandLine="*Invoke-WebRequest*") OR (CommandHistory="*Invoke-WebRequest*")))) OR (ImageFileName="*\\cmd.exe" AND (CommandLine="*rundll32.exe*" OR CommandHistory="*rundll32.exe*") AND (CommandLine="*startWorker*" OR CommandHistory="*startWorker*") AND (CommandLine="*comDll.dll*" OR CommandHistory="*comDll.dll*") AND (CommandLine="*system32*" OR CommandHistory="*system32*")))

TA0005 – Defense Evasion

The threat actor targeting our client utilized various methods to avoid defensive measures within the network. These tactics included obfuscating PowerShell script names, using Base64 encoding, using valid account credentials, and abusing Domain Admin accounts. There was also detected usage of Impacket, LOLBins, and, perhaps the most interesting finding, disablement of the Palo Alto Cortex EDR’s alerting.

Impacket Tool Execution
((ImageFileName="*\\goldenPac*" OR ImageFileName="*\\karmaSMB*" OR ImageFileName="*\\kintercept*" OR ImageFileName="*\\ntlmrelayx*" OR ImageFileName="*\\rpcdump*" OR ImageFileName="*\\samrdump*" OR ImageFileName="*\\secretsdump*" OR ImageFileName="*\\smbexec*" OR ImageFileName="*\\smbrelayx*" OR ImageFileName="*\\wmiexec*" OR ImageFileName="*\\wmipersist*") OR (ImageFileName="*\\atexec_windows.exe" OR ImageFileName="*\\dcomexec_windows.exe" OR ImageFileName="*\\dpapi_windows.exe" OR ImageFileName="*\\findDelegation_windows.exe" OR ImageFileName="*\\GetADUsers_windows.exe" OR ImageFileName="*\\GetNPUsers_windows.exe" OR ImageFileName="*\\getPac_windows.exe" OR ImageFileName="*\\getST_windows.exe" OR ImageFileName="*\\getTGT_windows.exe" OR ImageFileName="*\\GetUserSPNs_windows.exe" OR ImageFileName="*\\ifmap_windows.exe" OR ImageFileName="*\\mimikatz_windows.exe" OR ImageFileName="*\\netview_windows.exe" OR ImageFileName="*\\nmapAnswerMachine_windows.exe" OR ImageFileName="*\\opdump_windows.exe" OR ImageFileName="*\\psexec_windows.exe" OR ImageFileName="*\\rdp_check_windows.exe" OR ImageFileName="*\\sambaPipe_windows.exe" OR ImageFileName="*\\smbclient_windows.exe" OR ImageFileName="*\\smbserver_windows.exe" OR ImageFileName="*\\sniffer_windows.exe" OR ImageFileName="*\\sniff_windows.exe" OR ImageFileName="*\\split_windows.exe" OR ImageFileName="*\\ticketer_windows.exe"))

Metasploit / Impacket PsExec Service Installation
"event_simpleName=*Service*
| regex ServiceImagePath=""^.*\\\\[a-zA-Z]{8}\.exe($|\"".*)""
| regex ServiceDisplayName=""^([a-zA-Z]{4}|[a-zA-Z]{8}|[a-zA-Z]{16})$""
| stats values(_time) as Occurrences, values(ServiceDisplayName) as serviceNames, values(ServiceImagePath) as servicePaths count by ComputerName, event_simpleName
| convert ctime(Occurrences)"

Defense Evasion Techniques of SystemBC Malware
((ImageFileName="*\\reg.exe") AND (((CommandLine="*HKLM\\Software\\Policies\\Microsoft\\Windows Defender*" OR CommandHistory="*HKLM\\Software\\Policies\\Microsoft\\Windows Defender*") AND (CommandLine="*add*" OR CommandHistory="*add*") AND (CommandLine="*Disable*" OR CommandHistory="*Disable*") AND (CommandLine="*/d 1 /f*" OR CommandHistory="*/d 1 /f*")) OR ((CommandLine="*DisableBlockAtFirstSeen /t REG_DWORD /d 1 /f*" OR CommandLine="*SpynetReporting /t REG_DWORD /d 0 /f*" OR CommandLine="*SubmitSamplesConsent /t REG_DWORD /d 2 /f*") OR (CommandHistory="*DisableBlockAtFirstSeen /t REG_DWORD /d 1 /f*" OR CommandHistory="*SpynetReporting /t REG_DWORD /d 0 /f*" OR CommandHistory="*SubmitSamplesConsent /t REG_DWORD /d 2 /f*"))))

Wevtutil Cleared Log
"ImageFileName=""*\\wevtutil.exe"" CommandLine IN (""* cl *"", ""* sl *"", ""*set-log*"", ""*clear-log*"") 
| stats  values(_time) as eventTimes, values(ImageFileName) as processPaths, values(ParentBaseFileName) as parentProcessNames, values(CommandLine) as commandLines count by ComputerName
| convert ctime(eventTimes)"
WMIC Uninstall Security Product
(((CommandLine="*wmic*" OR CommandHistory="*wmic*") AND (CommandLine="*product where *" OR CommandHistory="*product where *") AND (CommandLine="*call uninstall*" OR CommandHistory="*call uninstall*") AND (CommandLine="*/nointeractive*" OR CommandHistory="*/nointeractive*")) AND ((CommandLine="* name=*" OR CommandLine="*caption like *") OR (CommandHistory="* name=*" OR CommandHistory="*caption like *")) AND ((CommandLine="*Antivirus*" OR CommandLine="*AVG *" OR CommandLine="*Crowdstrike Sensor*" OR CommandLine="*DLP Endpoint*" OR CommandLine="*Endpoint Detection*" OR CommandLine="*Endpoint Protection*" OR CommandLine="*Endpoint Security*" OR CommandLine="*Endpoint Sensor*" OR CommandLine="*ESET File Security*" OR CommandLine="*Malwarebytes*" OR CommandLine="*McAfee Agent*" OR CommandLine="*Microsoft Security Client*" OR CommandLine="*Threat Protection*" OR CommandLine="*VirusScan*" OR CommandLine="*Webroot SecureAnywhere*" OR CommandLine="*Windows Defender*" OR CommandLine="*CarbonBlack*" OR CommandLine="*Carbon Black*" OR CommandLine="*Cb Defense Sensor 64-bit*" OR CommandLine="*Dell Threat Defense*" OR CommandLine="*Cylance *" OR CommandLine="*LogRhythm System Monitor Service*") OR (CommandHistory="*Antivirus*" OR CommandHistory="*AVG *" OR CommandHistory="*Crowdstrike Sensor*" OR CommandHistory="*DLP Endpoint*" OR CommandHistory="*Endpoint Detection*" OR CommandHistory="*Endpoint Protection*" OR CommandHistory="*Endpoint Security*" OR CommandHistory="*Endpoint Sensor*" OR CommandHistory="*ESET File Security*" OR CommandHistory="*Malwarebytes*" OR CommandHistory="*McAfee Agent*" OR CommandHistory="*Microsoft Security Client*" OR CommandHistory="*Threat Protection*" OR CommandHistory="*VirusScan*" OR CommandHistory="*Webroot SecureAnywhere*" OR CommandHistory="*Windows Defender*" OR CommandHistory="*CarbonBlack*" OR CommandHistory="*Carbon Black*" OR CommandHistory="*Cb Defense Sensor 64-bit*" OR CommandHistory="*Dell Threat Defense*" OR CommandHistory="*Cylance *" OR CommandHistory="*LogRhythm System Monitor Service*")))

TA0006 – Credential Access

This threat actor used valid credentials that were harvested through various means, Mimikatz to crack password hashes, and tools such as KrbRelayUp to elevate their access.

KrbRelayUp Hack Tool
event_simpleName="win"  AND (Image="*\\KrbRelayUp.exe" OR OriginalFilename="KrbRelayUp.exe" OR (CommandLine="* relay *" AND CommandLine="* -Domain *" AND CommandLine="* -ComputerName *") OR (CommandLine="* krbscm *" AND CommandLine="* -sc *") OR (CommandLine="* spawn *" AND CommandLine="* -d *" AND CommandLine="* -cn *" AND CommandLine="* -cp *"))

ADCSPwn Hack Tool
((CommandLine="* --adcs *" OR CommandHistory="* --adcs *") AND (CommandLine="* --port *" OR CommandHistory="* --port *"))

Findstr GPP Passwords
(ImageFileName="*\\findstr.exe" AND (CommandLine="*cpassword*" OR CommandHistory="*cpassword*") AND (CommandLine="*\\sysvol\\*" OR CommandHistory="*\\sysvol\\*") AND (CommandLine="*.xml*" OR CommandHistory="*.xml*"))

VeeamBackup Database Credentials Dump
(ImageFileName="*\\sqlcmd.exe" AND (CommandLine="*SELECT*" OR CommandHistory="*SELECT*") AND (CommandLine="*TOP*" OR CommandHistory="*TOP*") AND (CommandLine="*[VeeamBackup].[dbo].[Credentials]*" OR CommandHistory="*[VeeamBackup].[dbo].[Credentials]*"))

TA0007 – Discovery

As with most Windows environment attacks, this threat actor used various tools built into the Windows operating system (such as netsh) to map out the domain structure.

Enable Network Discovery - Netsh.exe
"(ImageFileName=""*netsh.exe"" CommandLine=""*advfirewall*"" CommandLine=""*set*"" CommandLine=""*rule*"" CommandLine=""*enable=Yes*"" CommandLine=""*Network Discovery*"")
| stats values(ParentBaseFileName) as ParentBaseFileName, values(CommandLine) as CommandLine BY ImageFileName, ComputerName"

WMIC Discovery
process=~"*wmic*" AND (cmdline=~"*path*" or cmdline=~"*get*" or cmdline=~"*list*")

Suspicious Process Patterns NTDS.DIT

((((ImageFileName="*\\NTDSDump.exe" OR ImageFileName="*\\NTDSDumpEx.exe") OR (((CommandHistory="*ntds.dit*") OR (CommandLine="*ntds.dit*")) AND ((CommandHistory="*system.hiv*") OR (CommandLine="*system.hiv*"))) OR (CommandHistory="*NTDSgrab.ps1*" OR CommandLine="*NTDSgrab.ps1*")) OR (((CommandHistory="*ac i ntds*") OR (CommandLine="*ac i ntds*")) AND ((CommandHistory="*create full*") OR (CommandLine="*create full*"))) OR (((CommandHistory="*/c copy *") OR (CommandLine="*/c copy *")) AND ((CommandHistory="*\\windows\\ntds\\ntds.dit*") OR (CommandLine="*\\windows\\ntds\\ntds.dit*"))) OR (((CommandHistory="*activate instance ntds*") OR (CommandLine="*activate instance ntds*")) AND ((CommandHistory="*create full*") OR (CommandLine="*create full*"))) OR (((CommandHistory="*powershell*") OR (CommandLine="*powershell*")) AND ((CommandHistory="*ntds.dit*") OR (CommandLine="*ntds.dit*")))) OR ((CommandHistory="*ntds.dit*" OR CommandLine="*ntds.dit*") AND ((ParentBaseFileName="*\\apache*" OR ParentBaseFileName="*\\tomcat*" OR ParentBaseFileName="*\\AppData\\*" OR ParentBaseFileName="*\\Temp\\*" OR ParentBaseFileName="*\\Public\\*" OR ParentBaseFileName="*\\PerfLogs\\*") OR (ImageFileName="*\\apache*" OR ImageFileName="*\\tomcat*" OR ImageFileName="*\\AppData\\*" OR ImageFileName="*\\Temp\\*" OR ImageFileName="*\\Public\\*" OR ImageFileName="*\\PerfLogs\\*"))))

TA0008 – Lateral Movement

SMB shares were accessed, SSH abused, and of course RDP exploited.

SMB Share Server Access by Admin
event_simpleName=SmbServerShareShareOpenedEtw UserName=Administrator | table _time ComputerName UserName FileName CommandLine

RDP Hijacking traces
"event_simpleName=""RegSystemConfigValueUpdate"" AND RegObjectName=""*\RDP-Tcp"" AND RegValueName=""PortNumber"" 
| rename RegNumericValue_decimal as ""NewRDPPort""
| table timestamp, ComputerName, NewRDPPort"


SSH outside USA

"event_platform=lin event_simpleName=CriticalEnvironmentVariableChanged, EnvironmentVariableName IN (SSH_CONNECTION, USER) 
| eventstats list(EnvironmentVariableName) as EnvironmentVariableName,list(EnvironmentVariableValue) as EnvironmentVariableValue by aid, ContextProcessId_decimal
| eval tempData=mvzip(EnvironmentVariableName,EnvironmentVariableValue,"":"")
| rex field=tempData ""SSH_CONNECTION\:((?<clientIP>\d+\.\d+\.\d+\.\d+)\s+(?<rPort>\d+)\s+(?<serverIP>\d+\.\d+\.\d+\.\d+)\s+(?<lPort>\d+))""
| rex field=tempData ""USER\:(?<userName>.*)""
| where isnotnull(clientIP)
| iplocation clientIP
| lookup local=true aid_master aid OUTPUT Version as osVersion, Country as sshServerCountry
| fillnull City, Country, Region value=""-""
| table _time aid ComputerName sshServerCountry osVersion serverIP lPort userName clientIP rPort City Region Country
| where isnotnull(userName)
| sort +ComputerName, +_time | search NOT Country IN (""-"", ""United States"")"


Remote Desktop Protocol (RDP) port manipulation
"(RegObjectName=""*Terminal Server\\WinStations\\RDP-Tcp*"" OR CommandLine=""*Terminal Server\\WinStations\\RDP-Tcp*"")
| rex field=CommandLine ""(?i).*[d|value] (?<cmd_value>\\d.*?)( |\""|'|$)""
| eval rdpPort=coalesce(RegNumericValue_decimal,cmd_value)
| rename ComputerName as hostname
| stats values(UserName) as username values(_time) as occurrences values(rdpPort) as rdpPort values(ImageFileName) as initiatingProcess  count by hostname
| convert ctime(occurrences)"

RDP Reverse Tunnel
"RPort=3389 AND (RemoteAddressIP6=""::1"" OR RemoteAddressIP6=""0:0:0:0:0:0:0:1"" OR RemoteAddressIP4=""127.*"")
| stats values(_time) as Occurrences, values(RemoteAddressIP4) count by LocalAddressIP4
| convert ctime(Occurrences)"

TA0009 – Collection

Once the threat actor had achieved access to much of the network, they got to work stealing information from many places.  Browser profiles and credentials were stolen, Windows credentials were gathered, and data was written to archives for exfiltration.

SQLite Chromium Profile Data DB Access
((Product="SQLite" OR (ImageFileName="*\\sqlite.exe" OR ImageFileName="*\\sqlite3.exe")) AND ((CommandLine="*\\User Data\\*" OR CommandLine="*\\Opera Software\\*" OR CommandLine="*\\ChromiumViewer\\*") OR (CommandHistory="*\\User Data\\*" OR CommandHistory="*\\Opera Software\\*" OR CommandHistory="*\\ChromiumViewer\\*")) AND ((CommandLine="*Login Data*" OR CommandLine="*Cookies*" OR CommandLine="*Web Data*" OR CommandLine="*History*" OR CommandLine="*Bookmarks*") OR (CommandHistory="*Login Data*" OR CommandHistory="*Cookies*" OR CommandHistory="*Web Data*" OR CommandHistory="*History*" OR CommandHistory="*Bookmarks*")))

SQLite Firefox Profile Data DB Access
((Product="SQLite" OR (ImageFileName="*\\sqlite.exe" OR ImageFileName="*\\sqlite3.exe")) AND ((CommandLine="*cookies.sqlite*" OR CommandLine="*places.sqlite*") OR (CommandHistory="*cookies.sqlite*" OR CommandHistory="*places.sqlite*")))
Powershell ChromeLoader Browser Hijacker
(ImageFileName="*\\chrome.exe" AND (ParentBaseFileName="*\\powershell.exe" OR ParentBaseFileName="*\\pwsh.exe") AND ((CommandLine="*--load-extension=*") OR (CommandHistory="*--load-extension=*")) AND ((CommandLine="*\\AppData\\Local\\*") OR (CommandHistory="*\\AppData\\Local\\*")))

Suspicious Infostealer Malware
((ImageFileName="*\\powershell.exe*") AND (CommandLine="*Start*" OR CommandHistory="*Start*") AND (CommandLine="*-Sleep*" OR CommandHistory="*-Sleep*") AND (CommandLine="*-s10*" OR CommandHistory="*-s10*") AND (CommandLine="*Remove*" OR CommandHistory="*Remove*") AND (CommandLine="*-Item*" OR CommandHistory="*-Item*") AND (CommandLine="*-Path*" OR CommandHistory="*-Path*") AND (CommandLine="*\\Setupfinal.exe*" OR CommandHistory="*\\Setupfinal.exe*") AND (CommandLine="*-Force*" OR CommandHistory="*-Force*")) 

Browser Credential Store Access
(((FileName="*\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Network\\Cookies*" OR FileName="*\\Appdata\\Local\\Chrome\\User Data\\Default\\Login Data*" OR FileName="*\\AppData\\Local\\Google\\Chrome\\User Data\\Local State*") OR (FileName="*\\Appdata\\Local\\Microsoft\\Windows\\WebCache\\WebCacheV01.dat" OR FileName="*\\cookies.sqlite" OR FileName="*release\\key3.db" OR FileName="*release\\key4.db" OR FileName="*release\\logins.json")) AND NOT ((Image="*\\firefox.exe" OR Image="*\\chrome.exe") OR (Image="C:\\Program Files\\*" OR Image="C:\\Program Files (x86)\\*" OR Image="C:\\WINDOWS\\system32\\*") OR (Image="*\\MsMpEng.exe" OR Image="*\\MpCopyAccelerator.exe" OR Image="*\\thor64.exe" OR Image="*\\thor.exe") OR ParentImage="C:\\Windows\\System32\\msiexec.exe" OR (Image="System" AND ParentImage="Idle")))

TA0010 – Exfiltration

Once the data the threat actor was interested in was gathered, it was stored in zip archives and transported out of the network via multiple methods and the tunnels that had been established.

WinSCP Session Created - Possible Data Exfil
"TERM(""console"") AND CommandLine IN (""*sftp://*"" ""*scp://*"" ""*ftps://*"")
| stats values(_time) as eventTimes, values(GrandParentBaseFileName) as grandParentProcessNames, values(ParentBaseFileName) as ParentProcesses, values(CommandLine) as commandLines, values(ContextProcessId_decimal) as contextProcessDecimal count by ComputerName, ImageFileName
| convert ctime(eventTimes)"

Potential Data Staging or Exfiltration - Common Rclone Arguments
"TERM(""copy"") (CommandLine=""*copy*"" CommandLine=""*transfers*"" CommandLine=""*multi-thread-streams*""  CommandLine=""*-q*"" CommandLine=""*ignore-existing*"" CommandLine=""*auto-confirm*"") 
| rename CommandLine as commandLine ImageFileName as process RawProcessId_decimal as processID ParentBaseFileName as parentProcess ParentProcessId_decimal as parentProcessID ComputerName as hostname
| stats list(_time) as occurrences list(commandLine) as commandLine list(process) as process list(processID) as processID list(parentProcess) as parentProcess list(parentProcessID) as parentProcessID by hostname
| convert ctime(occurrences)"

Reverse Tunnel
FileName="Frps.exe" OR CommandLine="https://github.com/fatedier/frp/releases/download/v0.33.0/frp_0.33.0_windows_amd64.zip" OR FileName=SynchronizeTimeZone.xml ImageFileName IN (start.vbs,nvContainerRecovery.bat)


Exfiltration Domains
DomainName IN (filemail.com,ufile.io,mega.nz,easyupload.io)

Compress Data and Lock With Password for Exfiltration With 7-ZIP
(((CommandLine="*7z.exe*" OR CommandLine="*7za.exe*") OR (CommandHistory="*7z.exe*" OR CommandHistory="*7za.exe*")) AND (CommandLine="* -p*" OR CommandHistory="* -p*") AND ((CommandLine="* a *" OR CommandLine="* u *") OR (CommandHistory="* a *" OR CommandHistory="* u *"))) | table CommandLine,CommandHistory

TA0011 – Command and Control

Besides the usual Command and Control interfaces over tunnels that are seen frequently, the incident response team was able to find evidence that the threat actor leveraged two Twitter profiles for additional C2 activity. As of the time of publication one of these accounts has been suspended and the other is dormant. Kudelski Security will not be releasing the account information at this time to allow for further research.

Ngrok Tunnel
((CommandLine="* tcp 139*" OR CommandLine="* tcp 445*" OR CommandLine="* tcp 3389*" OR CommandLine="* tcp 5985*" OR CommandLine="* tcp 5986*") AND (CommandLine="* start *" AND CommandLine="*--all*" AND CommandLine="*--config*" AND CommandLine="*.yml*") AND ((ImageFileName="*ngrok.exe") AND (CommandLine="* tcp *" OR CommandLine="* http *" OR CommandLine="* authtoken *")))

Ngrok Tunnel Domains
DomainName IN (*tunnel.us.ngrok.com*,*tunnel.ap.ngrok.com*, *tunnel.au.ngrok.com*, *tunnel.sa.ngrok.com*, *tunnel.jp.ngrok.com*, *tunnel.in.ngrok.com*,*ngrok.io* )

Containment, Eradication, and Recovery

When possible, the Incident Response team leveraged CrowdStrike’s network containment feature to deny the adversary continued access to systems. This feature allowed us to perform analysis on the machine while simultaneously preventing network access from the system to other systems on the network. We then performed forensic artifact collection and were able to surgically remove persistence mechanisms and malicious binaries. While normally, backups would be leveraged to restore systems to a known good state, this was not possible due to the extended dwell time of the threat actor. Additionally, backups are not always healthy and sometimes will not restore properly if they have not been properly maintained.

Actions Taken

Kudelski Security has many different options to deal with an incident. Choices will always depend on the level of access provided by the client. These options include:

  • Process termination on detection of Indicators of Attack
    • Block malicious behaviors based on activity on endpoints.
  • Process termination upon detections of IoCs.
    • Block process execution based on custom rulesets developed from gathered forensic artifact collections.
  • Remediation of persistence mechanisms
    • Removal of known malware off disk
    • Removal of Scheduled Tasks or Cron jobs
    • Termination of known malicious processes in memory
  • Network Detection Rules
    • Blocks put in place for various protocols such as SSH at the host and network level.
    • Blocks entered into firewall rulesets for known malicious IPs
    • Stoppage of outbound SMB traffic to the Internet

Recovery

Patching

The Incident Response team performed several actions to assist with post breach remediation after this incident. The most important actions taken centered on vulnerability management. In one site alone we closed over 370,000 vulnerabilities. The record for one week of patching at a single site was over 100,000 vulnerabilities closed.

Purple Teaming

To identify additional vulnerabilities that needed remediation, we used a Purple Team approach. This provided information that would supplement reporting from various other tools such as CrowdStrike or Tenable Nessus scanners. Red Team operators were brought in to perform external and internal penetration testing and delivered their findings from across the organization to the remediation team.

Notable findings include:

  • Open camera access.
  • An open mail relay exposed to the Internet that allowed spoofing any email address in the domain.
  • Access to operational technology control and monitoring surfaces via unauthenticated VNC.
  • Critical vulnerabilities in appliances.

Post Incident Activities

Reporting and Lessons Learned

We generated a report for this incident that totaled over 600 pages and covered all known compromised and accessed devices. Every incident response client receives a report and a post-incident ‘Lessons Learned’ meeting where we explain the contents of the report.

MDR Onboarding

Many Kudelski Security clients choose to roll the existing work they have performed deploying EDR into our Managed Detection and Response services. In this case the client already utilized MDR through us and had the advantage of our assistance in bringing the subsidiary under management.

Continuous Vulnerability Scanning

We partner with Tenable to provide continuous vulnerability scanning as services through the Cyber Fusion Center (CFC), our MDR SOC.

Ongoing Threat Hunting

We performed additional threat hunts in not only the onboarded environment, but also across the client’s existing MDR footprint as part of this engagement. This means that the client not only got to leverage individualized threat hunts for the network in question but gained additional value for the parent company.

Benefits of MDR – Economies of Scale

After the engagement we ensured our detection engineering teams and the CFC teams at Kudelski Security got all the relevant intelligence, so that our entire client base could benefit. This work ultimately rolls up into our Threat Navigator tool, which is a MITRE ATT&CK visualization software designed to allow clients to find their detection gaps, prioritize gap elimination, and systematically strengthen their resilience to the threats that are targeting their organization.

4. Example of a MITRE ATT&CK tactic in Threat Navigator.

Get more information about the Kudelski Security Threat Navigator.

Conclusion

The Kudelski Security US Incident Response team would like to thank CactusCon 11 for allowing us to present this case to the international cybersecurity community present at the conference. The recorded presentation can be found on YouTube. We would also encourage anyone reading this article to consider Kudelski Security for their Incident Response needs. We are happy to discuss how we can provide ongoing and immediate coverage via an incident response retainer.

This article was written by Jacob Wellnitz with intense collaboration by James Navarro, both members of the Incident Response team that worked this case.

You don’t know what you don’t know – a compromise assessment will help you find out for sure if there is a threat active in your environment.

Leave a Reply