Holmes 2025 4: The Tunnel Without Walls

Scenario

Players will learn to create symbol tables for Volatility3 and then investigate a memory dump from a Linux server, discovering the motivation of the malware that has infected the system.

Executive Summary

An attacker reached a Linux server over SSH, ran basic reconnaissance from a shell, then switched to a second local account whose weak password fell to an offline hash crack. They pulled code from Pastebin that installed a kernel-module rootkit, then reconfigured the server’s networking and installed dnsmasq so it would hand out DHCP leases and DNS answers for the whole segment. One workstation took the malicious configuration, its user logged in to the internal CogWork-1 portal, and followed the portal’s advice to update a piece of software. The poisoned DNS record sent the update request to an attacker-controlled host behind an nginx proxy, delivering a trojanised installer. The whole story sits in a single memory image, which only becomes readable once a matching Volatility 3 symbol table has been built.

Artifacts

• Memory image: memdump.mem → Volatility 3 with a custom Linux symbol table, plus strings for anything that lives in free pages or the page cache
• Symbol table: not shipped with the case. Build it with dwarf2json from the matching Debian kernel debug package (dbgsym) and point Volatility at it with -s

Symbols note: Volatility 3 has no Linux profile for this kernel out of the box, and nothing works (linux.pslist, linux.bash) until the symbol table is built. The kernel banner from Q1 tells you which Debian package to fetch. The dbgsym download is about 800 MB.

Discovery

The kernel banner is a plain string in memory, so strings answers this without any symbols and also tells you which Debian build to fetch debug symbols for. The release suffix is the part to copy, not the whole compile string.

strings -n8 memdump.mem | grep 'Linux version'

With the symbol table built from the Q1 kernel (Debian snapshot dbgsym through dwarf2json), linux.bash recovers command history straight from the bash process heap. The reconnaissance commands (id, uname, ip a) sit together under one bash process, and the PID column on that block is the shell.

vol -s work/syms -f memdump.mem linux.pslist > work/pslist.txt

vol -s work/syms -f memdump.mem linux.bash > work/bash.txt

Tip: linux.pslist gives the process tree, so you can confirm that the PID is a child of sshd rather than a local console shell.

Privilege Escalation

The same linux.bash output from Q2 shows an su to a second account (jm) after the recon block, which gives the username. The password is not in the history, but the account’s shadow entry is in memory as an MD5-crypt string ($1$jm$...). Pull it out of the image with strings and crack it offline in hashcat mode 500 against rockyou. The format mask wants username:password.

hashcat -m 500 jm.hash /usr/share/dict/rockyou.txt

Persistence

The rootkit is a loadable kernel module, so the kernel log in memory records the module load and the page cache still lists the .ko on disk. List the cached files and look for a module under the running kernel’s modules tree that does not belong to any Debian package.

vol -s work/syms -f memdump.mem linux.pagecache.Files > work/pcf.txt

vol -s work/syms -f memdump.mem linux.lsmod

Tip: linux.lsmod can lie when a rootkit unlinks itself, so the page cache listing and the kernel log are the more reliable sources here.

Kernel modules carry an author= string in their modinfo. Dump the module’s cached pages from the Q4 path with linux.pagecache.InodePages and run strings over the result.

vol -s work/syms -f memdump.mem linux.pagecache.InodePages --find /usr/lib/modules/5.10.0-35-amd64/kernel/lib/Nullincrevenge.ko --dump

strings -n6 *.dump | grep author

Gotcha: “Alleged” is the key word. The author field is whatever the module builder typed, so it is a claim in the file, not attribution.

Credential Access

Back in the linux.bash history from Q2, the network changes are followed by an apt install of the package that turns the server into a rogue DNS/DHCP server. A package that ships a daemon is also a running process, so look it up in the linux.pslist output for its PID. The format mask is name,PID.

grep -iE 'apt|install' work/bash.txt

grep -i dnsmasq work/pslist.txt

The dnsmasq lease database is in memory, and strings over the whole image is faster than any Volatility plugin for it. A lease line pairs a client MAC and IP with the hostname the workstation announced. The one lease handed out under the new configuration is the tricked workstation, and its IP (192.168.211.52) becomes the pivot for the web logs in Q9.

strings -n6 memdump.mem > work/str.txt

grep -E 'dhcp|192\.168\.211\.52' work/str.txt

The portal traffic reached an nginx instance on the compromised server, so its login POST bodies are in memory. Search the strings for username= and keep the request that comes from the Q7 workstation. The username comes from the POST body of the login form.

grep -E 'username=' work/str.txt

Note: This one was reasoned rather than read from a labelled field. The POST body carrying username=...&password=... after the workstation received its lease pointed to the user, and HTB accepted it. The accepted value starts with a capital letter, while the string in the POST body is lowercase.

Initial Access

nginx access log lines are still in memory. Filter on the workstation IP from Q7 and look for a GET of an installer. The format mask (/*****/******/***********/**********-***-**.***) already says it is a four-part path ending in a versioned .exe.

grep '192.168.211.52' work/str.txt | grep -i 'GET'

Two config fragments from memory give both halves. The dnsmasq config has an address=/<domain>/<ip> line that forges the update domain’s DNS answer, and the nginx config on the receiving host has a proxy_pass to the final destination with its port. Join them in domain,IP:port order.

grep -E 'address=|proxy_pass' work/str.txt

Tip: The address= line gives the domain the victim thinks it is talking to. The proxy_pass target, not the DNS answer, is the real destination.

Attack Summary

IOCs

MITRE ATT&CK

Defender Takeaways

Linux memory forensics needs symbols ready before the incident. Volatility 3 returned nothing for this image until a symbol table for the exact kernel was built from the Debian debug package. Keep symbol tables for the kernels you run, or a pipeline that builds them, so a memory capture is usable in minutes.

A weak su password undid the privilege boundary. The second account fell to rockyou, so the attacker moved sideways with no exploit. Enforce strong password policy or key-only accounts, and alert on su to a different user straight after reconnaissance commands.

Unsigned kernel modules should block or alert. The rootkit was a .ko dropped into the running kernel’s module tree. Enforce module signing, watch for new files under /usr/lib/modules, and alert on module loads that do not match a package.

A new DHCP/DNS server on the segment is an attack, not a convenience. Installing dnsmasq on a server and reconfiguring its network settings let it speak for the whole subnet. DHCP snooping on switches and monitoring for unexpected DHCP offers would catch this, and software updates should be verified by signature, not by where the DNS answer sends the download.

Conclusion

The attacker never attacked the workstation directly. They made the server the network’s source of truth for DHCP and DNS, and a trusted update domain did the rest. Signature checks on updates and a watch on who answers DHCP would have stopped it.