In this very easy Sherlock, you will familiarize yourself with Unix auth.log and wtmp logs. We’ll explore a scenario where a Confluence server was brute-forced via its SSH service. After gaining access to the server, the attacker performed additional activities, which we can track using auth.log. Although auth.log is primarily used for brute-force analysis, we will delve into the full potential of this artifact in our investigation, including aspects of privilege escalation, persistence, and even some visibility into command execution.
An external host brute-forced SSH on a Confluence server and guessed the root password within seconds. The tool’s own credential check opened and closed a session instantly; about a minute later the attacker logged in by hand and got a terminal. From that session they created a local backdoor account and added it to the sudo group, then disconnected. They came back as the backdoor user, read /etc/shadow through sudo, and pulled a public Linux persistence toolkit from GitHub with curl. Every step is visible in auth.log alone, with wtmp providing the trustworthy clock.
auth.log → read with grep. Syslog format, so timestamps have no year and no timezonewtmp → utmpdump or last -f. Binary login records with epoch timestampsTimestamp note:
utmp.pyprints times withtime.localtime, so its output shifts with your own timezone. Force UTC (TZ=UTC) on every wtmp tool. The wtmp login lands one second after the matchingAccepted passwordline, which proves auth.log is UTC on this host too.
Count source IPs across every sshd line. One address accounts for 118 lines, made up of dozens of Invalid user attempts (admin, server_adm, svc_account, backup) seconds apart. The only other external source has a single successful root login, no failures, and wtmp history going back to January, so it is a regular admin, not the attacker.
grep -oE 'from [0-9.]+' auth.log | sort | uniq -c | sort -rn
Filter the attacker IP down to successful authentications. The first Accepted password from that source lands in the middle of the spray, which is the guess that worked. The same account is used again a minute later for the interactive login.
grep 65.2.161.68 auth.log | grep 'Accepted password'
There are two root logins from the attacker IP. The first opens and closes in the same second, which is the brute-force tool confirming the password, not a person. The manual login is the second one, and wtmp records its terminal (pts/1) one second after auth.log’s Accepted password line. Dump wtmp in UTC and read the USER_PROCESS record for the attacker IP.
TZ=UTC utmpdump wtmp
Gotcha: Running
utmp.pywithout forcing UTC prints local time. Cross-check withTZ=UTC last -F -f wtmp, which should give the same second.
systemd-logind logs New session N of user root for every login. The brute-force credential check got its own session number and was removed straight away; the session that matches the Q3 manual login (opened the same second as its Accepted password line) is the one to report.
grep -E 'New session|Removed session' auth.log
Gotcha: Taking the first new session after the brute force gives the tool’s throwaway session, not the attacker’s working one.
Follow the Q4 session to its close. sshd logs Disconnected from user root for the attacker IP and systemd-logind logs Removed session for the same session number at the same second. The wtmp DEAD_PROCESS record for pts/1 agrees.
grep -E 'Disconnected from user root|Removed session 37' auth.log
During the manual root session, useradd and usermod both fire from /dev/pts/1. The new account gets UID 1002, and about a minute later it is added to the sudo group.
grep -E 'useradd|usermod|groupadd|passwd' auth.log
A local account created with useradd on the compromised host is the Local Account sub-technique of Create Account. Adding it to sudo is separate Account Manipulation, so it doesn’t change this ID.
sudo logs every elevated command with the calling user, TTY, working directory and the full command line, which gives auth.log some command-execution visibility. The backdoor user first reads /etc/shadow, then uses curl to fetch linper.sh, a public Linux persistence toolkit.
grep 'sudo:' auth.log | grep COMMAND
Tip: The log only shows curl fetching to stdout. Nothing in auth.log shows whether the script was piped to a shell, so treat execution as unconfirmed.
| Phase | Action |
|---|---|
| Credential Access | SSH password spray against root and common service accounts from one external IP |
| Credential Access | Root password guessed; tool opens and closes a session to confirm it |
| Initial Access | Attacker logs in manually as root and gets a terminal on pts/1 |
| Persistence | Creates cyberjunkie (UID 1002) and adds it to sudo |
| Initial Access | Disconnects the root session |
| Credential Access | Returns as the backdoor user and reads /etc/shadow through sudo |
| Command and Control | Fetches the linper persistence toolkit from GitHub with curl |
| Timestamp (UTC) | Event |
|---|---|
| 2024-03-06 06:31:31 | Brute force begins from 65.2.161.68 |
| 2024-03-06 06:31:40 | Root password accepted; brute-force tool’s session opens and closes |
| 2024-03-06 06:32:44 | Manual root login accepted, session 37 |
| 2024-03-06 06:32:45 | wtmp records the terminal session on pts/1 |
| 2024-03-06 06:34:18 | cyberjunkie created |
| 2024-03-06 06:35:15 | cyberjunkie added to sudo |
| 2024-03-06 06:37:24 | Root session 37 disconnected |
| 2024-03-06 06:39:38 | sudo curl of linper.sh as cyberjunkie |
| Type | Value |
|---|---|
| IP (brute force + login) | 65.2.161[.]68 |
| Account (backdoor) | cyberjunkie (UID 1002, sudo) |
| URL | hxxps[://]raw[.]githubusercontent[.]com/montysecurity/linper/main/linper.sh |
| Technique | ID | Description |
|---|---|---|
| Brute Force: Password Guessing | T1110.001 | SSH password spray against root and service accounts |
| Valid Accounts: Local Accounts | T1078.003 | Interactive root login with the guessed password |
| Remote Services: SSH | T1021.004 | Hands-on session on pts/1 |
| Create Account: Local Account | T1136.001 | cyberjunkie backdoor user |
| Account Manipulation | T1098 | Backdoor added to the sudo group |
| OS Credential Dumping: /etc/passwd and /etc/shadow | T1003.008 | sudo cat /etc/shadow |
| Ingress Tool Transfer | T1105 | curl of the linper persistence toolkit |
Root over SSH with a password shouldn’t be possible. PermitRootLogin no and key-only authentication would have stopped this whole chain at the first guess. The spray targeted root directly because it was allowed.
A login that opens and closes in the same second is a tool, not a person. An Accepted password followed straight away by a disconnect, from an IP with a burst of failures, is a strong brute-force success signal. It’s worth its own alert, separate from failed-login volume.
Account creation and sudo-group changes are high-signal on a server. useradd and usermod -aG sudo from an interactive SSH TTY outside a change window should page someone. Both are written plainly to auth.log.
Normalise timezones at ingestion. Syslog with no year or timezone, plus helper tools that print local time, is exactly how investigators get timestamps wrong. Anchor syslog to a source with real epochs, like wtmp, before building a timeline.
A guessable root password over SSH handed the attacker the box in under ten seconds, and everything after it (backdoor account, sudo rights, a persistence toolkit) ran in plain sight in auth.log. The evidence was all there; the gap was that nothing was watching it.