Linux Defend

Linux Defend#

Triage recipes for Linux hosts. Artifact collection follows the order of volatility (RFC 3227), process forensics walks /proc, and memory capture preserves what the operator cannot recover from disk.

Order of volatility (RFC 3227)#

  • Registers, cache.

  • Routing table, ARP cache, process table, kernel statistics, memory.

  • Temporary file systems.

  • Disk.

  • Remote logging and monitoring data relevant to the system.

  • Physical configuration, network topology.

  • Archival media.

System information#

$ date
$ uname -a
$ hostname
$ cat /proc/version
$ lsmod
$ service -status-all
$ fdisk -l

Files, disk and space usage#

$ lsof -i
$ du
$ df

Network configuration / sockets#

$ ifconfig -a
$ netstat -apetul
$ netstat -plan
$ netstat -plant
$ ss -l
$ ss -ta
$ ss -tp

User / account info#

$ whoami
$ who
$ last
$ lastb
$ cat /var/log/auth.log
$ cat /etc/passwd
$ cat /etc/shadow
$ cat /etc/sudoers
$ cat /etc/sudoers.d/*
$ cut -d: -f1 /etc/passwd
$ getent passwd | cut -d: -f1
$ compgen -u
$ xclip -o

Processes, calls, traffic#

$ ps -s
$ ps -l
$ ps -o
$ ps -t
$ ps -m
$ ps -a
$ ps -aef
$ ps -auxwf
$ top
$ strace -f -e trace=network -s 10000 <PROGRAM>
$ strace -f -e trace=network -s 10000 -p <PID>

Environment, startup, tasks#

$ cat /etc/profile
$ ls /etc/profile.d/
$ cat /etc/profile.d/*
$ ls /etc/cron.*
$ ls /etc/cron.*/*
$ cat /etc/cron.*/*
$ cat /etc/crontab
$ ls /etc/*.d
$ cat /etc/*.d/*
$ cat /etc/bash.bashrc
$ cat ~/.bash_profile
$ cat ~/.bashrc

Kernel, PAM plugins#

$ ls -la /lib/modules/*/kernel/*
$ ls -la ~/.mozilla/plugins
$ ls -la /usr/lib/mozilla/plugins
$ ls -la /usr/lib64/mozilla/plugins
$ ls -la ~/.config/google-chrome/Default/Extensions/
$ cat /etc/pam.d/sudo
$ cat /etc/pam.conf
$ ls /etc/pam.d/

File and directory hunts#

$ find / -type d -name ".*"                                  # hidden directories
$ lsattr / -R 2> /dev/null | grep "\----i"                   # immutable
$ find / -type f \( -perm -04000 -o -perm -02000 \) -exec ls -lg {} \;   # SUID / SGID
$ find / \( -nouser -o -nogroup \) -exec ls -lg {} \;        # no owner / group
$ file * -p                                                  # file types
$ find / -type f -exec file -p '{}' \; | grep ELF            # executables
$ find / -name ".*" -exec file -p '{}' \; | grep ELF         # hidden executables
$ find / -mtime -1                                           # modified past day
$ ssh root@IP/HOST tcpdump -i any -U -s 0 -w - 'not port 22' # remote tcpdump

Persistence paths#

/etc/rc.local
/etc/initd
/etc/rc*.d
/etc/modules
/etc/cron*
/var/spool/cron/*

Audit logs#

$ ls -al /var/log/*
$ ls -al /var/log/*tmp
$ utmpdump /var/log/btmp
$ utmpdump /var/run/utmp
$ utmpdump /var/log/wtmp

Process forensics#

$ ls -al /proc/[PID]
# cwd = current working directory of malware
# exe = binary location and whether it has been deleted

# Recover deleted binary
$ cp /proc/[PID]/exe /[dest]/[binary]

# Capture binary data
$ cp /proc/[PID]/ /[dest]/[PID]/

# Hash
$ sha1sum /[dest]/[binary]
$ md5sum  /[dest]/[binary]

# Command line and process name
$ cat /proc/[PID]/cmdline
$ cat /proc/[PID]/comm

# Environment
$ strings /proc/[PID]/environ
$ cat     /proc/[PID]/environ

# File descriptors and maps
$ ls -al /proc/[PID]/fd
$ cat    /proc/[PID]/maps

# Stack and status
$ cat /proc/[PID]/stack
$ cat /proc/[PID]/status

# Deleted binaries currently running
$ ls -alr /proc/*/exe 2> /dev/null | grep deleted

# Process working directories in interesting paths
$ ls -alr /proc/*/cwd
$ ls -alr /proc/*/cwd 2> /dev/null | grep tmp
$ ls -alr /proc/*/cwd 2> /dev/null | grep dev
$ ls -alr /proc/*/cwd 2> /dev/null | grep var
$ ls -alr /proc/*/cwd 2> /dev/null | grep home

Memory forensics#

# Dump memory
$ dd if=/dev/kmem of=/root/kmem
$ dd if=/dev/mem  of=/root/mem

# LiME
# https://github.com/504ensicsLabs/LiME/releases
$ sudo insmod ./lime.ko "path=./Linmen.mem format=raw"

# Disk image
$ fdisk -l
$ dd if=/dev/sda1 of=/[output]

References#

Contents