Disks#

A “disk” on Linux is a block device. SATA / SCSI / USB drives appear under /dev/sd*, NVMe drives under /dev/nvme*n*, SD cards under /dev/mmcblk*, and virtual disks under /dev/vd*. The kernel exposes them via udev, and lsblk is the standard overview.

Filesystems on top are mounted into the tree at a path, which is what user space actually reads and writes. The block device itself stays out of the application’s view; everything above the mount point looks like ordinary files.

Volumes#

“Volume” on Linux is a soft word. It can mean any of the layers between the raw disk and the mounted filesystem the operator actually reads and writes. The stack is composable, and most production hosts run two or three of these layers in series. Knowing which layer a name refers to (and which tool owns it) is the first step to changing it.

Layer

Common name

What it is

Physical disk

block device

The hardware (or virtual hardware). /dev/sda, /dev/nvme0n1, /dev/vda. Owned by the kernel, exposed through udev.

Partition

partition

A contiguous slice of a disk described by an MBR or GPT partition table. /dev/sda1, /dev/nvme0n1p2. Created with parted / fdisk / gdisk.

Encrypted volume

LUKS volume, dm-crypt mapping

A device-mapper target that decrypts on read and encrypts on write. /dev/mapper/<name> after cryptsetup luksOpen. See Encryption.

RAID array

md device

A device-mapper target that combines several block devices for redundancy or striping. /dev/md0 from mdadm.

LVM physical / volume group / logical volume

PV / VG / LV

Abstracts one or more block devices into a pool (VG) carved into resizable, named pieces (LVs). /dev/<vg>/<lv>. Managed with pvcreate / vgcreate / lvcreate.

Btrfs / ZFS subvolume / dataset

subvolume, dataset

Filesystem-internal volume managed by the FS itself, not device-mapper. Snapshots and quotas live here.

Filesystem

filesystem

The on-disk format the kernel mounts (ext4, xfs, btrfs, zfs, vfat, f2fs). Created with the matching mkfs.*.

Mount point

mount, bind mount

The path in the directory tree where a filesystem appears. Listed by findmnt / mount, persisted in /etc/fstab.

A typical encrypted-LVM laptop stack looks like this. The filesystem and the underlying block device are mandatory; everything between them is optional and composable. Drop LUKS for an unencrypted host, drop LVM for a single-FS-per-partition layout, slot in md RAID between LUKS and the partitions for redundancy.

        flowchart TB
    subgraph US["User space"]
        APP["Application"]
    end

    subgraph FSL["Filesystem (mandatory)"]
        MNT["Mount point<br/>/, /home, /var"]
        FS["Filesystem<br/>ext4 / xfs / btrfs"]
    end

    subgraph VL["Volume layers (optional, stackable)"]
        LV["Logical volume<br/>/dev/vg0/root"]
        VG["Volume group<br/>vg0"]
        PV["Physical volume<br/>/dev/mapper/cryptroot"]
        LUKS["LUKS / dm-crypt"]
        MD["MD RAID array<br/>/dev/md0"]
    end

    subgraph BD["Block device (mandatory)"]
        PART["Partition<br/>/dev/nvme0n1p3"]
        DISK["Physical disk<br/>/dev/nvme0n1"]
    end

    APP --> MNT --> FS --> LV --> VG --> PV --> LUKS --> MD --> PART --> DISK

    classDef opt fill:#1f2933,stroke:#58a6ff,stroke-dasharray:5 5;
    classDef req fill:#1f2933,stroke:#56d364,stroke-width:2px;
    class LV,VG,PV,LUKS,MD opt
    class APP,MNT,FS,PART,DISK req
    

Common compositions the operator meets in the wild.

  • Single disk, single FS. /dev/sda1 formatted ext4, mounted at /. Most VMs and containers ship like this.

  • Encrypted root. Partition → LUKS → LVM → ext4. The default for Debian and Ubuntu when the installer’s “encrypt the entire disk” box is checked.

  • RAID + LVM. Two disks → md0 → LVM → XFS. The classic small-server layout for storage that survives a single-disk failure.

  • Btrfs / ZFS standalone. mkfs.btrfs (or zpool create) manages disks, RAID, snapshots, and subvolumes itself. No LVM, no mdadm.

  • Bind mount. No new device, no new FS. mount --bind src dst exposes one path under another. Common in containers and chroots.

The rest of this page walks the tools layer by layer. lsblk -f is the standard “what’s actually here” command and labels every layer of the stack in one tree.

Disk layers#

A write from an application crosses seven layers of kernel machinery before bits land on the platter. Each layer has its own caches, queues, and tooling; performance bugs almost always live in the one the operator wasn’t watching.

        flowchart TB
    APP["Application<br/><i>read() / write() / mmap()</i>"]
    VFS["VFS<br/><i>generic file API; common inode / dentry cache</i>"]
    FS["Filesystem driver<br/><i>ext4 / xfs / btrfs / nfs</i>"]
    PC["Page cache<br/><i>RAM-backed read / write buffer</i>"]
    BLK["Block layer<br/><i>bio, request queues, multi-queue (blk-mq)</i>"]
    SCH["I/O scheduler<br/><i>mq-deadline / kyber / bfq / none</i>"]
    DM["Device-mapper / md / LVM<br/><i>optional; LUKS, RAID, LVM, multipath</i>"]
    DRV["Device driver<br/><i>nvme / scsi-mod / ahci / usb-storage / virtio-blk</i>"]
    BUS["Host bus<br/><i>PCIe / SATA / SAS / USB / virtio</i>"]
    HW["Storage media<br/><i>NVMe flash / SSD / spinning platter</i>"]

    APP --> VFS --> FS
    FS  --> PC
    PC  --> BLK
    BLK --> SCH --> DM --> DRV --> BUS --> HW

    classDef us fill:#1f6feb22,stroke:#58a6ff;
    classDef ks fill:#23863622,stroke:#56d364;
    classDef hw fill:#8957e522,stroke:#a371f7;
    class APP us
    class VFS,FS,PC,BLK,SCH,DM,DRV ks
    class BUS,HW hw
    

The same layers in tabular form, each with the tool the operator reaches for to inspect or tune it.

Layer

Where it lives

Inspection / tuning tool

Application

user space

strace -e openat,read,write, perf trace

VFS

kernel

/proc/sys/fs/*, slabtop (dentry / inode caches)

Filesystem driver

kernel

tune2fs, xfs_info, btrfs filesystem usage

Page cache

kernel RAM

free -h, vmstat 1, echo 3 > /proc/sys/vm/drop_caches

Block layer

kernel

iostat -xz 1, blktrace, /sys/block/<dev>/queue/

I/O scheduler

kernel

cat /sys/block/<dev>/queue/scheduler

Device-mapper / md

kernel (optional)

dmsetup, cat /proc/mdstat, lvs, multipath -ll

Device driver

kernel module

lsmod, modinfo, dmesg

Host bus

hardware

lspci, lsusb, lsscsi

Storage media

hardware

smartctl -a, nvme smart-log, hdparm -I

List and Identify#

Command

Effect

lsblk

Tree of block devices, sizes, mount points

lsblk -f

Add filesystem type, label, UUID

lsblk -o NAME,SIZE,MODEL,SERIAL

Custom columns

blkid

UUIDs, labels, FS types per device

blkid /dev/sda1

Just one device

ls -l /dev/disk/by-uuid/

Disks indexed by filesystem UUID

ls -l /dev/disk/by-label/

Disks indexed by FS label

ls -l /dev/disk/by-id/

Stable hardware IDs (preferred for fstab)

ls -l /dev/disk/by-partuuid/

GPT partition UUIDs

lsusb

USB tree (not just storage)

lsusb -t

USB tree with bandwidth and driver info

udevadm info /dev/sdb1

Full udev attribute dump

hwinfo --disk

Detailed hardware report (when installed)

smartctl -a /dev/sda

SMART health and stats

hdparm -I /dev/sda

Identify ATA / SATA disk (model, firmware)

Mount and Unmount#

Command

Effect

mount

Show all current mounts (no args)

findmnt

Tree of mounts (cleaner than mount)

findmnt /mnt/usb

Resolve where a path is mounted

findmnt --target /var/lib/docker

Find the mount that owns a path

mount /dev/sdb1 /mnt/usb

Mount with type auto-detected

mount -t vfat /dev/sdb1 /mnt/usb

Mount with explicit FS type

mount -o ro,noexec /dev/sdb1 /mnt/usb

Mount with options

mount -o remount,rw /

Remount in place with new options

mount -o loop disk.img /mnt

Mount an image file via a loop device

mount --bind SRC DST

Make SRC also visible at DST

mount -t nfs server:/share /mnt/share

NFS mount

mount -t cifs //server/share /mnt/share -o username=op

CIFS / SMB mount

mount -a

Mount everything in /etc/fstab

umount /mnt/usb

Unmount

umount -l /mnt/usb

Lazy unmount (when device is busy)

umount -f /mnt/usb

Force (last resort, e.g. dead NFS)

sync

Flush kernel buffers to disk before unplug

When a mount is “busy” and umount refuses, find the offending process before reaching for -l / -f.

$ sudo lsof +f -- /mnt/data
$ sudo fuser -mv /mnt/data
$ sudo lsof | grep '(deleted)' | head

USB and Removable Media#

USB sticks and SD cards usually appear as /dev/sdX (or /dev/mmcblk* for SD) and auto-mount under /media/<user>/<label> on desktops via udisks2. Headless or scripted work mounts manually under /mnt and runs sync before pulling the device.

Command

Effect

lsblk -o NAME,SIZE,TYPE,MOUNTPOINT

Confirm the new device path

dmesg -w

Watch kernel messages on plug-in

udisksctl status

List drives udisks2 knows about

udisksctl mount -b /dev/sdb1

Mount as the desktop would

udisksctl unmount -b /dev/sdb1

Unmount via udisks2

udisksctl power-off -b /dev/sdb

Spin down + safe-remove the drive

eject /dev/sr0

Eject optical media

eject /dev/sdb

Eject removable USB

sync && udisksctl power-off -b /dev/sdb

Flush, then safe-remove

Burn an ISO to a USB stick.

Warning

These commands overwrite the entire device unconditionally. Confirm the target with lsblk first; writing to the wrong /dev/sdX will destroy the data on it. Always target the whole disk (/dev/sdb), never a partition (/dev/sdb1).

$ lsblk -o NAME,SIZE,MODEL,TRAN,MOUNTPOINT

$ for p in /dev/sdb?*; do sudo umount "$p" 2>/dev/null; done

$ sudo dd if=distro.iso of=/dev/sdb \
        bs=4M conv=fsync oflag=direct status=progress

$ sudo cp distro.iso /dev/sdb && sync
$ sudo sh -c 'cat distro.iso > /dev/sdb' && sync

$ pv distro.iso | sudo dd of=/dev/sdb bs=4M conv=fsync oflag=direct
$ pv distro.iso | sudo tee /dev/sdb >/dev/null && sync

$ sudo usbimager
$ sudo usbimager -i distro.iso -o /dev/sdb -v

$ sync
$ sudo udisksctl power-off -b /dev/sdb

$ sha256sum distro.iso
$ sudo dd if=/dev/sdb bs=4M count=$(stat -c%s distro.iso) iflag=count_bytes \
     | sha256sum

Other common writers.

Tool

Notes

dd

POSIX, ubiquitous, no progress without status=progress

cp / cat / tee

Raw byte copy; simple, no extra package

pv

Pipes a progress bar in front of any writer

usbimager

Lightweight cross-platform CLI + GUI

balena-etcher

Cross-platform GUI; verifies after write

gnome-multi-writer

GNOME GUI; writes one ISO to many USBs in parallel

fedora-mediawriter

Fedora-flavored GUI; can also fetch the ISO

unetbootin

Older GUI; supports persistence

ventoy

Special bootloader; copy multiple ISOs onto one stick

mkusb / mkusb-dus

Debian/Ubuntu-focused safety wrapper around dd

woeusb / woeusb-ng

Specifically for Windows ISOs (NTFS layout)

Notes worth keeping in mind.

  • bs=4M chooses a 4 MiB block size (good throughput on most USB sticks); conv=fsync flushes at the end; status=progress prints throughput while dd runs.

  • Modern hybrid ISOs (Debian, Ubuntu, Arch, Kali, Fedora) are designed for raw write and will boot directly from the stick.

  • For Windows ISOs, use woeusb. A raw dd produces a stick that BIOS / UEFI cannot boot from.

  • For booting multiple ISOs from one stick, use ventoy. It installs a small bootloader, then ISOs are plain files copied to the FAT / exFAT partition.

Persistent Mounts (/etc/fstab)#

Each line is six whitespace-separated fields (device  mountpoint  type  options  dump  pass). Prefer UUID= or LABEL= over /dev/sdX so cabling changes don’t break boot.

/etc/fstab is one mount per line, fields separated by tabs or spaces, # starts a comment.

$ cat /etc/fstab
# <device>        <mountpoint>    <type>  <options>                       <dump> <pass>
UUID=8e0f1c5a...        /          ext4   defaults,noatime                   0      1
UUID=1234-ABCD          /boot/efi  vfat   umask=0077,shortname=winnt         0      2
LABEL=HOME              /home      ext4   defaults,noatime                   0      2
UUID=abcd1234-...       /mnt/data  ext4   defaults,noatime,nofail            0      2
UUID=11112222-...       none       swap   sw                                 0      0
tmpfs                   /tmp       tmpfs  defaults,nosuid,nodev,size=2G      0      0
/dev/.../USB            /mnt/usb   auto   noauto,user,exec                   0      0
nas.lan:/exports/media  /mnt/media nfs    _netdev,rw,noatime                 0      0
//fileserver/share      /mnt/share cifs   credentials=/etc/smb.cred,_netdev  0      0

Device#

What to mount. The operator picks one of several forms; UUID is the safe default on bare metal because cabling and BIOS enumeration can shuffle /dev/sdX between boots.

Form

When to use it

UUID=<uuid>

Preferred. Filesystem UUID from blkid -s UUID -o value /dev/sdX. Survives cabling and controller-order changes.

LABEL=<name>

Filesystem label from e2label / xfs_admin -L / fatlabel. Friendlier than UUID but only stable if you enforce label uniqueness.

PARTUUID=<uuid> / PARTLABEL=<name>

GPT partition identity (not FS identity). Useful when the filesystem will be reformatted but the partition stays put.

/dev/disk/by-id/<id>

Stable hardware ID from ls -l /dev/disk/by-id/. Encodes model + serial, so it follows the physical drive.

/dev/sdX / /dev/nvme0n1pN

Kernel-assigned path. Avoid for permanent mounts; order changes silently on hardware swaps.

host:/path

NFS export (e.g. nas.lan:/exports/media).

//host/share

CIFS / SMB share (note the two leading slashes).

tmpfs / none / proc / sysfs

Pseudo-filesystem; no backing device. none is the conventional placeholder for swap files and bind mounts.

Mountpoint#

The directory where the filesystem becomes visible to userspace.

  • Must exist before mount -a runs (mkdir -p it first).

  • Convention. / for root, /boot, /boot/efi, /home, /var, /srv, /opt, /tmp, /mnt/<name> for ad-hoc mounts, /media/<user>/<label> for desktop auto-mounts. See FHS.

  • none is the placeholder when the entry has no mountpoint semantics (swap files, bind mount sources in some forms).

Type#

The filesystem driver. Either spell out the type (ext4, xfs, btrfs, vfat, exfat, f2fs, ntfs3, nfs, nfs4, cifs, tmpfs, proc, sysfs, overlay, squashfs) or use auto to have the kernel probe the superblock. Specifying the type explicitly is faster and fails loudly when the device is unexpected.

Options#

A comma-separated list of mount flags. Reading these correctly matters because they decide what userspace can do on the mount. defaults expands to rw,suid,dev,exec,auto,nouser,async.

Option

Effect

defaults

rw,suid,dev,exec,auto,nouser,async (the implicit baseline).

ro / rw

Read-only or read-write.

noatime

Skip atime updates on every read. Standard performance win, safe on most workloads.

relatime

Update atime only when older than mtime / ctime (the modern kernel default).

nodiratime

Skip atime on directory reads only.

noexec

Refuse to execute binaries from this mount. Common on /tmp, /var, /dev/shm, removable media.

nosuid

Ignore SUID / SGID bits on this mount.

nodev

Ignore device-special files on this mount.

noauto

Skip on mount -a. Pair with user for opt-in mounts.

user / users

Allow a regular user to mount / unmount the entry.

nofail

Boot continues if the device is missing. Essential for removable / network mounts on headless hosts.

_netdev

Wait for the network before mounting. Required for NFS, CIFS, iSCSI-backed mounts.

x-systemd.automount

Mount lazily on first access (systemd).

x-systemd.requires=<unit>

Wait for the named unit before mounting.

discard

Issue TRIM on every delete. Prefer fstrim.timer for SSDs (less per-write overhead).

compress=zstd (Btrfs)

Transparent compression algorithm.

subvol=@home (Btrfs)

Which subvolume to mount as the root of this entry.

uid=1000,gid=1000,umask=0077 (vfat / exfat / cifs)

Ownership and mode for filesystems without Unix permissions.

credentials=/etc/smb.cred (cifs)

File holding username= and password= so they stay out of fstab.

size=2G (tmpfs)

Upper bound on RAM consumed by the mount.

Operator-relevant combinations.

  • defaults,noatime,nofail for a data disk that might be missing.

  • defaults,noatime,nosuid,nodev,noexec for /tmp, /var, /home, removable media. Hardens against SUID-staged escalations and dropped-binary execution.

  • ro,nosuid,nodev,noexec for an audit-log volume.

Dump#

The legacy dump(8) backup flag.

  • 0 skip (almost always correct on modern systems; dump is retired).

  • 1 include in dump backups.

Leave at 0 unless the host actually runs dump.

Pass#

The fsck order at boot.

  • 0 skip the filesystem entirely (pseudo-filesystems, swap, network mounts).

  • 1 reserved for the root filesystem; checked first.

  • 2 non-root filesystems; checked in parallel after root.

Most operator-managed entries use 2 for ext / xfs data volumes and 0 for everything that has no on-disk fsck path.

Common snippets.

Snippet

Meaning

UUID=<uuid> /mnt/data ext4 defaults 0 2

Typical data mount

LABEL=BOOT /boot vfat defaults 0 2

Mount by FS label

/dev/sdb1 /mnt/usb auto noauto,user 0 0

User-mountable, not on boot

tmpfs /tmp tmpfs defaults,size=2G 0 0

RAM-backed /tmp

//host/share /mnt/share cifs credentials=/etc/smb.cred 0 0

CIFS / SMB share (credentials file is in /etc/smb.cred)

host:/exp /mnt/nfs nfs _netdev,rw 0 0

NFS export

/swapfile none swap sw 0 0

Enable a swap file

/src /dst none bind 0 0

Bind mount

Partition and Format#

These commands change disk contents. Run them only against the device you intend to overwrite. parted and fdisk write on confirmation; mkfs formats unconditionally.

Command

Effect

fdisk -l

List partition tables on all disks

fdisk /dev/sdb

Interactive MBR / GPT partitioning

parted /dev/sdb print

Print partition table

parted -s /dev/sdb mklabel gpt

Create a fresh GPT label

parted -s /dev/sdb mkpart primary ext4 1MiB 100%

Single full-disk partition

sgdisk -p /dev/sdb

Scriptable GPT inspection

gdisk /dev/sdb

GPT-only interactive fdisk

wipefs -a /dev/sdb

Erase all FS / partition signatures

mkfs.ext4 -L data /dev/sdb1

Create ext4 with a label

mkfs.ext4 -m 1 /dev/sdb1

Reserve only 1% for root (raise free space on data disks)

mkfs.xfs /dev/sdb1

Create XFS

mkfs.btrfs /dev/sdb1

Create Btrfs

mkfs.btrfs -d raid1 /dev/sdb /dev/sdc

Btrfs across two disks, mirrored data

mkfs.f2fs -l data /dev/sdb1

F2FS (flash-friendly; SD cards, eMMC)

mkfs.vfat -F 32 /dev/sdb1

FAT32 (USB / EFI)

mkfs.exfat /dev/sdb1

exFAT (cross-platform USB)

mkswap /dev/sdb2 && swapon /dev/sdb2

Format and enable swap

e2label /dev/sdb1 DATA

Set ext-family label

xfs_admin -L DATA /dev/sdb1

Set XFS label

fatlabel /dev/sdb1 DATA

Set FAT label

Repair and Maintenance#

Command

Effect

fsck /dev/sdb1

Check / repair (run on unmounted FS)

fsck.ext4 -f /dev/sdb1

Force-check ext4

xfs_repair /dev/sdb1

Repair XFS (unmounted)

btrfs check /dev/sdb1

Check Btrfs (unmounted)

tune2fs -l /dev/sdb1

Inspect / tune ext-family parameters

tune2fs -m 1 /dev/sdb1

Adjust the reserved-block percentage

tune2fs -L newlabel /dev/sdb1

Relabel an ext volume

resize2fs /dev/sdb1

Grow / shrink ext (after partition change)

xfs_growfs /mnt/data

Grow XFS online (cannot shrink)

btrfs filesystem resize max /mnt

Grow Btrfs online

fstrim -av

TRIM all SSD-backed mounts (run weekly)

systemctl enable --now fstrim.timer

Weekly TRIM via systemd timer

smartctl -t short /dev/sda

Run a short SMART self-test

smartctl -l selftest /dev/sda

Read the SMART self-test log

badblocks -nv /dev/sdb

Non-destructive surface scan

dd if=/dev/zero of=/dev/sdb bs=1M status=progress

Wipe (destructive!)

Capacity#

Block-device tools answer “what disks are here”. Capacity tools answer “where is the space going”. Both questions show up in the first hour of any storage incident.

Command

Effect

df -hT

Free / used per mounted filesystem, with type

df -i

Inode usage (the silent killer when many small files)

du -sh /var/* | sort -h

Top-level usage under a path, sorted ascending

du -h --max-depth=2 /var | sort -h

Two levels deep

du --apparent-size -sh path

Logical size (vs on-disk; handy with sparse files)

ncdu /

Interactive disk-usage browser (install separately)

lsof +f -- /mnt

Processes holding files on a busy mount

fuser -mv /mnt

Same question, simpler output

lsof | grep '(deleted)'

Open-but-deleted files still consuming space

When df says the disk is full but du cannot find the bytes, something is holding a deleted file open. Restart or kill the process; the space frees the moment the last reference closes.

LVM#

LVM (Logical Volume Manager) sits between the block layer and the filesystem. The operator gets a pool of storage (a VG built from one or more PVs) carved into resizable, named pieces (LVs). Filesystems sit on the LVs; growing or shrinking is online.

        flowchart BT
    subgraph BLOCK["Block devices (PVs)"]
        direction LR
        PV1["/dev/sdb<br/><i>pvcreate</i>"]
        PV2["/dev/sdc<br/><i>pvcreate</i>"]
        PV3["/dev/sdd<br/><i>pvcreate</i>"]
    end

    subgraph GROUP["Volume group (vgcreate)"]
        VG[("vg0<br/><i>pool of extents</i>")]
    end

    subgraph LVS["Logical volumes (lvcreate)"]
        direction LR
        LV1["/dev/vg0/root<br/>50 GiB"]
        LV2["/dev/vg0/home<br/>200 GiB"]
        LV3["/dev/vg0/data<br/>1 TiB"]
    end

    subgraph FS["Filesystems"]
        direction LR
        F1["ext4 → /"]
        F2["xfs  → /home"]
        F3["ext4 → /mnt/data"]
    end

    PV1 --> VG
    PV2 --> VG
    PV3 --> VG
    VG --> LV1
    VG --> LV2
    VG --> LV3
    LV1 --> F1
    LV2 --> F2
    LV3 --> F3
    

Command

Effect

pvs, vgs, lvs

One-line summary of PVs, VGs, LVs

pvdisplay, vgdisplay, lvdisplay

Verbose form

pvcreate /dev/sdb

Mark a block device as an LVM physical volume

vgcreate vg0 /dev/sdb /dev/sdc

Pool one or more PVs into a volume group

vgextend vg0 /dev/sdd

Add a PV to an existing VG

vgreduce vg0 /dev/sdd

Drop a PV from a VG (move data first with pvmove)

lvcreate -L 100G -n logs vg0

Carve a 100 GiB LV named logs from vg0

lvcreate -l 100%FREE -n data vg0

Use all remaining free extents

lvextend -L +20G -r /dev/vg0/logs

Grow the LV by 20 GiB; -r resizes the FS in the same call

lvreduce -L -10G /dev/vg0/logs

Shrink (must shrink the FS first; ext4 supports it, XFS does not)

lvrename vg0 old new

Rename an LV

lvremove /dev/vg0/logs

Delete an LV

pvmove /dev/sdb /dev/sdd

Migrate extents from one PV to another, online

Snapshots. LVM snapshots are copy-on-write. Create one before a risky operation, mount it read-only, recover from it if the operation goes wrong.

$ sudo lvcreate -s -L 10G -n root_snap /dev/vg0/root
$ sudo mount -o ro /dev/vg0/root_snap /mnt/snap
$ sudo umount /mnt/snap
$ sudo lvremove /dev/vg0/root_snap

Thin provisioning. Allocate from a thin pool only when blocks are written, so the sum of LV sizes can exceed the pool. The container-storage backend (devicemapper, overlay2 on top of thinpool) is built on this.

$ sudo lvcreate --thinpool tp0 -L 500G vg0
$ sudo lvcreate -V 200G -T vg0/tp0 -n vol1
$ sudo lvcreate -V 200G -T vg0/tp0 -n vol2

End-to-end create and grow.

$ sudo pvcreate /dev/sdb
$ sudo vgcreate vg0 /dev/sdb
$ sudo lvcreate -L 100G -n data vg0
$ sudo mkfs.ext4 /dev/vg0/data
$ sudo mount /dev/vg0/data /mnt/data

$ sudo vgextend vg0 /dev/sdc
$ sudo lvextend -L +50G -r /dev/vg0/data

RAID#

Software RAID (mdadm) combines block devices for redundancy or striping. The kernel exposes the array as a single /dev/mdN; the filesystem above sees one disk.

Command

Effect

mdadm --create /dev/md0 -l 1 -n 2 /dev/sd[bc]

Create a RAID-1 mirror across two disks

mdadm --create /dev/md0 -l 5 -n 3 /dev/sd[bcd]

RAID-5 across three disks

cat /proc/mdstat

Live array state (rebuild progress, faulty members)

mdadm --detail /dev/md0

Verbose array status

mdadm --examine /dev/sdb

Read the array superblock from a member device

mdadm --add /dev/md0 /dev/sde

Add a spare to the array

mdadm --fail /dev/md0 /dev/sdb

Mark a member faulty (testing or replacement prep)

mdadm --remove /dev/md0 /dev/sdb

Drop the member from the array

mdadm --grow /dev/md0 -n 4

Reshape the array (e.g. add a disk to a RAID-5)

mdadm --stop /dev/md0

Stop the array (unmount first)

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Persist the array so it assembles on boot

Rebuild progress lives in /proc/mdstat; echo check > /sys/block/md0/md/sync_action triggers a periodic scrub.

LUKS#

cryptsetup manages LUKS-encrypted block devices. The decrypted view appears as /dev/mapper/<name>; the filesystem above is unaware of the encryption.

Command

Effect

cryptsetup luksFormat /dev/sdb1

Initialize a LUKS volume (destructive)

cryptsetup luksOpen /dev/sdb1 vault

Unlock; map at /dev/mapper/vault

cryptsetup luksClose vault

Lock and remove the mapping

cryptsetup luksDump /dev/sdb1

Show keyslots, cipher, MK digest

cryptsetup luksAddKey /dev/sdb1

Add another passphrase or keyfile (up to 8 slots)

cryptsetup luksRemoveKey /dev/sdb1

Revoke one key

cryptsetup resize vault

Re-read the underlying device size after grow

cryptsetup benchmark

Cipher / hash speed for this CPU

For full encryption coverage (per-archive, per-file, per-disk), see Encryption.

Snapshots#

Three sources of point-in-time copies on Linux. Each one freezes a different layer of the stack.

Source

Notes

LVM

lvcreate -s makes a copy-on-write snapshot of an LV. Lives at the device-mapper layer; works for any FS on top. Sized in advance (it fills as the origin diverges).

Btrfs

btrfs subvolume snapshot SRC DST is instant, takes no extra space until divergence. btrfs send | btrfs receive replicates between filesystems.

ZFS

zfs snapshot pool/dataset@name listed under .zfs/snapshot, rolled back with zfs rollback, replicated with zfs send.

For consistent snapshots while the filesystem is live, freeze writes first.

$ sudo fsfreeze --freeze /mnt/data
$ sudo lvcreate -s -L 5G -n data_snap /dev/vg0/data
$ sudo fsfreeze --unfreeze /mnt/data

Swap#

Swap is space the kernel uses when RAM is tight. It can be a partition, a separate file, or a zram device backed by RAM compression. The operator picks based on host profile (laptops favor swap files, dense servers often skip swap entirely).

Command

Effect

swapon --show

List active swap (file, partition, priority, used)

free -h

RAM and swap totals at a glance

mkswap /dev/sdb2

Format a partition as swap

swapon /dev/sdb2

Enable a swap area

swapoff /dev/sdb2

Disable a swap area

sysctl vm.swappiness

Read or tune how aggressively the kernel swaps (0-200; default 60)

Swap-file pattern, the modern default on systems that need swap at all.

$ sudo fallocate -l 4G /swapfile
$ sudo chmod 600 /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile
$ echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Encrypted swap is required for hibernation. Either put swap on a LUKS-backed partition, or use a randomly-keyed swap device (lost across reboots, no hibernation).

Loop devices and image files#

A loop device exposes a regular file as a block device. The operator uses this to mount disk images, build VM images, test filesystem layouts without touching real hardware, and assemble sparse storage that grows on demand.

Command

Effect

losetup -fP --show disk.img

Attach the next free loop device, scan for partitions

losetup -a

List all attached loop devices

losetup -d /dev/loop0

Detach

losetup -c /dev/loop0

Re-read partition table after image edit

mount -o loop disk.img /mnt

One-shot loop mount (no explicit losetup)

fallocate -l 10G disk.img

Pre-allocate a 10 GiB sparse file

truncate -s 10G disk.img

Same idea, holes only (no allocation)

dd if=/dev/zero of=disk.img bs=1M count=1024

Create a fully written 1 GiB image (slow, predictable)

End-to-end build of a partitioned image.

$ fallocate -l 4G disk.img
$ sudo losetup -fP --show disk.img
$ sudo parted -s /dev/loop0 mklabel gpt mkpart primary ext4 1MiB 100%
$ sudo mkfs.ext4 /dev/loop0p1
$ sudo mount /dev/loop0p1 /mnt
$ sudo umount /mnt
$ sudo losetup -d /dev/loop0

Bind mounts#

A bind mount exposes a directory under a second path without copying data. The kernel routes I/O against either path to the same underlying inodes. Used to remap paths in chroots, containers, and service-isolation setups.

Command

Effect

mount --bind /src /dst

Make /src visible at /dst

mount --rbind /src /dst

Recursive bind (nested mounts come too)

mount -o remount,ro,bind /dst

Remount the bind read-only

mount --make-private /

Stop mount events propagating across namespaces

mount --make-shared /

Default; mount events propagate

In /etc/fstab a bind looks like /src /dst none bind 0 0.

Quotas#

Per-user, per-group, and (on XFS / ext4 / Btrfs) per-project disk quotas. Cap how much a runaway tenant can consume on a shared filesystem.

Command

Effect

mount -o usrquota,grpquota /dev/sdb1 /mnt

Enable user / group quotas at mount

quotacheck -cum /mnt

Build the quota database (offline, ext)

quotaon /mnt

Activate quotas

edquota -u alice

Edit a user’s quota interactively

setquota -u alice 1G 1.2G 0 0 /mnt

Soft / hard block quota for one user

repquota -a

Report quotas across all enabled filesystems

xfs_quota -x -c 'limit bsoft=10g bhard=12g project1' /mnt

XFS project quota (no daemon needed)

Performance and I/O#

When a host feels slow and the question is “is the disk the bottleneck”, these tools answer it. Keep iostat -xz 1 and fio close to hand.

Tool

Scope

What it shows

iostat

per-device

Throughput, IOPS, await, util; refresh each second.

iotop

per-process

Top-style live view of disk read / write rates.

pidstat -d

per-process

Read / write rates summarized per PID.

dstat

host

Combined disk + network throughput on one screen.

biolatency-bpfcc

block layer

Latency histogram of block-layer I/O (bcc-tools).

biosnoop-bpfcc

block layer

Per-I/O latency trace, one line per request.

blktrace / blkparse

block layer

Full event-level trace of every I/O on a device.

fio

synthetic

Configurable benchmark (rand / seq, read / write).

Worked invocations.

$ iostat -xz 1
$ sudo iotop -oP
$ pidstat -d 1
$ dstat -dn 1
$ sudo biolatency-bpfcc
$ sudo biosnoop-bpfcc
$ sudo blktrace -d /dev/sdb -o trace && blkparse trace
$ fio --name=t --rw=randread --bs=4k --size=1G \
      --runtime=30 --time_based --filename=/dev/sdb

I/O scheduler and queue tuning sit under /sys/block/<dev>/queue/.

Path

Meaning

scheduler

Active scheduler (one of mq-deadline, kyber, bfq, none)

nr_requests

Maximum in-flight requests in the I/O queue

read_ahead_kb

Sequential read-ahead window (256 KiB default)

rotational

1 for spinning disks, 0 for SSDs / NVMe

rq_affinity

CPU on which I/O completions are processed

Common defaults. mq-deadline for SATA SSD, none for NVMe (the device queue is large enough), bfq for laptops where interactive responsiveness matters more than throughput.

Enterprise storage#

The block devices on a serious server are rarely a single disk attached over SATA. They reach the host through SAN or NAS protocols, multipath wiring, NVMe over Fabrics, or a hardware RAID controller. Each layer adds its own tooling.

Multipath I/O. When the same LUN is reachable over two HBAs, device-mapper multipath presents a single /dev/mapper/mpath* device; the FS uses one logical path while the kernel fails over underneath.

Command

Effect

multipath -ll

List active multipath devices and their paths

multipathd show paths

Live path state from the daemon

multipath -F

Flush all unused multipath maps

/etc/multipath.conf

Aliases, blacklist, path-checker, failover policy

iSCSI (initiator). Attach a remote LUN as a local block device.

$ sudo iscsiadm -m discovery -t st -p portal.example.com
$ sudo iscsiadm -m node -T iqn.2025-01.example:lun1 -l
$ lsblk
$ sudo iscsiadm -m node -T iqn.2025-01.example:lun1 -u

NVMe. nvme-cli reads namespaces, SMART, and per-controller state.

Command

Effect

nvme list

Controllers and namespaces present

nvme id-ctrl /dev/nvme0

Controller identify (model, serial, features)

nvme smart-log /dev/nvme0

Wear, temperature, error counters

nvme format /dev/nvme0n1 -s 1

Crypto-erase a namespace (verify before running)

nvme fw-log /dev/nvme0

Firmware slots and active slot

Hardware RAID. Vendor utilities read the controller’s view; the OS sees only the exposed virtual disks.

Tool

Vendor / use

storcli / MegaCli

LSI / Broadcom / Avago MegaRAID

ssacli / hpssacli

HPE Smart Array

perccli

Dell PERC (rebadged MegaRAID)

arcconf

Adaptec / Microsemi

mvcli

Marvell

Forensics and recovery#

When a disk is dying, deleted files matter, or the question is “what was on this drive before someone started writing to it”, forensic tooling separates the evidence from the changes.

Command

Effect

ddrescue /dev/sdb image.img map.log

dd for failing disks; retries bad sectors, resumable via the map

dd if=/dev/sdb of=image.img bs=4M conv=noerror,sync

Plain-dd image (use ddrescue when the source is faulty)

sha256sum /dev/sdb

Hash a whole block device for chain-of-custody

testdisk /dev/sdb

Partition-table recovery; rebuild lost partitions

photorec

Carve files from raw blocks (filesystem-agnostic)

extundelete --restore-all /dev/sdb1

Recover deleted files from ext3 / ext4 (best-effort)

fls /dev/sdb1 / icat / istat

Sleuthkit, per-inode forensic inspection

autopsy

Web GUI for sleuthkit

For acquisition, mount the source read-only or via a write-blocker; work from the image, never the original disk.

Advanced unlock#

LUKS by default takes a passphrase. For unattended boot, bind the unlock to a hardware or network secret so the host can come up without a human at the console.

Mechanism

Notes

TPM2

systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=7 /dev/sdb1 binds the unlock to the platform’s secure-boot state.

Network-bound (NBDE)

Tang server + Clevis client; the disk unlocks only when the host can reach the Tang server.

Smartcard / FIDO2

systemd-cryptenroll --fido2-device=auto binds to a hardware token.

Common Tasks#

The recipes the operator runs on target.

Set up a fresh USB stick from scratch.

$ lsblk -f
$ sudo wipefs -a /dev/sdb
$ sudo parted -s /dev/sdb mklabel gpt
$ sudo parted -s /dev/sdb mkpart primary 1MiB 100%
$ sudo mkfs.exfat -n DATA /dev/sdb1
$ sudo mkdir -p /mnt/usb && sudo mount /dev/sdb1 /mnt/usb

Persist a new data disk in ``/etc/fstab`` so it mounts on boot.

$ uuid=$(sudo blkid -s UUID -o value /dev/sdb1)
$ echo "UUID=$uuid /mnt/data ext4 defaults,nofail 0 2" | sudo tee -a /etc/fstab
$ sudo mkdir -p /mnt/data
$ sudo mount -a

Free space when ``df`` and ``du`` disagree.

$ df -h
$ sudo du -sh /var/* | sort -h
$ sudo lsof | grep '(deleted)' | head
$ sudo systemctl restart suspect.service

Grow a logical volume and its filesystem in one step.

$ sudo vgextend vg0 /dev/sdc
$ sudo lvextend -L +50G -r /dev/vg0/data

Snapshot a live filesystem before a risky write.

$ sudo fsfreeze --freeze /mnt/data
$ sudo lvcreate -s -L 5G -n data_snap /dev/vg0/data
$ sudo fsfreeze --unfreeze /mnt/data

Image a failing disk with retries and a recovery map.

$ sudo ddrescue -d -r 3 /dev/sdb image.img map.log
$ sudo ddrescue -d -r 5 -R /dev/sdb image.img map.log
$ sha256sum image.img > image.sha256

Find what is using disk I/O right now.

$ iostat -xz 1
$ sudo iotop -oP
$ sudo biolatency-bpfcc

References#

  • man 8 lsblk, man 8 blkid, man 8 findmnt, man 8 udevadm, man 1 lsof, man 1 fuser (inspection).

  • man 8 mount, man 8 umount, man 5 fstab, man 1 udisksctl, man 8 fsfreeze (mounting and bind mounts).

  • man 8 parted, man 8 fdisk, man 8 gdisk, man 8 wipefs, man 8 mkfs, man 8 mkswap, man 8 swapon (partitioning and format).

  • man 8 fsck, man 8 tune2fs, man 8 resize2fs, man 8 xfs_growfs, man 8 fstrim, man 8 badblocks (repair).

  • man 8 lvm, man 8 pvcreate, man 8 vgcreate, man 8 lvcreate, man 8 lvextend, man 8 pvmove (LVM).

  • man 8 mdadm, man 4 md (software RAID).

  • man 8 cryptsetup, man 1 systemd-cryptenroll (LUKS and unlock).

  • man 8 losetup, man 1 fallocate, man 1 truncate (loop and images).

  • man 1 quota, man 8 quotacheck, man 8 quotaon, man 8 edquota, man 8 xfs_quota (quotas).

  • man 1 iostat, man 1 iotop, man 1 pidstat, man 1 fio, man 8 blktrace, man 8 blkparse (performance).

  • man 8 multipath, man 8 multipathd, man 5 multipath.conf, man 8 iscsiadm, man 1 nvme (enterprise storage).

  • man 1 ddrescue, man 8 testdisk, man 1 photorec, man 1 sha256sum (forensics).

  • man 8 smartctl, man 8 hdparm (drive health).

  • Encryption for the encryption stack (LUKS, fscrypt, age / gpg).

  • FHS for where mountpoints conventionally live in the tree.

  • Permissions for nosuid / noexec / nodev mount options.

  • Filesystems for ext4 / XFS / Btrfs / ZFS / FAT comparison.

  • References for per-OS filesystem layouts.

  • LVM2 documentation

  • mdadm wiki

  • cryptsetup / LUKS2 spec