LVM and RAID¶

Practical 4.5 lvm-and-raid-annex.pdf · PDF

Friday, 18:00. Joan, from design at Tramuntana Apps, tries to save a video and gets No space left on device. Last week you gave design a partition of 2 GB. It is full. And partitions do not stretch. What now?

The old answer: buy a bigger disk, copy everything across, switch the server off. Tonight. In Mounting file systems you attached partitions to folders. Today you meet two tools that put several disks behind one folder: one to make space flexible, one to survive a dead disk.

LVM: a binder instead of a wall¶

LVM (Logical Volume Manager) puts a layer between the disks and the file systems, so space can be moved and grown.

A partition is a wall of bricks: once it is built, it stays where it is. LVM works like a ring binder. You put packs of blank sheets into it, and you place dividers wherever you want. Need a bigger section? Add another pack and move the divider. Nobody takes the section out to do it.

flowchart TB
    subgraph PVs["Physical volumes (the packs of sheets)"]
        VDC["<b>/dev/vdc</b><br>1 GiB disk"]
        VDD["<b>/dev/vdd</b><br>1 GiB disk"]
    end
    VG["<b>vg_tram</b><br>volume group: the binder"]
    LV["<b>lv_media</b><br>logical volume: one section"]
    FS["<b>ext4</b><br>file system"]
    MNT["<b>/srv/media</b><br>the folder Joan uses"]
    VDC --> VG
    VDD -->|"added later with vgextend"| VG
    VG -->|"lvcreate cuts a section"| LV
    LV -->|"mkfs.ext4"| FS
    FS -->|"mount"| MNT

The device you format and mount is /dev/vg_tram/lv_media. In df it appears as /dev/mapper/vg_tram-lv_media: two names, one volume.

Build a volume and fill it¶

In deb-lab you add four new virtual disks of 1 GiB (they appear as vdc, vdd, vde, vdf; use lsblk to check yours). Then you install the tools and build the three layers on the first disk:

sudo apt install lvm2 mdadm
sudo pvcreate /dev/vdc
sudo vgcreate vg_tram /dev/vdc
sudo lvcreate -n lv_media -L 800M vg_tram
sudo mkfs.ext4 /dev/vg_tram/lv_media
sudo mkdir /srv/media
sudo mount /dev/vg_tram/lv_media /srv/media
sudo chown $USER: /srv/media
df -h /srv/media

Each step answers with one line: Physical volume "/dev/vdc" successfully created., Volume group "vg_tram" successfully created, Logical volume "lv_media" created. and df -h shows a volume of about 770M.

Why 800M and not 1G? A 1 GiB disk gives 1020.00m to the group, not 1024: LVM keeps a little for its own notes.

Now play Joan's Friday. A 700 MB video fits; the next 200 MB one does not:

cd /srv/media
head -c 700M /dev/urandom > campaign.mp4
head -c 200M /dev/urandom > more.mp4

head: error writing 'standard output': No space left on device

Grow it without stopping¶

Add the second disk to the binder, then move the divider:

sudo vgextend vg_tram /dev/vdd
sudo vgs
sudo lvextend -r -L +1G vg_tram/lv_media
df -h /srv/media

• vgextend adds vdd to the group. vgs now shows #PV 2 and about 1.21g free (VFree).
• lvextend -L +1G grows the logical volume by 1 GB. The -r also resizes the file system inside it.
• df -h jumps to about 1.8G, and the 200 MB video fits.

Did you unmount? Did you copy anything? Did Joan have to stop? No, three times. lsblk now shows vg_tram-lv_media under both vdc and vdd.

RAID: several disks, one volume¶

In 1988, at the University of California, Berkeley, David Patterson, Garth Gibson and Randy Katz published A Case for Redundant Arrays of Inexpensive Disks. Their idea: instead of one huge, expensive mainframe disk, use many cheap PC disks together.

There was a catch. With many disks, one of them fails much more often. So some of the space must be redundant: extra information that lets you survive a dead disk. The paper described several ways to do it and called them levels. Today the "I" in RAID is usually read as independent.

RAID (Redundant Array of Independent Disks) combines several disks into one volume, using redundancy so it can keep working when a disk dies.

Stay with paper: the disks are notebooks, and you are writing a story across them. Take n disks of size S:

Parity in one minute¶

You know binary. Take two data pages and add them bit by bit without carry: same bits give 0, different bits give 1. Store the result on a third disk.

disk A   1 0 1 0
disk B   0 1 1 0
parity   1 1 0 0      <- stored on disk C

Now disk B dies. Add A and the parity the same way: 1010 and 1100 give 0110. Disk B is back. That is how RAID 5 survives one dead disk with only one extra disk of space.

Choosing a level¶

Tramuntana's server has four bays and 4 TB disks. They need at least 10 TB and must survive one dead disk. RAID 0 gives 16 TB but survives nothing. RAID 1, 6 and 10 give 8 TB or less. RAID 5 gives (4 − 1) × 4 = 12 TB and survives one disk: that is the one.

A mirror, a dead disk, a deleted file¶

Aina, from admin, wants the invoices to survive a dead disk. You build a RAID 1 with mdadm, the Linux tool for software RAID, on the two remaining disks:

sudo mdadm --create /dev/md0 --level=1 \
  --raid-devices=2 /dev/vde /dev/vdf

The \ at the end of a line means "the command goes on". mdadm asks two questions: about a bitmap (press Enter) and Continue creating array [y/N]? (type y). Then you format, mount and save an invoice:

cat /proc/mdstat
sudo mkfs.ext4 /dev/md0
sudo mkdir /srv/admin
sudo mount /dev/md0 /srv/admin
sudo chown $USER: /srv/admin
echo '1200 EUR' > /srv/admin/invoice-001.txt
df -h /srv/admin

df says about 988M: two disks of 1 GiB, room for one. The second disk is the copy.

Reading /proc/mdstat¶

The file /proc/mdstat is the array's health report:

md0 : active raid1 vdf[1] vde[0]
      ... [2/2] [UU]

Now kill a disk on purpose, read the invoice, and replace the disk:

sudo mdadm /dev/md0 --fail /dev/vdf
cat /proc/mdstat
cat /srv/admin/invoice-001.txt
sudo mdadm /dev/md0 --remove /dev/vdf
sudo mdadm /dev/md0 --add /dev/vdf
cat /proc/mdstat

After --fail you see [U_], and the invoice still prints. After --add, run cat /proc/mdstat several times quickly: the rebuild of 1 GiB takes under ten seconds, then [UU] again.

The other accident¶

Pau, the intern, cleans the wrong folder:

rm /srv/admin/invoice-001.txt
ls /srv/admin
cat /proc/mdstat

The array is perfectly healthy, [UU]. And the invoice is on neither disk. You said "delete", and the mirror deleted it on both disks at the same moment. RAID copies your mistakes as fast as your work.

Clean up¶

The four disks must end empty: one becomes the backup disk in Backups. Undo everything, top to bottom:

cd ~
sudo umount /srv/admin /srv/media
sudo mdadm --stop /dev/md0
sudo mdadm --zero-superblock /dev/vde /dev/vdf
sudo lvremove -y vg_tram/lv_media
sudo vgremove vg_tram
sudo pvremove /dev/vdc /dev/vdd
sudo rmdir /srv/admin /srv/media
lsblk

lsblk must show vdc to vdf with nothing under them. All the steps, with space for your evidence, are on the Practical 4.5 sheet.

Key takeaways¶

• LVM has three layers: physical volumes (disks) go into a volume group, and you cut logical volumes from it to format and mount.
• vgextend + lvextend -r grow a mounted volume with a new disk, without unmounting.
• LVM gives flexibility, not safety: a volume spread over two disks breaks if either dies.
• RAID uses redundancy to survive dead disks: RAID 1 mirrors, RAID 5 and 6 use parity, RAID 0 has none.
• /proc/mdstat shows the array's health: [UU] is healthy, [U_] is degraded but still working.
• RAID answers one question only: what happens when a disk dies. A deleted file is deleted on every disk.

Quick check¶

Your mirror did its job perfectly, and the invoice is still gone. A mirror is not a copy in another place or another moment. Next: what really protects your data — Backups.