Setting up a large disk array in Linux through LVM

A very large disk array (hardware RAID) combined with LVM (Logical Volume Management) will give you a vast amount of flexibility. Without LVM, you’ll have to reboot the entire machine to realize the new space when adding RAID members. Using LVM, you can take advantage of OLE (On Line Expansion) and expand your existing volumes without rebooting. In this post, I discuss the creation of the initial VG (Volume Group), PV (Physical Volume) and LV (Logical Volume) which compose LVM. I’ll discuss OLE later in another post.

In this example, a large RAID array exists at /dev/sdb. There are no partitions on this disk; it is freshly created in the RAID card’s BIOS.

root@rj04:~# fdisk -l

...snip...

Disk /dev/sdb: 2977.4 GB, 2977474543616 bytes
255 heads, 63 sectors/track, 361990 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

1. Create a new PV (Physical Volume) on the new disk. This is the ‘top layer’ of LVM:

pvcreate /dev/hdb

2. Create a new VG (Volume Group) inside the new PV. This is the ‘middle layer’ of LVM:

vgcreate vg0 /dev/hdb

3. Create a new LV (full size). This is the ‘bottom layer’ of LVM where you can put your filesystems:

lvcreate -n lvol0 vg0 -L 2.70T

4. Create and format the filesystem. Remember to choose the correct block size (4k block size allows up to 8Tb size, which is enough for the max our array will expand to):

mkfs -t ext3 -m 0 -v -b 4096 /dev/vg0/lvol0

5. Add mount point, update fstab, and mount the new disk:

mkdir /data ## Create mount
echo "/dev/vg0/lvol0  /data   ext3    defaults        0       2" >> /etc/fstab ## Make filesystem aware of the mount
mount /data  ## Actually mount the disk

6. Verify your work:

root@rj04:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
...snip...
/dev/mapper/vg0-lvol0
                      2.7T  202M  2.7T   1% /data

2 Responses to Setting up a large disk array in Linux through LVM

  1. makj says:

    and once you have that configuration, what if you change the physical disks replacing them for bigger ones?

    the adaptec card rebuilds the RAID perfectly, leaving some spare space unused, now… how can I add that space to my LVM??

    i have an Adaptec3805 with 4+4 disks (in 2 channels), 4 per logical drive in RAID5
    initially, they were 4x250Gb disks, now they’re 4×500 (each channel)

    on top of that, i have 2 PV joined in a single VG and then mapped in 5 LV