XenServer 6.2 with Software RAID

Posted on by Randy

Historically XenServer does not support software RAID out-the-box, and this is unchanged in the latest 6.2 release. We can convert it to RAID after installation.

First we set up the 2nd disk by copying the partition tables whilst enabling a degraded RAID on it, then copy data over with custom initrd.

sgdisk -R/dev/sdb /dev/sda  # Replicate partion table from /dev/sda to /dev/sdb with unique identifier
sgdisk --typecode=1:fd00 /dev/sdb
sgdisk --typecode=2:fd00 /dev/sdb
sgdisk --typecode=3:fd00 /dev/sdb
# Sleep 5 seconds here if you script this...
yes|mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 missing # Create md0 (root)
yes|mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb2 missing # Create md0 (swap)
yes|mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sdb3 missing # Create md0 (storage)
# Sleep 5 seconds here if you script this...
mkfs.ext3 /dev/md0 # Create root FS
mount /dev/md0 /mnt # Mount root FS
cp -xR --preserve=all / /mnt # Replicate root files
sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/etc/fstab # Update fstab for new RAID device
mkdir /mnt/root/initrd-raid && cd /mnt/root/initrd-raid
mkinitrd -v --fstab=/mnt/etc/fstab /mnt/root/initrd-raid/initrd-`uname -r`-raid.img `uname -r`
zcat initrd-`uname -r`-raid.img | cpio -i
sed -i 's/raidautorun \/dev\/md0/raidautorun \/dev\/md0\nraidautorun \/dev\/md1\nraidautorun \/dev\/md2/' init
rm -f initrd-`uname -r`-raid.img
find . -print | cpio -o -Hnewc | gzip -c > /mnt/boot/initrd-`uname -r`-raid.img
rm -f /mnt/boot/initrd-2.6-xen.img
ln -s initrd-`uname -r`-raid.img /mnt/boot/initrd-2.6-xen.img
sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/boot/extlinux.conf
cat /mnt/usr/share/syslinux/gptmbr.bin > /dev/sdb
cd /mnt && extlinux  --raid -i boot/
sgdisk /dev/sda --attributes=1:set:2
sgdisk /dev/sdb --attributes=1:set:2
cd && umount /dev/md0
sync
reboot

Now, make sure BIOS forces booting from 2nd disk. This is VERY important! After reboot, finish off:

sgdisk -R/dev/sda /dev/sdb  # Replicate partion table from /dev/sdb to /dev/sda with unique identifier
mdadm -a /dev/md0 /dev/sda1
mdadm -a /dev/md1 /dev/sda2
mdadm -a /dev/md2 /dev/sda3  # If this command gives error, you need to forget/destroy an active SR first
mdadm --detail --scan > /etc/mdadm.conf
xe sr-create content-type=user device-config:device=/dev/md2 host-uuid=[host_uuid] name-label="Local Storage" shared=false type=lvm

Before going into production, try booting from each disk (with the other removed from the boot priorities), then restore the boot priority to normal.

Be careful with Xenserver patches, especially any patch which requires a reboot – if the initrd image is rewritten (for example, kernel updated), you need to carefully rebuild the initrd for RAID support again which is NOT covered in this article.

EXT and LVM Local Storage in XenServer

Posted on by Randy

On a new XenServer deployment EXT storage is default. Although EXT is nice (ability to manipulate raw VHD files and use sparse storage), LVM is faster and more stable. Here is how to switch between storage modes (be sure to back up all data first, as this will wipe everything in the repo):

Use ‘xe sr-list’ and ‘xe pbd-list’ to find your local storage SR and it’s PBD. Also note the physical device number / partition in the PBD, so you can re-create it on the same device later on.

Unplug the active SR’s PBD (physical block device):

xe pbd-unplug uuid=<pbd_uuid>

Destroy the SR:

xe sr-destroy uuid=<sr_uuid>

Create a new SR of the specified type and location:

xe sr-create content-type=user device-config:device=/dev/<device> host-uuid=<host_uuid> name-label="Local Storage" shared=false type=<lvm|ext>

You can verify the SR again with ‘xe sr-list’ and it should already be in XenCenter as the default.

Removing RAID metadata

Posted on by Randy

Sometimes a hardware RAID controller or fakeraid (BIOS) can leave metadata that makes it impossible to install Windows or Linux, or it installs correctly, but causes a kernel panic or a 0xb7 blue screen error on the first boot. The only method I could find to delete the metadata *quickly* is to zero out the last 512KB of data on the disk using the following command:

dd if=/dev/zero of=$YOUR_DEV bs=512 seek=$(( $(blockdev --getsz $YOUR_DEV) - 1024 )) count=1024

Replace $YOUR_DEV with the physical device, such as /dev/sda

You could just zero the whole disk, but that could take hours. This command executes in less than a second.

Syncing User Filesystems Between Two Cpanel Servers

Posted on by Randy

After doing a WHM multi-account copy, you may want to refresh the users home directories (including email and web files). This one liner uses rsync over SSH to do a quick sync of only changed files, emails, and website files. Run these commands on the NEW server.

First you need to create a list of domains in a text file ‘domains.txt’. Replace 1.2.3.4 with the remote hostname or IP.

while read i ; do rsync -aHxv root@1.2.3.4:/home/$(/scripts/whoowns $i)/ /home/$(/scripts/whoowns $i) ; done < domains.txt

Alternatively, you can use a list of users instead of domain names (use this method if there are multiple domains per user):

while read i ; do rsync -aHxv root@1.2.3.4:/home/$i/ /home/$i ; done < users.txt

Syncing All MySQL Databases Between two Cpanel Servers

Posted on by Randy

After you do a WHM multi-site transfer, you may want to resync all the MySQL databases before doing a DNS change. Here’s a quick and easy way to sync each Cpanel user’s databases on the new Cpanel server. After manually setting up a remote mysql root user on the original server (and opening a firewall port if needed), run this command on the NEW server.

cat /var/cpanel/databases/dbindex.db | grep -E ': .+' | awk '{FS=": ";print $1}' |[ \t]+|:)/, "")};1' | xargs -L 1 /root/sync_db.sh <remote_host> <remote_root_pass>

If you only want to sync (or resync) one or two databases, you can copy /var/cpanel/databases/dbindex.db to a different location and edit it as needed.

Here’s the bash sync script required for this command:

http://djlab.com/stuff/sync_db.sh

MySQL Relationships (JOIN explained)

Posted on by Randy

The visual diagrams still confuse me after so many years and I always forget…so here it is:

INNER JOIN gets all records from one table that have some related entry in a second table

LEFT JOIN gets all records from the LEFT linked table but if you have selected some columns from the RIGHT table, if there is no related records, these columns will contain NULL

RIGHT JOIN is like the above but gets all records in the RIGHT table

FULL JOIN gets all records from both tables and puts NULL in the columns where related records do not exist in the opposite table