Cpanel/Exim with external spam filtering appliances

We have a unique setup at my work.  All of the webhosting appliances have built in spam-filtering software.  However, we have external spam-filtering appliances that are 99% more effective.  It makes sense to keep CPU intensive spam filtering off of the webhosting servers which should be focusing their efforts on serving up PHP and MySQL queries.

It seems spammers are getting more and more savvy (or should I say desperate?) and have developed several means to bypass spam-filters.  One of those which seems to be getting more popular lately, is to completely ignore a domain’s MX records and deliver spam directly to the website’s IP address.  For most virtual hosting and some dedicated hosting setups using an external or 3rd party spam-filtering service, this can be quite effective.  Lately it has been plaguiing several of our clients who are reporting an increase in spam, but cannot find copies of it in the spam filter interface.

To make a long story short, I was able to coerce Exim into rejecting these direct delivery attempts.

Log into WHM and click “Exim Configuration Editor”

Then click “Advanced Editor”

Add the following to the empty box at the top:

domainlist relay_domains = lsearch;/etc/secondarymx

Now we need to tell Exim to be nice to our spam-filtering appliances (don’t reject the clean mail coming in from them) by adding a list of their IPs in to /etc/alwaysrelay.

Restart Exim, done!

Cpanel/Exim ‘535 Incorrect authentication data’ when sending mail

So this issue has been plaguing some customers ever since I started migrating sites from older Ensim boxes to new Cpanel servers. It seems 50% of the email users (or more, haven’t heard from everyone yet) is getting authentication errors on their outbound mail – even in Webmail. So the first solution I’ve been telling people for the past few weeks is to remove then re-add the email account in Cpanel. It works…but it’s like using an axe instead of a scalpel. For those who rely solely on webmail, this cannot be done as all their email would be lost.

So I happened along this thread.

The problem ended up being the permissions of everyone’s /home/<user>/etc/<domain>/shadow files. For some reason the Ensim migration script leaves them as 0600 when they need to be 0640. This single command fixed all 300+ domains in one fell swoop:

chmod 640 /home/*/etc/*/shadow

So rather than use a single axe, I took 300+ scalpels at once the problem.

Building and Installing LiquidSoap on CentOS5

So I spend nearly half of an otherwise productive day fighting dependency hell with LiquidSoap.  I haven’t tested it yet but here’s a cleaned up and organized command history of how it happened:


## Liquid soap install

cd /usr/src

rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

yum install automake pkgconfig ncurses ncurses-devel ocaml ocaml-camlp4 ocaml-labltk libtheora libtheora-devel libao libao-devel /
portaudio portaudio-devel alsa-lib alsa-lib-devel glib2-devel avahi avahi-devel /
libmad libmad-devel taglib taglib-devel lame lame-devel faac faac-devel faad2 faad2-devel /
ladspa ladspa-devel soundtouch soundtouch-devel doxygen

wget http://linuxsoft.cern.ch/epel/5/x86_64/ocaml-findlib-devel-1.1.2pl1-16.el5.x86_64.rpm
wget http://linuxsoft.cern.ch/epel/5/x86_64/ocaml-findlib-1.1.2pl1-16.el5.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-devel-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-lib-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-lib-devel-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-module-x11-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-lib-glib2-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-module-zeroconf-0.9.5-5.el5.kb.x86_64.rpm
wget http://centos.karan.org/el5/extras/testing/x86_64/RPMS/pulseaudio-lib-zeroconf-0.9.5-5.el5.kb.x86_64.rpm

rpm -ivh *.rpm

### Misc audio stuff, libshout, ect
### Compile libshout/speex from source because packaged version of speex is too old for liquidsoap.

wget http://downloads.xiph.org/releases/speex/speex-1.2rc1.tar.gz
tar -zxf speex-1.2rc1.tar.gz
cd speex-1.2rc1
./configure
make install

cd ..

wget http://downloads.us.xiph.org/releases/libshout/libshout-2.2.2.tar.gz
tar -zxf libshout-2.2.2.tar.gz
cd libshout-2.2.2
./configure
make
make install

cd ..

wget http://www.grame.fr/~letz/jack-1.9.2.tar.bz2
tar -jxf jack-1.9.2.tar.bz2
./waf configure
./waf build
./waf install

cd ..

wget http://voxel.dl.sourceforge.net/sourceforge/gmerlin/gavl-1.1.0.tar.gz
tar -zxf gavl-1.1.0.tar.gz
cd gavl-1.1.0
./configure
make install

cd ..

### OCAML stuff

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-7.6.tar.gz
tar -zxf pcre-7.6.tar.gz
cd pcre-7.6
./configure
make install

cd ..

wget http://voxel.dl.sourceforge.net/sourceforge/savonet/ocaml-ladspa-0.1.1.tar.gz
tar -zxf ocaml-ladspa-0.1.1.tar.gz
cd ocaml-ladspa-0.1.1
./configure
make install

cd ..

wget http://download.camlcity.org/download/ocamlnet-2.2.9.tar.gz
tar -zxf ocamlnet-2.2.9.tar.gz
cd ocamlnet-2.2.9
./configure
make
make install

cd ..

wget http://www.rastageeks.org/~toots/xml-light/xml-light-2.2+findlib.tar.gz
tar -zxf xml-light-2.2+findlib.tar.gz
cd xml-light
make install

echo /usr/local/lib &gt;&gt; /etc/ld.so.conf
ldconfig

wget http://voxel.dl.sourceforge.net/sourceforge/savonet/liquidsoap-full-0.9.0.tar.gz
tar -zxf liquidsoap-full-0.9.0.tar.gz
cd liquidsoap-full-0.9.0
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
./bootstrap
./configure
make
make install

Convert Single disk to RAID on Live Linux Server

Warning!  If you don’t have a full backup of the data on your server, DO NOT ATTEMPT THIS!  If you’re not familiar with basic disk utilities (fdisk, grub, ect) DO NOT ATTEMPT THIS!  If you don’t have KVM access in case something goes wrong, DO NOT ATTEMPT THIS!  Although I have checked my instructions for consistency, chances are you are human and are going to mess this up.  If downtime costs you $$$, skip this how-to get a new server with a real (hardware) RAID.

In this example, /dev/hda is our primary existing disk and /dev/hdd is the new disk we want to use to form a RAID mirror.  First, we examine the main disk’s partition table so we can recreate it on the secondary disk.

fdisk -l /dev/hda

Then, you’ll need to create the partition layout on /dev/hdd exactly how it is layed out on /dev/hda.  All partition types should be set to FD (Linux Raid).  I won’t describe this part in great detail since every disk geometry is different, but you’ll need to run this command:

fdisk /dev/hdd

Once you have applied your partition layout, you need to create a degraded RAID member on the new disk.  In this example, we have three main partitions we want to mirror  (swap, /boot, and /).

mdadm –create /dev/md0 -c 128 -l 1 -n 2 /dev/hdd1 missing
mdadm –create /dev/md1 -c 128 -l 1 -n 2 /dev/hdd2 missing
mdadm –create /dev/md2 -c 128 -l 1 -n 2 /dev/hdd3 missing

Now, we create the filesystems

mkfs.ext3 /dev/md0
mkswap /dev/md1
mkfs.ext3 /dev/md2

Mount the new RAID disk members:

mkdir /mnt/tmp
mount /dev/md2 /mnt/tmp
mkdir /mnt/tmp/boot
mount /dev/md0 /mnt/tmp/boot

Copy data from your original, live disk:

cd /mnt/tmp
tar -C / -clspf – . | tar -xlspvf –
cd /mnt/tmp/boot
tar -C /boot -clspf – . | tar -xlspvf –

Assuming you’re using the Grub bootloader (who uses LILO anymore?), install the bootloader on the new disk so it is bootable:

grub /dev/hdd
root (hd1,0)
setup (hd1)
exit

Now we need to modify the /mnt/tmp/etc/fstab file on the new disk so the Kernel can find your new RAID partitions next time you boot.  Below I have shown what mine looks like AFTER editing it:

### vi /mnt/tmp/etc/fstab

/dev/md2                /                       ext3    defaults        1 1
/dev/md0                /boot                   ext3    defaults        1 2
none                    /dev/pts                devpts  gid=5,mode=620  0 0
none                    /dev/shm                tmpfs   defaults        0 0
none                    /proc                   proc    defaults        0 0
none                    /sys                    sysfs   defaults        0 0
/dev/md1                swap                    swap    defaults        0 0
/dev/hdc                /media/cdrom            auto    pamconsole,exec,noauto,managed 0 0

Now modify grub.conf on the new boot partition so Grub can find the Kernel.  Below, you will see what mine looks like AFTER editing it.  The primary change is the root= directive.  I have also duplicated the original entry so that you have the option to boot from either disk, in case one is missing (Grub isn’t aware of Linux RAID).  The last entry is a fail-safe that allows you to boot from the original disk should the first reboot fail.

### vi /mnt/tmp/boot/grub/grub.conf

default=0
fallback=1
timeout=5

## Option 0 default
title CentOS (2.6.9-55.0.2.EL) (disc0)
root (hd0,0)
kernel /vmlinuz-2.6.9-55.0.2.EL ro root=/dev/md2
initrd /initrd-2.6.9-55.0.2.EL.img

## Option 1 in case disk0 fails
title CentOS (2.6.9-55.0.2.EL) (disk1)
root (hd1,0)
kernel /vmlinuz-2.6.9-55.0.2.EL ro root=/dev/md2
initrd /initrd-2.6.9-55.0.2.EL.img

## Option 2 in case new RAID won’t boot during conversion, boot original disk
title CentOS (2.6.9-55.0.2.EL) (old)
root (hd0,0)
kernel /vmlinuz-2.6.9-55.0.2.EL ro root=/dev/hda3
initrd /initrd-2.6.9-55.0.2.EL.img

Now, reboot the server.  Since the default in Grub is entry 0, you should automatically boot from the new single (degraded) RAID disk you created.

All good?  Next, fdisk /dev/hda and change the partition types on old disk to FD (Linux RAID).  Again, I’m not going to teach you how to use fdisk but here’s the command:

fdisk /dev/hda

Now, complete the array and start the RAID build process (this will wipe your original disk with a mirror copy of your new RAID disk):

mdadm /dev/md0 -a /dev/hdd1
mdadm /dev/md1 -a /dev/hdd2
mdadm /dev/md2 -a /dev/hdd3

Now we can monitor the build process.  Wait until it is complete before continuing.

watch cat /proc/mdstat

Now, reinstall the MBR on the old disk so it’s bootable:

grub
root (hd0,0)
setup (hd0)
exit

And now for a final drumroll….test your work:

reboot