MySQL LUN Migration
9:03 PM
This is a howto focused on migrating from one LUN to another LUN
where MySQL data is stored. This howto also covers the procedures used
when the MySQL service is clustered using RHCS
Once your new LUN has been provisioned by storage there are two ways to discover the new LUN(s) 1. Force a scan of the SCSI bus echo "scsi-qlascan" > /proc/scsi/qla2xxx/[0/1] - perform for each target there is 0/1/2/3, etc... echo "- - -">/sys/class/scsi_host/hostX/scan (repeat for all X) NOTE: Quick 'one-liner' to scan all hosts for a in {1..10} ;do echo "- - -:" > /sys/class/scsi_host/host$a/scan;done /sbin/multipath -v 3 /sbin/multipath -ll 2. shutdown -r nowOnce you've discovered you new LUN, proceed to the following steps: - Make sure you identify the new LUN path in '/dev/mapper/' this can usually be performed with a combination of 'df -h' to identify existing LUN's mountpoints an 'ls -l /dev/mapper/mpath*' or an 'multipath -ll'Format your new LUN - mkfs.ext3 -m 1 /dev/mpath/mpathX (use -m 1 to reduce the amount of reserved blocks for root on partition for max space) - mkdir /tmpsan - Mount the new san to /tmpsan: mount -t ext3 /dev/mpath/mpathX /tmpsanSome time prior to the maintenance window perform a live sync of data - time rsync -av --stats /var/lib/mysql/ /tmpsan (take note to record transfer times in ticket. The trailing '/' on the first directory matters) During Maintenance: FOR CLUSTERED SERVERS: Run 'clustat' to identify the name of the cluster_service: - Disable the cluster service - clusvcadm -d cluster_service_name Make a back up and edit the cluster.conf file: cp /etc/cluster/cluster.conf /etc/cluster/cluster.conf.mmddyy vi /etc/cluster/cluster.conf - change the name of the mountpoint(s) (fs device="/dev/mpath/mpathX") to the new /dev/mpath/mpathX path - Increment the 'cluster config_version="X"' up. So 8 would become 9, etc... - Save your changes Update the cluster version on all nodes: - ccs_tool update /etc/cluster/cluster.conf - This command updates the cluster.conf on all nodes - cman_tool version -r X - i've had issues with ccs_tool so I ran both commands the last time, but only cman_tool may be required. CLUSTERED or NON CLUSTERED environment: Align the mounts accordingly for a final sync: - umount /tmpsan - unmount new san - umount /var/lib/mysql - unmount current prod san - required in a NON clustered environment only - mount -t ext3 /dev/mpath/mpathX /var/lib/mysql - Mount new san to /var/lib/mysql - mount -t ext3 /dev/mpat/mpathX /tmpsan - Mount the old san to /tmpsan Rsync the data once again: - time rsync -av --stats /tmpsan/ /var/lib/mysql/ (take note to record transfer times in ticket) Unmount the new poduction and temp san LUN: - umount /var/lib/mysql - umount /tmpsan NON CLUSTERED: - /etc/init.d/mysql start Start the cluster after the rsync completes: - clusvcadm -e cluster_service_name Confirm everything is up and running as expected - tail -f /var/log/messages - clustat - the service should show as 'started' on one of the two nodes ****NOTE**** - If Mysql fails to start you may also check the /var/lib/mysql/hostname-err logfile
0 comments