How to Cleanup Multipathing

I have a server running redhat 5.5 and it has one SAN device presented to it as LUN9. How can I clean up the remaining entries. I cannot afford to interupt the service. Please assist.

[root@whatever_server device]# multipath -l
mpath0 (36000097000019260298953666633436) dm-11 EMC,SYMMETRIX
[features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 2:0:0:9     sda 8:0   [active][undef]
 \_ 3:0:0:9     sdb 8:16  [active][undef]
 \_ 2:0:0:32777 sdc 8:32  [active][undef]
 \_ 2:0:0:32793 sdd 8:48  [active][undef]
 \_ 3:0:0:32777 sde 8:64  [active][undef]
 \_ 3:0:0:32793 sdf 8:80  [active][undef]
 \_ 2:0:0:36873 sdg 8:96  [active][undef]
 \_ 2:0:0:36889 sdh 8:112 [active][undef]
 \_ 3:0:0:36873 sdi 8:128 [active][undef]
 \_ 3:0:0:36889 sdj 8:144 [active][undef]
 \_ 2:0:0:40969 sdk 8:160 [active][undef]
 \_ 2:0:0:40985 sdl 8:176 [active][undef]
 \_ 3:0:0:40969 sdm 8:192 [active][undef]
 \_ 3:0:0:40985 sdn 8:208 [active][undef]
 \_ 2:0:0:45065 sdo 8:224 [active][undef]
 \_ 3:0:0:45065 sdp 8:240 [active][undef]
 \_ 2:0:0:45081 sdq 65:0  [active][undef]
 \_ 3:0:0:45081 sdr 65:16 [active][undef]

In short: you can't.

Long version: The different sd<x> devices represent different views on the same disk over different paths (hence *multi*-path). Usually you don't use these (because every one of them represents one single connection to the disk) but the multipath-device.

Your actual problem probably is that the additional devices mess up your LVM2 configuration. This is taken care of the following way:

The first layer to deal with the LUNs from SAN is the multipath-software. Make sure you give meaningful alias names in "/etc/multipath.conf". That makes administration a lot easier. I use <vgname>pv<nr> (like: "myvgpv01", "myvgpv02", etc.) for the disks. "/etc/multipath.conf" looks like:

[...]
multipaths {
        #----------------- [dbvg Start] ----------------------
        multipath {
                wwid                    360050768018200878000000000000108
                alias                   dbvgpv01
        }
        multipath {
                wwid                    360050768018200878000000000000140
                alias                   dbvgpv02
        }
        #----------------- [dbvg End] -------------------------
        #----------------- [appvg Start] ----------------------
       multipath {
                wwid                    360050768018200878000000000000149
                alias                   appvgpv01
        }
        #----------------- [appvg End] ------------------------
}
[...]

This will create these disk aliases and the respective udev rules should create device mappings in "/dev/mpath" like this:

root@host # ls -l /dev/mpath
total 0
lrwxrwxrwx 1 root root 8 Jun 20 12:31 dbvgpv01 -> ../dm-13
lrwxrwxrwx 1 root root 8 Jun 20 12:31 dbvgpv01p1 -> ../dm-21
lrwxrwxrwx 1 root root 8 Jun 20 12:31 dbvgpv02 -> ../dm-14
lrwxrwxrwx 1 root root 8 Jun 20 12:31 dbvgpv02p1 -> ../dm-20
lrwxrwxrwx 1 root root 8 Jun 20 12:31 appvgpv01 -> ../dm-11
lrwxrwxrwx 1 root root 8 Jun 20 12:31 appvgpv01p1 -> ../dm-17
[...]

After getting this to work you have to configure the LVM2 by creating a filter-rule and a directory to look for possible PVs. The configuration file is "/etc/lvm/lvm.conf":

[...]

devices {
    [...]
    # An array of directories that contain the device nodes you wish
    # to use with LVM2. "/dev" is for the local disk(s) /dev/sda, /dev/sdb,
    # etc., "/dev/mpath" is for the SAN disks

    scan = [ "/dev", "/dev/mpath" ]

    # filter rules are basically regexps. "a|" means "accept", "r|" means "remove".
    # The rules in detail:
    # "r/.*/" removes every device not explicitly allowed by the rules before
    #
    # "a|vg[0-9]*pv[0-9]*|" my VG names always end in "vg", sometimes followed by a number
    # allow everything ending in "vg" or "vg<nr>" followed by "pv<nr>" like
    # "appvgpv01" or "db5vgp03", etc.
    #
    # "a|sd[ab]$|", "a|sd[ab][1-9]*$|" allows "/dev/sda" or "/dev/sdb",
    # both optionally followed by a number (/dev/sda1), but NOT a
    # character. This filters out "/dev/sdaa" and similar devices.

    filter = [ "a|vg[0-9]*pv[0-9]*|", "a|sd[ab]$|", "a|sd[ab][1-9]*$|", "r/.*/" ]

    [...]
}

You should by now be able to use "pvscan"/"vgscan" to find your VGs and LVM2 should exclusively make use of the multipath devices.

I hope this helps.

bakunin

1 Like

How about if I do the following will it clean up the stale LUN's

#echo 1 > /sys/block/sdc/device/delete
through
#echo 1 > /sys/block/sdr/device/delete