Need LVM Help

Hello,

I need some help, it is slightly urgent so any help is appreciated.

We were doing a data migration, during a migratepv command, our SAN ran out of quota space and locked all of the filesystems, now I'm left with a few errors and am not able to do migratepv again.

[root]/[]:migratepv hdisk18 hdisk100
0516-1244 migratepv: Destination physical volume hdisk100 contains
        partitions of the logical volume lvmhsrwriter.
0516-812 migratepv: Warning, migratepv did not completely succeed;
        all physical partitions have not been moved off the PV.

[root]/[]:lslv -l lvmhsrwriter
0516-1939 : PV identifier not found in VGDA.

root]/db/mhsrwriter[]:lsvg -l datavg
0516-1147 : Warning - logical volume lvmhsrwriter may be partially mirrored.
lvmhsrwriter        jfs2       268     279     18   open/syncd    /db/mhsrwriter

[root]/[]:lslv lvmhsrwriter
LOGICAL VOLUME:     lvmhsrwriter           VOLUME GROUP:   datavg
LV IDENTIFIER:      00c0815e00004c0000000106840bfa30.37 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            512                    PP SIZE:        128 megabyte(s)
COPIES:             2                      SCHED POLICY:   parallel
LPs:                268                    PPs:            536
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       maximum                RELOCATABLE:    yes
INTRA-POLICY:       edge                   UPPER BOUND:    64
MOUNT POINT:        /db/mhsrwriter         LABEL:          /db/mhsrwriter
DEVICE UID:         0                      DEVICE GID:     0
DEVICE PERMISSIONS: 432
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO
INFINITE RETRY:     no                     PREFERRED READ: 0
DEVICESUBTYPE:      DS_LVZ
COPY 1 MIRROR POOL: None
COPY 2 MIRROR POOL: None
COPY 3 MIRROR POOL: None

So we tried rmlvcopy (A suggestion from the internet)

[root]/[]:rmlvcopy lvmhsrwriter 1 hdisk100
0516-1939 lquerypv: PV identifier not found in VGDA.
0516-304 getlvodm: Unable to find device id 0000000000000000 in the Device
        Configuration Database.
0516-848 rmlvcopy: Failure on physical volume 0000000000000000, it may be missing
        or removed.

Any help to resolve this issue is greatly appreciated.

I'm surprised that the process didn't just get an IO error and abort in a consistent way, but I had something similar years ago after a disk failure. We ended up editing the ODM to correct the conflict, but this is a specialised task and we had IBM walk us through it. If you are paying for software support from IBM or another party then this is the time to use it.

Sorry I can't give you more specific advice.

Robin

A few general words first: i try to avoid migratepv. There is too much tinkering and low-level hassle involved for it to work reliably. Instead i create a new mirror on the target PV (if that fails i just throw it away completely), then remove the old one on the source PV, like this:

extendvg myvg newPV
mirrorvg -s myvg newPV
syncvg -P 32 -v myvg &    # this is faster than let doing it mirrorvg on its own
unmirrorvg myvg oldPV
reducevg myvg oldPV

Second: fire your SAN guys. The necessary storage (whatever "necessary" means) has to be there - period. If a disk needs to be 100G then 99G won't do. And quotas are good, but they have to allow the admins to still do their work.

Now back to your problem: you first need to get correct VG descriptions onto the disks involved in your VG (that means: their VGDA) and your ODM. Stop the system and reboot (without mounting). Then selectively varyon the VG with the error and do a synclvodm . In many cases this works, if it doesn't: you may have to use redefinevg to correct this information manually.

Once you have corrected the VG information to a consistent state: reboot and mount normally to see if there are still errors left. If there aren't: my suggestion is to get yet another disk from the SAN (sufficiently sized this time) temporarily and then execute the process i described above. If that works throw away the old disks which you can give back to SAN.

I hope this helps.

bakunin