Hey guys, wondering if this is possible to accomplish.
PowerHA SystemMirror7.1 active/passive cluster. Restored a clustered system in test and upgraded to the latest version of PowerHA. Things are working great. However in the interest of time during cut over to the new system, I was hoping to take a flash copy of the data lun (datavg) from the current production cluster and import that to the new system, however, I'm getting stuck removing hdisk1 on the restored system.
Steps to recreate on the new cluster:
Stop the cluster services on both nodes so datavg is varied off
exportvg datavg
on both nodes - success
rmdev -l hdisk1 -Rd
on both nodes - success on node2(passive) fail on node1(active)
No matter what I try I can't remove the disk on node1, getting this error.
Method error (/etc/methods/ucfgdevice):
0514-062 Cannot perform the requested function because the
specified device is busy.
The PVID's are going to change with the flash copy, so my thought was to remove these from ODM on the new system and then drop that lun. Next, add the flashed lun and do:
Your ODM might need to be massaged a bit (on some rare occasions it gets into an inconsistent state). Try the "synclvodm" command to correct that.
You might find out why the disk is busy by using "fuser" on the disk device ("/dev/hdiskX"). Maybe some monitoring agent or other software is holding it open.
Thanks for the reply. I actually resolved this yesterday.
The only way to do this was to first remove the cluster definition, because when volumes are part of a cluster resource group, you need to use cluster commands to make changes so the cluster is aware.
After removing the cluster definition I varied off the vg and exported the vg on both nodes. I could see the physical volume was still busy when trying to remove it, so I located the parent via:
lsdev -l hdisk1 -F parent
Turns out the fiber adapter was keeping the volume busy, this adapter is also used for the rootvg so disabling wasn't an option. A reboot was required.
Following the reboot I was able to remove the volume and add the newly flashed lun. Lastly I redefined the cluster, tested, success.
Interesting. This should not be the case. I remove volumes and change volume groups actually all the time, in 6.x-clusters as well as in 7.x-clusters. To have one disk remain constant in a VG and do "learning import" ( importvg -L ) on the other node is usually all it takes.
I usually even replace disks completely by: adding the new disk, mirror, importvg on the passive node, then unmirror, reducevg, another importvg on the passive node. Not even a takeover or application touchdown is necessary that way.
My "gut feeling" rather says the FC adapter was the culprit, not the cluster software. Anyways, i am glad you resolved it.
You're probably right, I think removing the cluster definition wouldn't have been necessary had I known it was the Fc adapter holding onto the volume. Of course in the process of narrowing it down it just turned out to be the way I did it. Since it was the only physical volume in the VG (and concurrent) I felt it was the safest way to go about it.
For my scenario I will need to do this (redefine the cluster) anyway due to a complete IP schema change. I may even try out the cluster snapshot feature going forward. Thanks again for the follow-up!