No need to break the mirror (if you break the mirror, there will be no assignment anymore which PP is a copy of which, so what shall it sync?!), just run it.
The number of stale PPs should go to zero after some time. You can also check the number of stale PPs with just "lsvg rootvg".
The easiest way for syncing a stale AIX LVM mirror is
# varyonvg <stalevg>
You could consider the varyonvg command as a very basic repairvg. Varyonvg will call syncvg among other things. This will run in the background so you might see a lresynclv process in the processlist. If the lv stays stale after this finished pls post the output of
(The drawback of varyonvg might be that it syncs one PP per time. Using syncvg makes sense when syncing large mirrors as you can tell it (-P) to sync more PPs per time hence being faster. When it comes to Terrabytes this does make a difference.)
Before I type varyonvg ihslv. I need to ask something, there is no problem to run this command. because this box its a production server.
I send the commands that you told me before run the command.
Thanks for the comments.
lslv -l ihslv
ihslv:/usr/IBMIHS
PV COPIES IN BAND DISTRIBUTION
hdisk0 036:000:000 0% 036:000:000:000:000
hdisk1 036:000:000 0% 036:000:000:000:000
lslv ihslv
LOGICAL VOLUME: ihslv VOLUME GROUP: rootvg
LV IDENTIFIER: 0009d3bc0000d600000001095659b58d.12 PERMISSION: read/write
VG STATE: active/complete LV STATE: opened/stale
TYPE: jfs2 WRITE VERIFY: off
MAX LPs: 512 PP SIZE: 128 megabyte(s)
COPIES: 2 SCHED POLICY: parallel
LPs: 36 PPs: 72
STALE PPs: 36 BB POLICY: relocatable
INTER-POLICY: minimum RELOCATABLE: yes
INTRA-POLICY: middle UPPER BOUND: 32
MOUNT POINT: /usr/IBMIHS LABEL: /usr/IBMIHS
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?: NO
lsvg rootvg
VOLUME GROUP: rootvg VG IDENTIFIER: 0009d3bc0000d600000001095659b58d
VG STATE: active PP SIZE: 128 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 1092 (139776 megabytes)
MAX LVs: 256 FREE PPs: 668 (85504 megabytes)
LVs: 11 USED PPs: 424 (54272 megabytes)
OPEN LVs: 10 QUORUM: 1
TOTAL PVs: 2 VG DESCRIPTORS: 3
STALE PVs: 1 STALE PPs: 36
ACTIVE PVs: 2 AUTO ON: yes
MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32
LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
The information about your vg/lv is looking good. There is no risk in running a varyonvg against an already active volume group. Indeed varyonvg is meant to be used like this as it repairs some vg problems.
I type the command varyonvg ihslv to try to repair the stale error
and It doesnt work
varyonvg ihslv
0516-008 varyonvg: LVM system call returned an unknown
error code (3).
I got the same thing on these commands like yesterday when I typed
lslv -l ihslv
ihslv:/usr/IBMIHS
PV COPIES IN BAND DISTRIBUTION
hdisk0 036:000:000 0% 036:000:000:000:000
hdisk1 036:000:000 0% 036:000:000:000:000
lslv ihslv
LOGICAL VOLUME: ihslv VOLUME GROUP: rootvg
LV IDENTIFIER: 0009d3bc0000d600000001095659b58d.12 PERMISSION: read/write
VG STATE: active/complete LV STATE: opened/stale
TYPE: jfs2 WRITE VERIFY: off
MAX LPs: 512 PP SIZE: 128 megabyte(s)
COPIES: 2 SCHED POLICY: parallel
LPs: 36 PPs: 72
STALE PPs: 36 BB POLICY: relocatable
INTER-POLICY: minimum RELOCATABLE: yes
INTRA-POLICY: middle UPPER BOUND: 32
MOUNT POINT: /usr/IBMIHS LABEL: /usr/IBMIHS
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?: NO
lsvg rootvg
VOLUME GROUP: rootvg VG IDENTIFIER: 0009d3bc0000d600000001095659b58d
VG STATE: active PP SIZE: 128 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 1092 (139776 megabytes)
MAX LVs: 256 FREE PPs: 668 (85504 megabytes)
LVs: 11 USED PPs: 424 (54272 megabytes)
OPEN LVs: 10 QUORUM: 1
TOTAL PVs: 2 VG DESCRIPTORS: 3
STALE PVs: 1 STALE PPs: 36
ACTIVE PVs: 2 AUTO ON: yes
MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32
LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
It worked in a sense that now I can tell you that your ODM is corrupted. To repair an inconsistency between the device configuration database and the LVM information you could try the synclvodm command:
# synclvodm -v -P rootvg ihslv
Do you have a bad disk? Are there any P/H errors logged on the disk? If there are, run diags on it. If that runs clean, break the mirror, remove it from rootvg and run diags again, format it and then re-add it to the VG.
That's odd. Using fsck or recreating the LV would both be disruptive to service. Given that you use IHS with that stale mirror an online repair should still be possible. Hence next thing I would try is
if you have two disks in your server that can be used by rootvg:
breaking the LV mirror and mirroring the LV anew. If you did not already - backup your FS data (e.g. tar locally to another filesystem) before you start.
# rmlvcopy ihslv 1
Important: do not give any information about from which disk the mirror copy should be removed but let LVM sort this out. LVM should detect and keep the good copy. Immediately after removing the copy you can mirror the LV again:
# mklvcopy ihslv 2
# varyonvg rootvg
wait till sync'd, check.
if you have three disks (could be connected temporarily) in your server that can be used by rootvg:
Extend rootvg to this disk and make a third mirror copy on the third disk
# extendvg rootvg hdisk2
# mklvcopy ihslv 3 hdisk2
# varyonvg rootvg
wait till sync'd! Only then do
# rmlvcopy ihslv 2 hdisk1
# rmlvcopy ihslv 1 hdisk0
Then rebuild copies on the original disks:
# mklvcopy ihslv 2 hdisk0
# mklvcopy ihslv 3 hdisk1
# varyonvg rootvg
wait till sync'd, check, if OK remove third mirror copy
# rmlvcopy ihslv 2 hdisk2
Reduce rootvg.
# reducevg rootvg hdisk2