Free PP Allocation problem

Firstly, I can't stress enough how much of a newb I am to AIX or Linux in general so please be very patient. I am strictly a MS man, however I now need to administer an IBM power series blade server which sits on top of an AIX VIOS. The VIOS version I am running is 2.1.2.0 and it is mirrored on two physical disks.

I am currently trying to install the latest version of the VIOS but when I attempt to run the command updateios it complains that there is not enough space in the /usr filesystem.

I then attempted to extend the /usr filesytem using extendlv but I get the error "0516-404 allocp: This system cannot fulfill the allocation request.There are not enough free partitions or not enough physical volumes to keep strictness and satisfy allocation requests. The command should be retried with different allocation characteristics."

In troubleshooting this error on google I think the problem lies in the allocation of my free PPs over the two mirrored disks. Here is the output of lsvg -pv rootvg.

$ lsvg -pv rootvg
rootvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk0            active            546         0           00..00..00..00..00
hdisk1            active            546         301         83..00..00..109..109

I think what I need to do is move some of the 301 free PPs from hdisk1 over to hdisk0 but I haven't a clue where to even start. Can anyone help? Again, go easy I'm extremely new to this and barely know the connection between LVs, PVs, PPs and LPs.

Obviously your rootvg mirror is not clean, hdisk0 has more data than hdisk1. Check lsvg -l rootvg to see which logical volumes are not mirrored and then decide what to do.

As firefox111 stated run;

lsvg -l rootvg

This will display something like;

root@ap1:/u/amilburn $ lsvg -l rootvg
rootvg:
LV NAME             TYPE       LPs   PPs   PVs  LV STATE      MOUNT POINT
hd5                 boot       1     2     2    closed/syncd  N/A
hd6                 paging     16    32    2    open/syncd    N/A
hd8                 jfslog     1     2     2    open/syncd    N/A
hd4                 jfs        1     2     2    open/syncd    /
hd2                 jfs        7     14    2    open/syncd    /usr
hd9var              jfs        2     4     2    open/syncd    /var
hd7                 sysdump    5     5     1    open/syncd    N/A

As you have probably already figured out you are looking for the number of LPs compared to the number of PPs. if the PPs are doubled then it is mirrored. If it is not then figure out where that LV resides. If it already resides on hdisk1 then it will not do you any good but if it resides on hdisk0 then you can use;

migratepv -l logicalvolume hdisk0 hdisk1

to move that LV from hdisk0 to hdisk1 to free up some space on hdisk0.

You can use

lspv -l hdisk?

to figure out which disk the LV in question resides.

In my case my the only LVs that are not mirrored are my sysdump devices so that might be something to look for.

Here is my lsvg output:

$ lsvg -lv rootvg
rootvg:
LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT
hd5                 boot       1       2       2    closed/syncd  N/A
hd6                 paging     2       4       2    open/syncd    N/A
paging00            paging     4       8       2    open/syncd    N/A
hd8                 jfs2log    1       2       2    open/syncd    N/A
hd4                 jfs2       1       2       2    open/syncd    /
hd2                 jfs2       14      28      2    open/syncd    /usr
hd9var              jfs2       3       6       2    open/syncd    /var
hd3                 jfs2       14      28      2    open/syncd    /tmp
hd1                 jfs2       40      80      2    open/syncd    /home
hd10opt             jfs2       8       16      2    open/syncd    /opt
hd11admin           jfs2       1       2       2    open/syncd    /admin
livedump            jfs2       1       2       2    open/syncd    /var/adm/ras/livedump
lg_dumplv           sysdump    4       4       1    open/syncd    N/A
fwdump              jfs2       3       6       2    open/syncd    /var/adm/ras/platform
loglv00             jfslog     1       1       1    closed/syncd  N/A

As you can see /usr is in fact mirrored and resides on hd2. Is this good or bad news?

I presume the hd2 LV has no direct realtionship to PV hdisk2 which appears is lspv as below?

$ lspv
NAME             PVID                                 VG               STATUS
hdisk0           0007f6fa67f32350                     rootvg           active
hdisk1           0007f6fa6c827ecf                     rootvg           active
hdisk2           none                                 None
hdisk3           none                                 None
hdisk4           none                                 None
hdisk5           none                                 None
hdisk6           none                                 None
hdisk7           none                                 None

looking into your output from your last output you should not use more than 88 physical partitions per disk - I am wondering where the other 400+ are as your first post shows 546 pps in use on hdisk0 ... but if they are in use they obviously should be displayed somehere which they are not.

Can we start from the beginning - can you show us the output of

lsvg rootvg
lsvg -M rootvg 
lspv -l hdisk0 
lspv -l hdisk1

has this system been migrated from old small disks to new a lot larger disks maybe ?

regards
zxmaus

$ lsvg rootvg
VOLUME GROUP:       rootvg                   VG IDENTIFIER:  0007f6fa0000d4000000012667f326b5
VG STATE:           active                   PP SIZE:        256 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      1092 (279552 megabytes)
MAX LVs:            256                      FREE PPs:       301 (77056 megabytes)
LVs:                16                       USED PPs:       791 (202496 megabytes)
OPEN LVs:           14                       QUORUM:         1 (Disabled)
TOTAL PVs:          2                        VG DESCRIPTORS: 3
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         2                        AUTO ON:        yes
MAX PPs per VG:     32512
MAX PPs per PV:     1016                     MAX PVs:        32
LTG size (Dynamic): 1024 kilobyte(s)         AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable

In my version of AIX -M is not a valid argument for LSVG, there is however an argument -map for which the below is displayed. Is that what you were looking for?

$ lsvg -map rootvg
PHYSICAL        LOGICAL
rootvg
hdisk0:1        hd5:1:1
hdisk0:111      hd6:1:1
hdisk0:112      hd6:2:1
hdisk0:113      livedump:1:1
hdisk0:114      lg_dumplv:1
hdisk0:115      lg_dumplv:2
hdisk0:116      lg_dumplv:3
hdisk0:117      lg_dumplv:4
hdisk0:118      fwdump:1:1
hdisk0:119      fwdump:2:1
hdisk0:120      fwdump:3:1
hdisk0:220      paging00:1:1
hdisk0:221      paging00:2:1
hdisk0:222      paging00:3:1
hdisk0:223      paging00:4:1
hdisk0:224      hd8:1:1
hdisk0:225      hd4:1:1
hdisk0:226      hd2:1:1
hdisk0:227      hd2:2:1
hdisk0:228      hd2:3:1
hdisk0:229      hd2:4:1
hdisk0:230      hd2:5:1
hdisk0:231      hd2:6:1
hdisk0:232      hd2:7:1
hdisk0:233      hd2:8:1
hdisk0:234      hd2:9:1
hdisk0:235      hd2:10:1
hdisk0:236      hd2:11:1
hdisk0:237      hd2:12:1
hdisk0:238      hd2:13:1
hdisk0:239      hd9var:1:1
hdisk0:240      hd9var:2:1
hdisk0:241      hd9var:3:1
hdisk0:242      hd3:1:1
hdisk0:243      hd3:2:1
hdisk0:244      hd3:3:1
hdisk0:245      hd3:4:1
hdisk0:246      hd3:5:1
hdisk0:247      hd3:6:1
hdisk0:248      hd3:7:1
hdisk0:249      hd3:8:1
hdisk0:250      hd3:9:1
hdisk0:251      hd3:10:1
hdisk0:252      hd3:11:1
hdisk0:253      hd3:12:1
hdisk0:254      hd3:13:1
hdisk0:255      hd3:14:1
hdisk0:256      hd1:1:1
hdisk0:257      hd1:2:1
hdisk0:258      hd1:3:1
hdisk0:259      hd1:4:1
hdisk0:260      hd1:5:1
hdisk0:261      hd1:6:1
hdisk0:262      hd1:7:1
hdisk0:263      hd1:8:1
hdisk0:264      hd1:9:1
hdisk0:265      hd1:10:1
hdisk0:266      hd1:11:1
hdisk0:267      hd1:12:1
hdisk0:268      hd1:13:1
hdisk0:269      hd1:14:1
hdisk0:270      hd1:15:1
hdisk0:271      hd1:16:1
hdisk0:272      hd1:17:1
hdisk0:273      hd1:18:1
hdisk0:274      hd1:19:1
hdisk0:275      hd1:20:1
hdisk0:276      hd1:21:1
hdisk0:277      hd1:22:1
hdisk0:278      hd1:23:1
hdisk0:279      hd1:24:1
hdisk0:280      hd1:25:1
hdisk0:281      hd1:26:1
hdisk0:282      hd1:27:1
hdisk0:283      hd1:28:1
hdisk0:284      hd1:29:1
hdisk0:285      hd1:30:1
hdisk0:286      hd1:31:1
hdisk0:287      hd1:32:1
hdisk0:288      hd1:33:1
hdisk0:289      hd1:34:1
hdisk0:290      hd1:35:1
hdisk0:291      hd1:36:1
hdisk0:292      hd1:37:1
hdisk0:293      hd1:38:1
hdisk0:294      hd1:39:1
hdisk0:295      hd1:40:1
hdisk0:296      hd10opt:1:1
hdisk0:297      hd10opt:2:1
hdisk0:298      hd10opt:3:1
hdisk0:299      hd10opt:4:1
hdisk0:300      hd10opt:5:1
hdisk0:301      hd11admin:1:1
hdisk0:302      hd2:14:1
hdisk0:303      hd10opt:6:1
hdisk0:304      hd10opt:7:1
hdisk0:305      hd10opt:8:1
hdisk1:1        hd5:1:2
hdisk1:2-84
hdisk1:111      hd6:1:2
hdisk1:112      hd6:2:2
hdisk1:113      livedump:1:2
hdisk1:114      fwdump:1:2
hdisk1:115      fwdump:2:2
hdisk1:116      fwdump:3:2
hdisk1:117      loglv00:1
hdisk1:220      paging00:1:2
hdisk1:221      paging00:2:2
hdisk1:222      paging00:3:2
hdisk1:223      paging00:4:2
hdisk1:224      hd8:1:2
hdisk1:225      hd4:1:2
hdisk1:226      hd2:1:2
hdisk1:227      hd2:2:2
hdisk1:228      hd2:3:2
hdisk1:229      hd2:4:2
hdisk1:230      hd2:5:2
hdisk1:231      hd2:6:2
hdisk1:232      hd2:7:2
hdisk1:233      hd2:8:2
hdisk1:234      hd2:9:2
hdisk1:235      hd2:10:2
hdisk1:236      hd2:11:2
hdisk1:237      hd2:12:2
hdisk1:238      hd2:13:2
hdisk1:239      hd2:14:2
hdisk1:240      hd9var:1:2
hdisk1:241      hd9var:2:2
hdisk1:242      hd9var:3:2
hdisk1:243      hd3:1:2
hdisk1:244      hd3:2:2
hdisk1:245      hd3:3:2
hdisk1:246      hd3:4:2
hdisk1:247      hd3:5:2
hdisk1:248      hd3:6:2
hdisk1:249      hd3:7:2
hdisk1:250      hd3:8:2
hdisk1:251      hd3:9:2
hdisk1:252      hd3:10:2
hdisk1:253      hd3:11:2
hdisk1:254      hd3:12:2
hdisk1:255      hd3:13:2
hdisk1:256      hd3:14:2
hdisk1:257      hd1:1:2
hdisk1:258      hd1:2:2
hdisk1:259      hd1:3:2
hdisk1:260      hd1:4:2
hdisk1:261      hd1:5:2
hdisk1:262      hd1:6:2
hdisk1:263      hd1:7:2
hdisk1:264      hd1:8:2
hdisk1:265      hd1:9:2
hdisk1:266      hd1:10:2
hdisk1:267      hd1:11:2
hdisk1:268      hd1:12:2
hdisk1:269      hd1:13:2
hdisk1:270      hd1:14:2
hdisk1:271      hd1:15:2
hdisk1:272      hd1:16:2
hdisk1:273      hd1:17:2
hdisk1:274      hd1:18:2
hdisk1:275      hd1:19:2
hdisk1:276      hd1:20:2
hdisk1:277      hd1:21:2
hdisk1:278      hd1:22:2
hdisk1:279      hd1:23:2
hdisk1:280      hd1:24:2
hdisk1:281      hd1:25:2
hdisk1:282      hd1:26:2
hdisk1:283      hd1:27:2
hdisk1:284      hd1:28:2
hdisk1:285      hd1:29:2
hdisk1:286      hd1:30:2
hdisk1:287      hd1:31:2
hdisk1:288      hd1:32:2
hdisk1:289      hd1:33:2
hdisk1:290      hd1:34:2
hdisk1:291      hd1:35:2
hdisk1:292      hd1:36:2
hdisk1:293      hd1:37:2
hdisk1:294      hd1:38:2
hdisk1:295      hd1:39:2
hdisk1:296      hd1:40:2
hdisk1:297      hd10opt:1:2
hdisk1:298      hd10opt:2:2
hdisk1:299      hd10opt:3:2
hdisk1:300      hd10opt:4:2
hdisk1:301      hd10opt:5:2
hdisk1:302      hd10opt:6:2
hdisk1:303      hd10opt:7:2
hdisk1:304      hd10opt:8:2
hdisk1:305      hd11admin:1:2
hdisk1:329-546

Again, I don't have -l as a valid arg for lspv but I do have -lv, is this what you are looking for?

$ lspv -pv hdisk0
hdisk0:
PP RANGE  STATE   REGION        LV NAME             TYPE       MOUNT POINT
  1-1     used    outer edge    hd5                 boot       N/A
  2-110   used    outer edge    VMLibrary           jfs2       /var/vio/VMLibrary
111-112   used    outer middle  hd6                 paging     N/A
113-113   used    outer middle  livedump            jfs2       /var/adm/ras/livedump
114-117   used    outer middle  lg_dumplv           sysdump    N/A
118-120   used    outer middle  fwdump              jfs2       /var/adm/ras/platform
121-219   used    outer middle  VMLibrary           jfs2       /var/vio/VMLibrary
220-223   used    center        paging00            paging     N/A
224-224   used    center        hd8                 jfs2log    N/A
225-225   used    center        hd4                 jfs2       /
226-238   used    center        hd2                 jfs2       /usr
239-241   used    center        hd9var              jfs2       /var
242-255   used    center        hd3                 jfs2       /tmp
256-295   used    center        hd1                 jfs2       /home
296-300   used    center        hd10opt             jfs2       /opt
301-301   used    center        hd11admin           jfs2       /admin
302-302   used    center        hd2                 jfs2       /usr
303-305   used    center        hd10opt             jfs2       /opt
306-328   used    center        VMLibrary           jfs2       /var/vio/VMLibrary
329-437   used    inner middle  VMLibrary           jfs2       /var/vio/VMLibrary
438-546   used    inner edge    VMLibrary           jfs2       /var/vio/VMLibrary
$ lspv -pv hdisk1
hdisk1:
PP RANGE  STATE   REGION        LV NAME             TYPE       MOUNT POINT
  1-1     used    outer edge    hd5                 boot       N/A
  2-84    free    outer edge
 85-110   used    outer edge    VMLibrary           jfs2       /var/vio/VMLibrary
111-112   used    outer middle  hd6                 paging     N/A
113-113   used    outer middle  livedump            jfs2       /var/adm/ras/livedump
114-116   used    outer middle  fwdump              jfs2       /var/adm/ras/platform
117-117   used    outer middle  loglv00             jfslog     N/A
118-219   used    outer middle  VMLibrary           jfs2       /var/vio/VMLibrary
220-223   used    center        paging00            paging     N/A
224-224   used    center        hd8                 jfs2log    N/A
225-225   used    center        hd4                 jfs2       /
226-239   used    center        hd2                 jfs2       /usr
240-242   used    center        hd9var              jfs2       /var
243-256   used    center        hd3                 jfs2       /tmp
257-296   used    center        hd1                 jfs2       /home
297-304   used    center        hd10opt             jfs2       /opt
305-305   used    center        hd11admin           jfs2       /admin
306-328   used    center        VMLibrary           jfs2       /var/vio/VMLibrary
329-437   free    inner middle
438-546   free    inner edge

This machine was a completely fresh install by an IBM guy. As stated in my OP the machine is a power blade server which acts as the VIOS for a IBM i server. Hope this helps.

Apologies, I forgot that you have a VIO server - my commands do work on 'normal aix'.

Ok so the missing pps are used by /var/vio/VMLibrary which doesn't seem to be mirrored.
I am not sure if commands like df -g or chfs commands are working on vio servers (as these are managed by another team in my company) but I would try to find out if I can make one of the existing filesystems either smaller or can live without a mirror for it - I would say /home would be such a candidate as it seems relatively huge and it doesnt necessarily have to be mirrored - depending on how full it is you could try to reduce the size by 1 GB and give the space to /usr instead with chfs -a size=-1G /home and chfs -a size=+1G /usr ?

Regards
zxmaus

On your VIO Server type

oem_setup_env

You should get a # prompt.

This will put into more of a AIX compatible mode where you can run most of the normal AIX commands that zxmaus mentioned.

---------- Post updated at 11:00 PM ---------- Previous update was at 10:55 PM ----------

It looked to me like lg_dumplv is on hdisk0 and is not mirrored. You could move that lv from hdisk0 to hdisk1 with the command below after running the oem command mentioned above.

migratepv -l lvname hdisk0 hdisk1

That would fee up about 1Gig on hdisk0.

I don't mean to butt in on this to much just throwing some other thoughts out there.

1 Like

That's exactly what I'll do, I'll reduce the size of /home as it's huge.

Two things I don't understand though:

  1. I deleted a huge 20G SAVSYS from /var/vio/VMLibrary but it didn't free up any PPs on either physical disk. Would I have to delete files then reduce the filesystem using chfs before I see any free PPs?
  2. If /var/vio/VMLibrary isn't mirrored how does it appear on both physical disks?

Cheers

Craig

When you delete any files on the FS level, the LV/VG below does not know about it. Reduce the FS with chfs -a size=-1G or from "smit fs".

/var/vio/VMLibrary -> When that FS is mirrored, it should show "n" PPs in lsvg -l and n x 2 LPs. This doesn't meant that it is properly mirrored, but it is mirrored. Since you have just 2 disks the chance that it is correctly mirrored is pretty high :slight_smile:

Did that also. Thanks for you help.

How much were you trying to expand "/usr" by? From your original post, you have almost 77 GB free!

FREE PPs:       301 (77056 megabytes)

My /usr filesystem is only about 6 GB and I'm on the lastest version of VIO.

vio1:/home/padmin:# ioslevel
2.2.0.11-FP-24 SP-01
vio1:/home/padmin:# df -Im /usr
Filesystem    MB blocks      Used      Free %Used Mounted on
/dev/hd2        6144.00   4510.93   1633.07   74% /usr
vioe1:/home/padmin:#

From memory, I think I need about another 2.5 GB.

After logging in as "padmin" and then running "oem_setup_env", did you run this and it failed?:

chfs -a size=+2560MB /usr

But from my lsvg I can see that the PPs are not double the LPs for /var/vio/VMLibrary. Does that not mean the filesystem isn't mirrored?

Also on looking at the below output I can see the distribution of LPs and PPs. If I reduce the size of /var/vio/VMLibrary using chfs will it always take PPs and LPs from hdisk1 until there are no longer any more LPs and PPs and then move to hdisk0 or can I force chfs to remove PPs and LPs from hdsik0?

# lspv -l hdisk0
hdisk0:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
VMLibrary             449     449     109..99..23..109..109 /var/vio/VMLibrary
hd10opt               8       8       00..00..08..00..00    /opt
hd11admin             1       1       00..00..01..00..00    /admin
fwdump                3       3       00..03..00..00..00    /var/adm/ras/platform
livedump              1       1       00..01..00..00..00    /var/adm/ras/livedump
hd9var                3       3       00..00..03..00..00    /var
hd3                   14      14      00..00..14..00..00    /tmp
hd1                   32      32      00..00..32..00..00    /home
paging00              4       4       00..00..04..00..00    N/A
hd8                   1       1       00..00..01..00..00    N/A
hd4                   1       1       00..00..01..00..00    /
hd2                   26      26      00..04..22..00..00    /usr
hd5                   1       1       01..00..00..00..00    N/A
hd6                   2       2       00..02..00..00..00    N/A
# lspv -l hdisk1
hdisk1:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
VMLibrary             147     147     22..102..23..00..00   /var/vio/VMLibrary
hd10opt               8       8       00..00..08..00..00    /opt
hd11admin             1       1       00..00..01..00..00    /admin
fwdump                3       3       00..03..00..00..00    /var/adm/ras/platform
loglv00               1       1       00..01..00..00..00    N/A
livedump              1       1       00..01..00..00..00    /var/adm/ras/livedump
lg_dumplv             4       4       04..00..00..00..00    N/A
hd9var                3       3       00..00..03..00..00    /var
hd3                   14      14      00..00..14..00..00    /tmp
hd1                   32      32      00..00..32..00..00    /home
paging00              4       4       00..00..04..00..00    N/A
hd8                   1       1       00..00..01..00..00    N/A
hd4                   1       1       00..00..01..00..00    /
hd2                   26      26      00..00..22..04..00    /usr
hd5                   1       1       01..00..00..00..00    N/A
hd6                   2       2       00..02..00..00..00    N/A

PPs (physical partitions) represent free space on each disk. Each PP is 256 MB in your rootvg. You can see that in the "lsvg rootvg" output:

PP SIZE:        256 megabyte(s)

1 PP is the same as 256 MB of free space. The only way to get more free PPs is to move stuff off the disk to the other disk. LPs represent how may logical partitions of your rootvg are being used. If the disk is not mirrored, then a LP is the same as a PP. If your disk is mirrored, then LPs will be half the number of your PPs unless the logical volume is not mirrored like your /var/vio/VMLibrary. Since your "/var/vio/VMLibrary" is only on hdisk1, shrinking that filesystem size will only free PPs on hdisk1. There is no way to move free PPs from one disk to another. You can move logical volumes between your disks (migratepv -l) to free up PPs on one disk but you can't move free space from one disk over to another.

If your goal is to increase /usr by 2.5 GB, you should have the space according to your "lsvg rootvg" output:

$ lsvg rootvg
VOLUME GROUP:       rootvg                   VG IDENTIFIER:  0007f6fa0000d4000000012667f326b5
VG STATE:           active                   PP SIZE:        256 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      1092 (279552 megabytes)
MAX LVs:            256                      FREE PPs:       301 (77056 megabytes)
LVs:                16                       USED PPs:       791 (202496 megabytes)
OPEN LVs:           14                       QUORUM:         1 (Disabled)
TOTAL PVs:          2                        VG DESCRIPTORS: 3
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         2                        AUTO ON:        yes
MAX PPs per VG:     32512
MAX PPs per PV:     1016                     MAX PVs:        32
LTG size (Dynamic): 1024 kilobyte(s)         AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable

Run this after logging in as padmin:

oem_setup_env
chfs -a size=+2560MB /usr

But isn't my /var/vio/VMLibrary spread over both disks, 449 LPs on hdisk0 and 147 LPs on hdisk1?

yes it is - and in oem mode you can move without a problem partitions from hdisk0 to hdisk1

if you run again my lsvg -M rootvg command, you should see something similar to the output you have posted in your second or third post ...

hdisk0 VMLibrary:1
hdisk0 VMLibrary:2
hdisk0 VMLibrary:3
hdisk0 VMLibrary:4
hdisk0 VMLibrary:5 ....

if you want to move any of these partitions from hdisk0 to hdisk1, than your commands are

migratelp VMLibrary/1 hdisk1
migratelp VMLibrary/2 hdisk1 ...

Yes, it is spread across both of your disks but that is irrelevant if you are just trying to increase the size of your /usr filesystem by 2.5 GB. Deal with that later. Run the chfs command I posted earlier and your /usr should increase without any error. You don't have to first increase the LV before increasing the FS. Some old versions of Linux I worked with had to do that but I've never had to do it with AIX 5.2 or newer. AIX is smart enough to figure out that the LV needs increased before the FS and will do it automatically.

1 Like