Creating AIX LV VG and JFS2 issues?

Can someone please help me understand what it is I am doing incorrectly?

I have 22 PVs (disks) assigned to a VG, the stats on the disks:

# lspv hdisk1
PHYSICAL VOLUME:    hdisk1                   VOLUME GROUP:     vg_usr1
PV IDENTIFIER:      0000150179158027 VG IDENTIFIER     000015010000d60000000149c                                                                                                                               f0a568a
PV STATE:           active
STALE PARTITIONS:   0                        ALLOCATABLE:      yes
PP SIZE:            128 megabyte(s)          LOGICAL VOLUMES:  1
TOTAL PPs:          546 (69888 megabytes)    VG DESCRIPTORS:   1
FREE PPs:           0 (0 megabytes)          HOT SPARE:        no
USED PPs:           546 (69888 megabytes)    MAX REQUEST:      256 kilobytes
FREE DISTRIBUTION:  00..00..00..00..00
USED DISTRIBUTION:  110..109..109..109..109
MIRROR POOL:        None

I have a VG created..

# lsvg vg_usr1
VOLUME GROUP:       vg_usr1                  VG IDENTIFIER:  000015010000d60000000149cf0a568a
VG STATE:           active                   PP SIZE:        128 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      12012 (1537536 megabytes)
MAX LVs:            256                      FREE PPs:       2 (256 megabytes)
LVs:                1                        USED PPs:       12010 (1537280 megabytes)
OPEN LVs:           0                        QUORUM:         12 (Enabled)
TOTAL PVs:          22                       VG DESCRIPTORS: 22
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         22                       AUTO ON:        yes
MAX PPs per VG:     32512
MAX PPs per PV:     1016                     MAX PVs:        32
LTG size (Dynamic): 256 kilobyte(s)          AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable
PV RESTRICTION:     none                     INFINITE RETRY: no
DISK BLOCK SIZE:    512

I have a LV created..

# lslv lv_usr1
LOGICAL VOLUME:     lv_usr1                VOLUME GROUP:   vg_usr1
LV IDENTIFIER:      000015010000d60000000149cf0a568a.1 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       closed/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            32512                  PP SIZE:        128 megabyte(s)
COPIES:             2                      SCHED POLICY:   parallel
LPs:                6005                   PPs:            12010
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    32
MOUNT POINT:        N/A                    LABEL:          usr1
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO
INFINITE RETRY:     no

I created a mirrored LV (Copies: 2), so next I want to create the JFS2 FS..

  Volume group name                                   vg_usr1
  SIZE of file system
          Unit Size                                   Megabytes
*         Number of units                            [768000]
* MOUNT POINT                                        [/usr1]
  Mount AUTOMATICALLY at system restart?              yes
  PERMISSIONS                                         read/write
  Mount OPTIONS                                      []
  Block Size (bytes)                                  4096
  Logical Volume for Log
  Inline Log size (MBytes)                           []
  Extended Attribute Format
  ENABLE Quota Management?                            no
  Enable EFS?                                         no
  Allow internal snapshots?                           no
  Mount GROUP                                        []

The error I get is..

Command: failed        stdout: yes           stderr: no
Before command completion, additional instructions may appear below.
0516-404 allocp: This system cannot fulfill the allocation request.
        There are not enough free partitions or not enough physical volumes
        to keep strictness and satisfy allocation requests.  The command
        should be retried with different allocation characteristics.
0516-822 mklv: Unable to create logical volume.
crfs: 0506-972 Cannot create logical volume for file system.
rmlv: Logical volume loglv00 is removed.

I have no idea what I am doing wrong? I am trying to maximize my usage of PP/LPs, so basically use it all up and assign the largest possible FS on the JFS2.

When creating the JFS2 FS, its asking for the Number of Units so if my LV has 6005 LPs, and my PP size is 128, 6005x128 .. 768640MB.. to leave a bit of breathing room I tried to set it to 768000 but the above error happens.

What am i doing wrong.. really would like to understand how to do this correctly starting from the PPs up to creating the JFS2.

Thank you for your help.

I suppose you fell for a very simple thing, but at one point this happened perhaps to all of us. As fas as i can tell from your posted output you are using SMIT to create the LVs and FSes.

In the respective SMIT dialogue there are two entries:

                                      Enhanced Journaled File Systems

Move cursor to desired item and press Enter.

  Add an Enhanced Journaled File System
  Add an Enhanced Journaled File System on a Previously Defined Logical Volume

Note that you need to use the second option when you have defined the LV already. Otherwise a new one will be created before a FS is created on it. This is perhaps where your error comes from.

I hope this helps.

bakunin

Oh wow, that worked. thank you

So just so I understand..

The process I took was to first verify my PVs were not assigned anywhere, then I took note of the PV's PP Size and Total PPs.

Next I proceeded to create the VG, in there I specified the PVs I want to use and give it a name, next I create the Logical Volume from the VG, here I can specify copies, max LPs and Number of LPs.

If my PV has 546 Total PPs (128MB per PP), then when I create the LV I can assume that LP has a 1:1 relation to PP on the disk? So (example) if I assign 10 PVs (disks) and I want to use maximum space, 10 (PVs) x 546 = 5460. When creating my LV my Number of LPs would be 5460, if I set to "2 copies" this would be half (2730)?

So that takes care of the LV (basically a partition in MSWIN world?), now we need to give it a filesystem, when using SMITTY and adding an enhanced JFS and we select the 2nd option (previously defined logical volume) it is by default consuming all space on the defined LV?

My initial approach (original post) when I was going to the first option, was creating a enhanced JFS, it would of attempted to create a new LV? Thus this is why I was running into those errors?

As a note, I did find this interesting..

# lsvg vg_usr1
VOLUME GROUP:       vg_usr1                  VG IDENTIFIER:  000015010000d60000000149d31c8e44
VG STATE:           active                   PP SIZE:        128 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      13104 (1677312 megabytes)
MAX LVs:            256                      FREE PPs:       13104 (1677312 megabytes)
LVs:                0                        USED PPs:       0 (0 megabytes)
OPEN LVs:           0                        QUORUM:         13 (Enabled)
TOTAL PVs:          24                       VG DESCRIPTORS: 24
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         24                       AUTO ON:        yes
MAX PPs per VG:     32512
MAX PPs per PV:     1016                     MAX PVs:        32
LTG size (Dynamic): 256 kilobyte(s)          AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable
PV RESTRICTION:     none                     INFINITE RETRY: no
DISK BLOCK SIZE:    512

As we see I have 13104 Free PPs in my VG, when I attempt to create my LV I get an error.

# /usr/sbin/mklv -y'lv_usr1' -t'jfs2' -c'2' -L'usr1' -x'32512' vg_usr1 6552
0516-032 lcreatelv: Illegal physical partition number.
0516-822 mklv: Unable to create logical volume.

2 Copies, 6552 LPs (which totals to 13104 LPs available in the vg_usr1).

So why the error? It Seemed to work when i set the LPs to 6000, however now I have 1103 free PPs that I was hoping to allocate to the same jfs2 fs/mount.

VOLUME GROUP:       vg_usr1                  VG IDENTIFIER:  000015010000d60000000149d3251b14
VG STATE:           active                   PP SIZE:        128 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      13104 (1677312 megabytes)
MAX LVs:            256                      FREE PPs:       1103 (141184 megabytes)
LVs:                2                        USED PPs:       12001 (1536128 megabytes)
OPEN LVs:           2                        QUORUM:         13 (Enabled)
TOTAL PVs:          24                       VG DESCRIPTORS: 24
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         24                       AUTO ON:        yes
MAX PPs per VG:     32512
MAX PPs per PV:     1016                     MAX PVs:        32
LTG size (Dynamic): 256 kilobyte(s)          AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable
PV RESTRICTION:     none                     INFINITE RETRY: no
DISK BLOCK SIZE:    512

Am I hitting some kind of limitation? Just want to understand this fully.

Thanks for your help as always.

Basically yes. The long answer is that you seem not to have a firm grasp of LVM concepts and terminology. See below.

Lets first deal with your other problem (with the big FS). Notice the difference in your provided output, i have snipped the important part:

MAX LVs:            256                      FREE PPs:       13104 (1677312 megabytes)
LVs:                0                        USED PPs:       0 (0 megabytes)

# /usr/sbin/mklv -y'lv_usr1' -t'jfs2' -c'2' -L'usr1' -x'32512' vg_usr1 6552
0516-032 lcreatelv: Illegal physical partition number.
0516-822 mklv: Unable to create logical volume.

MAX LVs:            256                      FREE PPs:       1103 (141184 megabytes)
LVs:                2                        USED PPs:       12001 (1536128 megabytes)

Notice that you have 2(!) and not one LVs after creating the first LV and you have 1103 PPs free after using 12000 out of the initial 13104. You have one LV more than expected and one PP less than expected, right?

The reason for this is that JFS is a JOURNALED filesystem and as such it needs a journal. Issue a

lsvg -l <vgname>

and you will see an (automatically created) LV of type "jfslog" with the size of 1 PP. This has been created automatically when you issued your "mklv" command to create a LV without an "inline" log. JFS2 can use either an external jfslog device (one per VG) or an inline log (one per LV). This is why your "mklv" command failed: the jfslog LV was created first and then you missed one PP to fulfill your size demands.

Here is a short introduction to LVM concepts. I suggest you download and study some of IBMs Redbooks to that.

We start with disks. What a "disk" is can be a wide variety of devices: physical disks, RAID sets, LUNs, etc., basically everything that has a "hdisk"-device entry in "lsdev". We will consider a disk to be a real, physical disk at first. (Let us start simple and go from there.)

The first thing we have to create is a "volume group". VGs are purely logical constructs, containers. You create a VG and give it at least one disk. A disk inside a VG is called "physical volume", PV. Classical VGs can hold up to 32 PVs (there are "scalable" and "big" VGss, but about that later). There are commands ("extendvg", "reducevg") to add disks to a VG or remove them from one.

When a disk is added to a VG it is sliced into "physical partitions". A PP is the smallest amount of disk space a VG can deal with. Classical VGs can have a maximum of 1019 PPs in one PV, therefore a classical VG is limited to 32 x 1019 PPs in size. The PP size is therefore a vital decision and one should better err to the big side than to the small side. Bigger PPs mean you might waste some space because you cannot add less than a single PP to a FS. Selecting the PP size too small means you cannot use larger disks. Unfortunately the PP size cannot be changed, you will have to wipe the whole VG and create it anew.

Once you have a VG you can create volumes, LVs: volumes are similar to "partitions" in PCs. Raw disk space,which can be used either for filesystems or other purposes (raw devices, swap space, ...). LVs are created from (in principle) anonymized PPs. When you add a disk to a VG it is sliced up into PP-sized chunks (PPs) and these chunks are put into a big bag. Then you create a LV and give it a certain amount of PPs. You do not need to know where they come from (which PV and position) although you can control this process to the utmost extent (by using map files). You can, but do not have to (and one usually doesn't).

Notice that inside the LV there are "logical partitions", LPs. These are the logical equivalent of PPs, but abstracted. If you use a LV with one copy (unmirrored) then each PP corresponds to one LP and vice versa. When you have 2 copies there are two PPs corresponding to one LP, 3 copies would mean 3 PPs corresponding to 1 LP. The VG has an attribute controlling the allocation of these PPs: "each copy on a separate PV". This means that every PP resenting the same LP should be located on a different PV (disk). Switch the attribute off and all copies might land on the same disk.

Finally you can put filesystems (and some other things) onto a LV or use it as a raw device. The latter is often preferred by database guys (for no good reason any more, because the performance gain is next to nil nowadays). You can also put a JFS or JFS2 (and some other filesystems) ont it, you can create a paging device or system dump device. Creating a FS means you format the LV and put some FS information onto it. You can create a FS directly, in which case the underlying LV is automatically created for you (this is what you tried above and what failed). You can also grow and shrink the FS (command chfs ), which will also automatically enlarge/shrink the underlying LV acccordingly. Notice that stiil the PP size is set. You can add 1 byte to the FS, but it would result in adding a whole LP (and therefore, depending on the LV copies, 1, 2 or 3 PPs) to the LV and subsequently to the FS. You can try it:

chfs -a size=+1 /some/FS

will grow "/some/FS" by not 1 byte but by the PP size.

I hope this helps.

bakunin