Create volume using LVM over 2 physical disks

I wanted to know how we can combine volumes over 2 physical drives.

# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         131     1052226   83  Linux
/dev/sda2             132        5221    40885425   8e  Linux LVM

Disk /dev/sdb: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       13054   104856223+  8e  Linux LVM

Disk /dev/sdc: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

I am creating.

Filesystem Name	Size (In GB)
/u01	                50
/u02	                60
/u03	                10

Also was wondering if sdb has all its space of 107GB as there looks to be a Logical volume already configured there.

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVolRoot
                       19G  2.3G   16G  13% /
/dev/mapper/VolGroup00-LogVolTmp
                      9.5G  151M  8.9G   2% /tmp
/dev/mapper/VolGroup00-LogVolVar
                      4.8G  191M  4.3G   5% /var
/dev/sda1             996M   41M  904M   5% /boot
tmpfs                 6.9G     0  6.9G   0% /dev/shm

It looks like you already created the physical volumes. What does pvs display?

It's not totally clear what you are asking, but if I understand, you want to "combine" the disks /dev/sdb and /dev/sdc. You can do this in as much as adding them to the same volume group.

# vgcreate myvg01 /dev/sdb /dev/sdc

You can then create three logical volumes, one each for /u01, /u02 and /u03

i.e.

[root@RH631d ~]# lvcreate -L10G -n mylv01 myvg01
  Logical volume "mylv01" created

Then you can create the filesystem using the LV's.

(far from optimal, but quick for me to show...)

[root@RH631d ~]# mkfs -t ext3 /dev/mapper/myvg01-mylv01
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1310720 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
[root@RH631d ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2              3960348   2536864    196924  93% /
/dev/sda5             14301224    167312  13395728   2% /home
/dev/sda1                46633     11178     33047  26% /boot
tmpfs                   512360         0    512360   0% /dev/shm
.host:/              829127300 740882960  88244340  90% /mnt/hgfs
[root@RH631d ~]# mkdir /u01
[root@RH631d ~]# mount /dev/mapper/myvg01-mylv01 /u01
[root@RH631d ~]# df -hP
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             3.8G  2.5G  193M  93% /
/dev/sda5              14G  164M   13G   2% /home
/dev/sda1              46M   11M   33M  26% /boot
tmpfs                 501M     0  501M   0% /dev/shm
.host:/               791G  707G   85G  90% /mnt/hgfs
/dev/mapper/myvg01-mylv01  9.9G  151M  9.2G   2% /u01

THanks for reply.
I f I want part of the physical disk to have 2 volumes.
example: /dev/sdb device will have /dev/sdb1 and dev/sdb2

Now what I am doing is first assigning u02 (/dev/sdb1) with 60G and whatever is remaining to u01(/sdb/u02)... I got error after doing below steps while creating u01. The error is after the procedure mentioned bellow

fdisk /dev/sdb
n
p
1


t
8e
w

# Create new LVM Physical Volume
pvcreate /dev/sdb1


# Extend LVM Volume Group to new Physical Volume
vgextend VolGroup00 /dev/sdb1                       

# Make /u04 folder
mkdir -p /u02

# Create VolGroup00-LogVolU02
lvcreate -L 60G -n LogVolU02 VolGroup00 

# Format newly created LogVolu02
mkfs -text3 /dev/VolGroup00/LogVolU02

# Add LogVolU02 to /etc/fstab
echo -e "/dev/VolGroup00/LogVolU02\t/u02\t\text3\tdefaults\t1 2" >> /etc/fstab

# Mount all Volumes
mount -a

# pvcreate /dev/sdb2
  Device /dev/sdb2 not found (or ignored by filtering).

what have I done wrong.

I don't see the point, really, in partitioning a physical volume in this way. You either use LVM or you don't. I also don't remember the last time I used fdisk since using LVM!

Why not just give sdb over to the logical volume manager and create logical volumes for u01, u02 and u03? That's what LVM is for. fdisk is, IMO, archaic and past its time.

Now I need to shut down my VM, add a disk and try to remember how to use fdisk!. BRB :slight_smile:

well I am creating u01, u02 and u03 as LVM's. u01 and u02 will be on /dev/sdb
and u03 on /dev/sdc.

u02 will be 60G
u01 All remaining space on sdb

This is what I am trying to achieve

Ah, OK. Think I've got it.

[root@RH631d ~]# fdisk /dev/sdd
--- created primary partition 1 --- now /dev/sdd1
--- created primary partition 2 --- now /dev/sdd2

[root@RH631d ~]# pvcreate /dev/sdd1
  Physical volume "/dev/sdd1" successfully created
[root@RH631d ~]# pvcreate /dev/sdd2
  Physical volume "/dev/sdd2" successfully created
[root@RH631d ~]# lvmdiskscan
  /dev/ramdisk       [       16.00 MB] 
  /dev/myvg01/mylv01 [       10.00 GB] 
  /dev/ram           [       16.00 MB] 
  /dev/sda1          [       47.03 MB] 
  /dev/myvg01/mylv02 [       10.00 GB] 
  /dev/ram2          [       16.00 MB] 
  /dev/root          [        3.90 GB] 
  /dev/ram3          [       16.00 MB] 
  /dev/sda3          [        1.97 GB] 
  /dev/ram4          [       16.00 MB] 
  /dev/ram5          [       16.00 MB] 
  /dev/sda5          [       14.08 GB] 
  /dev/ram6          [       16.00 MB] 
  /dev/ram7          [       16.00 MB] 
  /dev/ram8          [       16.00 MB] 
  /dev/ram9          [       16.00 MB] 
  /dev/ram10         [       16.00 MB] 
  /dev/ram11         [       16.00 MB] 
  /dev/ram12         [       16.00 MB] 
  /dev/ram13         [       16.00 MB] 
  /dev/ram14         [       16.00 MB] 
  /dev/ram15         [       16.00 MB] 
  /dev/sdb           [       50.00 GB] LVM physical volume
  /dev/sdc           [       50.00 GB] LVM physical volume
  /dev/sdd1          [        2.30 GB] LVM physical volume
  /dev/sdd2          [        1.53 GB] LVM physical volume
  3 disks
  19 partitions
  2 LVM physical volume whole disks
  2 LVM physical volumes
[root@RH631d ~]# vgcreate myvg02 /dev/sdd1 /dev/sdd2
  Volume group "myvg02" successfully created

That's not really what you asked, but it was useful for me :slight_smile:

It looks to me like your sdb1 is using up the whole disk. That doesn't help you if you want sdb1 to be 110GB and sdb2 60GB. If sdb1 is using all the space, you can't create an sdb2 on that disk.

You never said what pvs showed. And while you're there, what does lvmdiskscan show? It's only for information, nothing else.

I would suggest, if you know that sdb1 is not being used, you remove it and repartition the disk with the sizes you want.

Over that, I would recommend this:

  • Remove "sdb1" using "fdisk" - actually I would remove the PV and rescan
  • Create a volume group called "oraclevg", or whatever it's for, using both sdb, and another for sdc - i.e. let LVM handle it.
  • Create logical volumes for u02 as 60 GB as whatever is left (100%FREE) from sdb
  • Create another logical volume using sdc for u03

I changed the blocks allotted. So was able to create 2 partitions in /sdb. But still get partition error

# pvcreate /dev/sdb2
  Device /dev/sdb2 not found (or ignored by filtering).

]# fdisk -l

Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         131     1052226   83  Linux
/dev/sda2             132        5221    40885425   8e  Linux LVM

Disk /dev/sdb: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        8705    69922881   8e  Linux LVM
/dev/sdb2            8706       13054    34933342+  8e  Linux LVM



# pvs
  /dev/hdc: open failed: No medium found
  PV         VG         Fmt  Attr PSize  PFree
  /dev/sda2  VolGroup00 lvm2 a-   38.97G     0
  /dev/sdb1  VolGroup00 lvm2 a-   99.97G 39.97G


# lvmdiskscan
  /dev/ramdisk               [       16.00 MB]
  /dev/root                  [       19.53 GB]
  /dev/ram                   [       16.00 MB]
  /dev/sda1                  [        1.00 GB]
  /dev/VolGroup00/LogVolTmp  [        9.75 GB]
  /dev/ram2                  [       16.00 MB]
  /dev/sda2                  [       38.99 GB] LVM physical volume
  /dev/VolGroup00/LogVolVar  [        4.88 GB]
  /dev/ram3                  [       16.00 MB]
  /dev/VolGroup00/LogVolSwap [        4.81 GB]
  /dev/ram4                  [       16.00 MB]
  /dev/VolGroup00/LogVolU02  [       60.00 GB]
  /dev/ram5                  [       16.00 MB]
  /dev/ram6                  [       16.00 MB]
  /dev/ram7                  [       16.00 MB]
  /dev/ram8                  [       16.00 MB]
  /dev/ram9                  [       16.00 MB]
  /dev/ram10                 [       16.00 MB]
  /dev/ram11                 [       16.00 MB]
  /dev/ram12                 [       16.00 MB]
  /dev/ram13                 [       16.00 MB]
  /dev/ram14                 [       16.00 MB]
  /dev/ram15                 [       16.00 MB]
  /dev/sdb1                  [       66.68 GB] LVM physical volume
  /dev/sdc                   [       20.00 GB]
  7 disks
  16 partitions
  0 LVM physical volume whole disks
  2 LVM physical volumes

No help, just amusing. The "dumb message of the day" award goes to fdisk

Command (m for help): t
Partition number (1-5): 5
Partition 5 does not exist yet!

Command (m for help): t
Partition number (1-5): 4
Hex code (type L to list codes): 8e
You cannot change a partition into an extended one or vice versa
Delete it first.

I'm not experiencing the problems you are with pvcreate. Can you remove the PV and rescan it and start from scratch? (unless, of course, someone has a better idea, or, of course, sdb1 is actually being used!).

... I tried creating an extended partition to see if that caused a problem, then got that silly 'error'. It's of no consequence to your issue, just amusing.

Regarding the pvcreate error: do you know if your lvm.conf has been manually modified? If you do any multipathing chances are that it has been (otherwise you'd constantly be getting duplicate PV errors in /var/log/messages). You may check /etc/lvm/lvm.conf and see if you have a filter set up.

I think I messed up by deleting the physical volumes.
I was getting errors

# vgscan -v
    Wiping cache of LVM-capable devices
    Wiping internal VG cache
  Reading all physical volumes.  This may take a while...
    Finding all volume groups
    Finding volume group "VolGroup00"
  Couldn't find device with uuid 'UPkdKc-Ki6u-5GUf-BY8u-WABy-ssZ6-lkPjJl'.
  Couldn't find device with uuid 'bJaD2H-hx8h-dmD5-AgAk-RvHD-utl0-Gc9ooK'.
  Couldn't find device with uuid 'eh2fqQ-RjQZ-eLOc-jVnL-oSd3-j8o2-JBEQny'.
    There are 3 physical volumes missing.
  Found volume group "VolGroup00" using metadata type lvm2

So removed these devices using: vgreduce --removemissing
And doing a reconfigure.

Now if I need to start and have 2 Logical Volumes on /sdb
thats /dev/sdb1 and /dev/sdb2. What steps are needed.

fdisk /dev/sdb
pvcreate /dev/sdb1
pvcreate /dev/sdb2  --- not sure if we need this

vgextend VolGroup00 /dev/sdb1
vgextend VolGroup00 /dev/sdb2

lvcreate -L 60G -n LogVolU02 VolGroup00
lvcreate -l 100%FREE -n LogVolU01 VolGroup00

Would this be correct?

Yes, that looks about right.

But I still don't see the point of both partitioning with fdisk and then again with logical volumes. You'd be as well just using fdisk to specify ext3, or whichever FS type you want, then creating a filesystem without LVM, since it would be just as easy to create three primary fdisk partitions for your filesystems, or easier, actually, than creating two and then using LVM to carve it up further. And given that sdb1 and sdb2 are on the same physical disk - and I noticed that sdc seems to have been quietly dropped(!), you're not really gaining anything in terms of performance or reliability with the hybrid fdisk / LVM approach.

sdc I dropped as it is a different disk and will be an entirely used for sdc1.

I am not sure how to partition /sdb into 2
If it can be done without fdisk. Can you please guide as to what steps are needed. THe reason I have been using fdisk in beginning is I saw it online.

How To Setup Logical Volume Manager (LVM) Step By Step | RouteMyBrain

In the article, he is using fdisk to partition the disk to use partition type "8e" - Linux LVM.

If you allocate all space on sdb to a single primary partition (like you had before), you can then use LVM to create your volume group, and lvcreate to create the sizes of logical volumes you need.

I've never used fdisk to do this. Normally, just presenting the disk to the system and using LVM commands to create PV's, VG's and LV's is enough.

So, I just added a new disk to my VM (BTW, you didn't say if you have a physical server or a virtual one):

[root@RH631d ~]# lvmdiskscan
  /dev/ramdisk       [       16.00 MB] 
  /dev/myvg01/mylv01 [       10.00 GB] 
  /dev/ram           [       16.00 MB] 
  /dev/sda1          [       47.03 MB] 
  /dev/myvg01/mylv02 [       10.00 GB] 
  /dev/ram2          [       16.00 MB] 
  /dev/root          [        3.90 GB] 
  /dev/ram3          [       16.00 MB] 
  /dev/sda3          [        1.97 GB] 
  /dev/ram4          [       16.00 MB] 
  /dev/ram5          [       16.00 MB] 
  /dev/sda5          [       14.08 GB] 
  /dev/ram6          [       16.00 MB] 
  /dev/ram7          [       16.00 MB] 
  /dev/ram8          [       16.00 MB] 
  /dev/ram9          [       16.00 MB] 
  /dev/ram10         [       16.00 MB] 
  /dev/ram11         [       16.00 MB] 
  /dev/ram12         [       16.00 MB] 
  /dev/ram13         [       16.00 MB] 
  /dev/ram14         [       16.00 MB] 
  /dev/ram15         [       16.00 MB] 
  /dev/sdb           [       50.00 GB] LVM physical volume
  /dev/sdc           [       50.00 GB] LVM physical volume
  /dev/sdd1          [        2.30 GB] LVM physical volume
  /dev/sdd2          [        1.53 GB] LVM physical volume
  /dev/sdd3          [      784.42 MB] LVM physical volume
  /dev/sde           [       20.00 GB] 
  4 disks
  19 partitions
  2 LVM physical volume whole disks
  3 LVM physical volumes

[root@RH631d ~]# pvcreate /dev/sde
  Physical volume "/dev/sde" successfully created

[root@RH631d ~]# vgcreate VolumeGroup00 /dev/sde
  Volume group "VolumeGroup00" successfully created

[root@RH631d ~]# lvcreate -L5G -n LogicalVolume01 VolumeGroup00
  Logical volume "LogicalVolume01" created

[root@RH631d ~]# lvcreate -L7G -n LogicalVolume02 VolumeGroup00
  Logical volume "LogicalVolume02" created

[root@RH631d ~]# lvcreate -l100%FREE -n LogicalVolume03 VolumeGroup00
  Logical volume "LogicalVolume03" created

[root@RH631d ~]# lvs
  LV              VG            Attr   LSize  Origin Snap%  Move Log Copy%  Convert
  LogicalVolume01 VolumeGroup00 -wi-a-  5.00G                                      
  LogicalVolume02 VolumeGroup00 -wi-a-  7.00G                                      
  LogicalVolume03 VolumeGroup00 -wi-a-  8.00G 

No need to fdisk. No need for fancy partitioning. I can now create my filesystems using these three LV's and I don't have to worry about it.

If, later on, I want to extend these filesystems, I can add a new disk, extend the VG, extend the LV, then easily extend the FS. It's hard to do that if you have partitioned everything, absolutely, with fdisk (although I suppose you could still extend the VGs with new disks, despite the strange fdisk partitioning).

THANKS Scott for you help.

Do we need to install LVM package, I have an Ubuntu VM machine. Will try and use LVM on this VM.

No. I can see from the lvmdiskscan output that you posted, you are good to go!

The issue was on RHEL 5.5 which is on VMWare