[linux-lvm] Underlying physical volume resized?!

[Gionatan Danti]

Dear all,
I have a question about a warning message LVM is showing. I am using
CentOS 6.8 x86_64. Here is my pvs/vgs/lvs configuration:

pvs:
PV VG Fmt Attr PSize PFree
/dev/sda2 vg lvm2 a--u 9,51g 0
/dev/sda3 vg lvm2 a--u 110,00g 0
/dev/sdb1 vg lvm2 a--u 120,00g 4,00g

vgs:
VG #PV #LV #SN Attr VSize VFree
vg 3 2 0 wz--n- 239,50g 4,00g

lvs:
LV VG Attr LSize Pool Origin Data% Meta% Move Log
Cpy%Sync Convert
lv_root vg -wi-ao---- 233,56g
lv_swap vg -wi-ao---- 1,94g

Both at kernel discovery (during boot) and when issuing lvs or other LVM
commands I have the warning:

"Device /dev/sda2 has size of 19945392 sectors which is smaller than
corresponding PV size of 19945472 sectors. Was device resized? One or
more devices used as PVs in VG vg have changed sizes."

However, my partitions where never shrunk. fdisk on /dev/sda:

Disk /dev/sda: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders, total 251658240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a3c73

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 20971439 9972696 8e Linux LVM
/dev/sda3 20971440 251658239 115343400 8e Linux LVM

By fdisk output, lvm is right: 20971439-1026048+1=19945392, so sda2 is
80 sectors (40 KB) smaller than lvm expects (19945472 sectors). I
expanded the root volume quite a few times; however, I *never* resized
sda2: at each expansion I took a snapshot of the disk's MBR, so I
already checked that I did not mess with sda2 in the past.

By using lvm metadata archived in /etc/lvm/archive, I think that the
missing 80 sectors are squarely in the swap space (lv_swap, which used
the last physical extents when lv_root was much smaller), but I am
somewhat worried that, given some process write to these "missing" 80
sectors, something bad can happen to the next adjacent physical extent
(where live data are store, as it is part of lv_root now).

In short:

1) can someone point me on what happen here, and why lvs only recently
started to complain?

2) do you think my data are at risk?

3) what I can do to solve the problem? I can think of two different
approach: a) run pvreside /dev/sda2 to shrunk the physical volume or b)
shrunk the swap partition to be sure nobody will ever write to the last
80 sectors.

Thanks.

[Marian Csontos]

Post by Gionatan Danti
Dear all,
I have a question about a warning message LVM is showing. I am using
PV VG Fmt Attr PSize PFree
/dev/sda2 vg lvm2 a--u 9,51g 0
/dev/sda3 vg lvm2 a--u 110,00g 0
/dev/sdb1 vg lvm2 a--u 120,00g 4,00g
VG #PV #LV #SN Attr VSize VFree
vg 3 2 0 wz--n- 239,50g 4,00g
LV VG Attr LSize Pool Origin Data% Meta% Move Log
Cpy%Sync Convert
lv_root vg -wi-ao---- 233,56g
lv_swap vg -wi-ao---- 1,94g
Both at kernel discovery (during boot) and when issuing lvs or other LVM
"Device /dev/sda2 has size of 19945392 sectors which is smaller than
corresponding PV size of 19945472 sectors. Was device resized? One or
more devices used as PVs in VG vg have changed sizes."

Hi, the warning was added only recently - commit c0912af3 added 2016-01-22:

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=c0912af3104cb72ea275d90b8b1d68a25a9ca48a

Were the partitions created by anaconda?
This might be an installer bug.

LVM allows overriding PV size to be larger than device size.

More about the feature here:
https://bugzilla.redhat.com/show_bug.cgi?id=1323950

I do not think pvmove could help here, as there will be read errors on
the device.

Deactivating swap (`swapoff /dev/$VG/$LV`)
deleting the swap LV (`lvremove $VG/$LV`)
shrinking the PV (simply running `pvresize /dev/sda2` without arguments),
recreating the swap LV (`lvcreate -n $LV -L $SIZE $VG`),
`mkswap /dev/$VG/$LV`
and `swapon /dev/$VG/$LV`
should do.

-- Marian

Post by Gionatan Danti
Disk /dev/sda: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders, total 251658240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a3c73
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 20971439 9972696 8e Linux LVM
/dev/sda3 20971440 251658239 115343400 8e Linux LVM
By fdisk output, lvm is right: 20971439-1026048+1=19945392, so sda2 is
80 sectors (40 KB) smaller than lvm expects (19945472 sectors). I
expanded the root volume quite a few times; however, I *never* resized
sda2: at each expansion I took a snapshot of the disk's MBR, so I
already checked that I did not mess with sda2 in the past.
By using lvm metadata archived in /etc/lvm/archive, I think that the
missing 80 sectors are squarely in the swap space (lv_swap, which used
the last physical extents when lv_root was much smaller), but I am
somewhat worried that, given some process write to these "missing" 80
sectors, something bad can happen to the next adjacent physical extent
(where live data are store, as it is part of lv_root now).
1) can someone point me on what happen here, and why lvs only recently
started to complain?
2) do you think my data are at risk?
3) what I can do to solve the problem? I can think of two different
approach: a) run pvreside /dev/sda2 to shrunk the physical volume or b)
shrunk the swap partition to be sure nobody will ever write to the last
80 sectors.
Thanks.

[Gionatan Danti]

Post by Marian Csontos
https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=c0912af3104cb72ea275d90b8b1d68a25a9ca48a

Ok, this explain why I only recently saw this warning

Post by Marian Csontos
Were the partitions created by anaconda?
This might be an installer bug.
LVM allows overriding PV size to be larger than device size.
https://bugzilla.redhat.com/show_bug.cgi?id=1323950
I do not think pvmove could help here, as there will be read errors on
the device.

This a cloud VM not directly installed by me, rather from the cloud
provider. I think they somewhat messed up their template partition table
but, with no warning and no apparent error, the problem was unrecognized

Post by Marian Csontos
Deactivating swap (`swapoff /dev/$VG/$LV`)
deleting the swap LV (`lvremove $VG/$LV`)
shrinking the PV (simply running `pvresize /dev/sda2` without arguments),
recreating the swap LV (`lvcreate -n $LV -L $SIZE $VG`),
`mkswap /dev/$VG/$LV`
and `swapon /dev/$VG/$LV`
should do.

This is a production machine, so I am somewhat reluctant to shrink
swap/physical volumes. From what I understand (and from my own testing
on a test machine) this slight discrepancy between partition and PV
*should* pose no problems to my setup. At most, when swap is full, I can
receive an error about a failed page swapping. Is this assumption correct?

Thank you very much.