[linux-lvm] vgdisplay - checksum error - what does it mean?

[Tomasz Chmielewski]

Recently, I used "vgdisplay", and noticed that it gives a "checksum error":

# vgdisplay
/dev/sda2: Checksum error
--- Volume group ---
VG Name LVM2
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 49
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 34
Open LV 34
Max PV 0
Cur PV 1
Act PV 1
VG Size 743.16 GB
PE Size 4.00 MB
Total PE 190250
Alloc PE / Size 175807 / 686.75 GB
Free PE / Size 14443 / 56.42 GB
VG UUID yV2ADe-MeJd-dHbd-ExLP-AIu3-OTA2-u46qhl


Should I be scared? What does it mean? What should I do about it? I
wouldn't like to loose the data.

If it helps, my setup looks like that:

HDD1-sda2-\
HDD2-sdb2-|__RAID-10--LVM-2
HDD3-sdc2-|
HDD4-sdd2-/

I'm running 2.6.17.8 kernel.

[Tomasz Chmielewski]

Post by Tomasz Chmielewski
# vgdisplay
/dev/sda2: Checksum error
--- Volume group ---
VG Name LVM2
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 49
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 34
Open LV 34
Max PV 0
Cur PV 1
Act PV 1
VG Size 743.16 GB
PE Size 4.00 MB
Total PE 190250
Alloc PE / Size 175807 / 686.75 GB
Free PE / Size 14443 / 56.42 GB
VG UUID yV2ADe-MeJd-dHbd-ExLP-AIu3-OTA2-u46qhl
Should I be scared? What does it mean? What should I do about it? I
wouldn't like to loose the data.
HDD1-sda2-\
HDD2-sdb2-|__RAID-10--LVM-2
HDD3-sdc2-|
HDD4-sdd2-/
I'm running 2.6.17.8 kernel.

...

Hmmm... isn't it something that tells about the history of that system?

# lvmdiskscan | egrep '(sd|md)'
/dev/sda2: Checksum error
/dev/md0 [ 1.03 GB]
/dev/md1 [ 1.03 GB]
/dev/sda2 [ 371.58 GB] LVM physical volume
/dev/md2 [ 743.16 GB] LVM physical volume
/dev/sdb2 [ 371.58 GB]
/dev/sdc2 [ 371.58 GB]
/dev/sdd2 [ 371.58 GB]


/dev/md0 is RAID-1 -> swap
/dev/md1 is RAID-1 -> root filesystem
/dev/md2 is RAID-10 -> all LVM volumes


So this basically means, that LVM was set up on /dev/sda2 some time ago,
but it was never removed from there - instead, RAID-10 was set up on
that partition?

Should I do something to fix the things? What?

[Luca Berra]

Post by Tomasz Chmielewski

Post by Tomasz Chmielewski
# vgdisplay
/dev/sda2: Checksum error

....

Post by Tomasz Chmielewski

Post by Tomasz Chmielewski
Should I be scared? What does it mean? What should I do about it? I
wouldn't like to loose the data.
HDD1-sda2-\
HDD2-sdb2-|__RAID-10--LVM-2
HDD3-sdc2-|
HDD4-sdd2-/
I'm running 2.6.17.8 kernel.

...

Post by Tomasz Chmielewski
So this basically means, that LVM was set up on /dev/sda2 some time ago,
but it was never removed from there - instead, RAID-10 was set up on
that partition?

I don't think so. if sda2 is part of a raid10 md array probably the
beginning sector of the md device maps to the beginning sector of the
real device, hence lvm will find an lvm signature on /dev/sda2.

Post by Tomasz Chmielewski
Should I do something to fix the things? What?

yes, re-enable md_component_detection in lvm.conf, why did you disable
that?

L.

[Tomasz Chmielewski]

Post by Luca Berra

Post by Tomasz Chmielewski

Post by Tomasz Chmielewski
# vgdisplay
/dev/sda2: Checksum error

....

Post by Tomasz Chmielewski

Post by Tomasz Chmielewski
Should I be scared? What does it mean? What should I do about it? I
wouldn't like to loose the data.
HDD1-sda2-\
HDD2-sdb2-|__RAID-10--LVM-2
HDD3-sdc2-|
HDD4-sdd2-/
I'm running 2.6.17.8 kernel.

...

Post by Tomasz Chmielewski
So this basically means, that LVM was set up on /dev/sda2 some time
ago, but it was never removed from there - instead, RAID-10 was set up
on that partition?

I don't think so. if sda2 is part of a raid10 md array probably the
beginning sector of the md device maps to the beginning sector of the
real device, hence lvm will find an lvm signature on /dev/sda2.

...

Is there a way to check if it's really the case?

There's something wrong with /dev/sda2 - lvmdiskscan claims it's a
371.58 GB LVM physical volume, while /dev/md2 is the physical volume I use.

/dev/sda2 [ 371.58 GB] LVM physical volume
/dev/md2 [ 743.16 GB] LVM physical volume

Post by Luca Berra

Post by Tomasz Chmielewski
Should I do something to fix the things? What?

yes, re-enable md_component_detection in lvm.conf, why did you disable
that?

Certainly I didn't touch anything in /etc/lvm/*.
If I look into /etc/lvm/lvm.conf, it says:

devices {
(...)
# By default, LVM2 will ignore devices used as components of
# software RAID (md) devices by looking for md superblocks.
# 1 enables; 0 disables.
md_component_detection = 1
}

It's enabled.

So the problem is somewhere else. Where?


BTW, the machine is running Debian etch (ARM port).

"smartctl" says all four disks are fine (they are quite new, too), so
it's definitely not a hardware problem.


I guess one way to fix it would be mark all partitions faulty on
/dev/sda, and then, to recreate the RAIDs.
But I'm curious to know how could I handle such a situation if I didn't
have RAID.

[Luca Berra]

Post by Tomasz Chmielewski

Post by Luca Berra

Post by Tomasz Chmielewski
So this basically means, that LVM was set up on /dev/sda2 some time
ago, but it was never removed from there - instead, RAID-10 was set up
on that partition?

I don't think so. if sda2 is part of a raid10 md array probably the
beginning sector of the md device maps to the beginning sector of the
real device, hence lvm will find an lvm signature on /dev/sda2.

Is there a way to check if it's really the case?

dd?

Post by Tomasz Chmielewski
There's something wrong with /dev/sda2 - lvmdiskscan claims it's a
371.58 GB LVM physical volume, while /dev/md2 is the physical volume I use.
/dev/sda2 [ 371.58 GB] LVM physical volume
/dev/md2 [ 743.16 GB] LVM physical volume

Post by Luca Berra

Post by Tomasz Chmielewski
Should I do something to fix the things? What?

yes, re-enable md_component_detection in lvm.conf, why did you disable
that?

Certainly I didn't touch anything in /etc/lvm/*.
devices {
(...)
# By default, LVM2 will ignore devices used as components of
# software RAID (md) devices by looking for md superblocks.
# 1 enables; 0 disables.
md_component_detection = 1
}
It's enabled.
So the problem is somewhere else. Where?

...

with md_component_detection enabled lvm should ignore /dev/sda2.
can you send the output of lvmdiskscan -vvv?

Post by Tomasz Chmielewski
BTW, the machine is running Debian etch (ARM port).

mmm i don't have an arm maybe this is the issue why lvm does not
understand that sda2 is a md component?!?

Post by Tomasz Chmielewski
"smartctl" says all four disks are fine (they are quite new, too), so
it's definitely not a hardware problem.

sure it is not an hw problem do not worry.

Post by Tomasz Chmielewski
I guess one way to fix it would be mark all partitions faulty on
/dev/sda, and then, to recreate the RAIDs.
But I'm curious to know how could I handle such a situation if I didn't
have RAID.

what would it fix? you have no problem except lvm2 believing sda2 is an
lvm volume which is not.

[Alasdair G Kergon]

Post by Luca Berra

Post by Tomasz Chmielewski
BTW, the machine is running Debian etch (ARM port).

mmm i don't have an arm maybe this is the issue why lvm does not
understand that sda2 is a md component?!?

I've not followed this thread, but you must use version 2.02.21 or later
for the ARM architecture with md (or backport the patch).

Alasdair

[Tomasz Chmielewski]

Post by Alasdair G Kergon

Post by Luca Berra

Post by Tomasz Chmielewski
BTW, the machine is running Debian etch (ARM port).

mmm i don't have an arm maybe this is the issue why lvm does not
understand that sda2 is a md component?!?

I've not followed this thread, but you must use version 2.02.21 or later
for the ARM architecture with md (or backport the patch).

Hmm, I must try that.

I'll let the list know in a week or so.

[Tomasz Chmielewski]

Post by Tomasz Chmielewski

Post by Alasdair G Kergon

Post by Luca Berra

Post by Tomasz Chmielewski
BTW, the machine is running Debian etch (ARM port).

mmm i don't have an arm maybe this is the issue why lvm does not
understand that sda2 is a md component?!?

I've not followed this thread, but you must use version 2.02.21 or later
for the ARM architecture with md (or backport the patch).

Hmm, I must try that.
I'll let the list know in a week or so.

Hmm, I can't compile that:

(...)
gcc -c -I. -I../include -DHAVE_CONFIG_H -fPIC -Wall -Wundef -Wshadow
-Wcast-align -Wwrite-strings -Wmissing-prototypes -Wmissing-declarations
-Wnested-externs -Winline -Wmissing-noreturn -O2 striped/striped.c -o
striped/striped.o
report/report.c:67: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:67: warning: its scope is only this definition or
declaration, which is probably not what you want
report/report.c:67: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_string_disp?:
report/report.c:69: warning: implicit declaration of function
?dm_report_field_string?
report/report.c:69: warning: nested extern declaration of
?dm_report_field_string?
report/report.c: At top level:
report/report.c:74: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:74: warning: ?struct dm_report? declared inside
parameter list
report/report.c:83: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:83: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_devices_disp?:
report/report.c:116: warning: implicit declaration of function ?dm_snprintf?
report/report.c:116: warning: nested extern declaration of ?dm_snprintf?
report/report.c:139: warning: implicit declaration of function
?dm_report_field_set_value?
report/report.c:139: warning: nested extern declaration of
?dm_report_field_set_value?
report/report.c: At top level:
report/report.c:146: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:146: warning: ?struct dm_report? declared inside
parameter list
report/report.c:176: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:176: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_modules_disp?:
report/report.c:189: warning: passing argument 1 of ?_tags_disp? from
incompatible pointer type
report/report.c:189: warning: passing argument 3 of ?_tags_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:194: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:194: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_vgfmt_disp?:
report/report.c:203: warning: passing argument 1 of ?_string_disp? from
incompatible pointer type
report/report.c:203: warning: passing argument 3 of ?_string_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:208: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:208: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_pvfmt_disp?:
report/report.c:218: warning: passing argument 1 of ?_string_disp? from
incompatible pointer type
report/report.c:218: warning: passing argument 3 of ?_string_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:223: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:223: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_lvkmaj_disp?:
report/report.c:230: warning: implicit declaration of function
?dm_report_field_int?
report/report.c:230: warning: nested extern declaration of
?dm_report_field_int?
report/report.c:232: warning: implicit declaration of function
?dm_report_field_uint64?
report/report.c:232: warning: nested extern declaration of
?dm_report_field_uint64?
report/report.c: At top level:
report/report.c:237: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:237: warning: ?struct dm_report? declared inside
parameter list
report/report.c:251: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:251: warning: ?struct dm_report? declared inside
parameter list
report/report.c:336: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:336: warning: ?struct dm_report? declared inside
parameter list
report/report.c:362: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:362: warning: ?struct dm_report? declared inside
parameter list
report/report.c:406: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:406: warning: ?struct dm_report? declared inside
parameter list
report/report.c:421: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:421: warning: ?struct dm_report? declared inside
parameter list
report/report.c:435: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:435: warning: ?struct dm_report? declared inside
parameter list
report/report.c:453: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:453: warning: ?struct dm_report? declared inside
parameter list
report/report.c:487: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:487: warning: ?struct dm_report? declared inside
parameter list
report/report.c:506: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:506: warning: ?struct dm_report? declared inside
parameter list
report/report.c:536: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:536: warning: ?struct dm_report? declared inside
parameter list
report/report.c:565: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:565: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_vgsize_disp?:
report/report.c:572: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:572: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:577: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:577: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_segstart_disp?:
report/report.c:584: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:584: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:589: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:589: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_segsize_disp?:
report/report.c:596: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:596: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:601: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:601: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_chunksize_disp?:
report/report.c:611: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:611: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:616: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:616: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_pvused_disp?:
report/report.c:627: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:627: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:632: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:632: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_pvfree_disp?:
report/report.c:643: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:643: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:648: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:648: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_pvsize_disp?:
report/report.c:659: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:659: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:664: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:664: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_devsize_disp?:
report/report.c:672: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:672: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:677: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:677: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_vgfree_disp?:
report/report.c:684: warning: passing argument 1 of ?_size64_disp? from
incompatible pointer type
report/report.c:684: warning: passing argument 3 of ?_size64_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:689: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:689: warning: ?struct dm_report? declared inside
parameter list
report/report.c:709: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:709: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_uint32_disp?:
report/report.c:711: warning: implicit declaration of function
?dm_report_field_uint32?
report/report.c:711: warning: nested extern declaration of
?dm_report_field_uint32?
report/report.c: At top level:
report/report.c:716: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:716: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_int32_disp?:
report/report.c:718: warning: implicit declaration of function
?dm_report_field_int32?
report/report.c:718: warning: nested extern declaration of
?dm_report_field_int32?
report/report.c: At top level:
report/report.c:723: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:723: warning: ?struct dm_report? declared inside
parameter list
report/report.c: In function ?_lvsegcount_disp?:
report/report.c:730: warning: passing argument 1 of ?_uint32_disp? from
incompatible pointer type
report/report.c:730: warning: passing argument 3 of ?_uint32_disp? from
incompatible pointer type
report/report.c: At top level:
report/report.c:735: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:735: warning: ?struct dm_report? declared inside
parameter list
report/report.c:785: warning: ?struct dm_report_field? declared inside
parameter list
report/report.c:785: warning: ?struct dm_report? declared inside
parameter list
report/report.c:856: error: array type has incomplete element type
report/report.c:873: error: array type has incomplete element type
In file included from report/report.c:874:
report/columns.h:21: error: ?DM_REPORT_FIELD_TYPE_STRING? undeclared
here (not in a function)
report/columns.h:24: error: ?DM_REPORT_FIELD_TYPE_NUMBER? undeclared
here (not in a function)
report/report.c: In function ?report_init?:
report/report.c:889: error: ?DM_REPORT_OUTPUT_ALIGNED? undeclared (first
use in this function)
report/report.c:889: error: (Each undeclared identifier is reported only
once
report/report.c:889: error: for each function it appears in.)
report/report.c:892: error: ?DM_REPORT_OUTPUT_BUFFERED? undeclared
(first use in this function)
report/report.c:895: error: ?DM_REPORT_OUTPUT_HEADINGS? undeclared
(first use in this function)
report/report.c:897: warning: implicit declaration of function
?dm_report_init?
report/report.c:897: warning: nested extern declaration of ?dm_report_init?
report/report.c:898: warning: return makes pointer from integer without
a cast
report/report.c: In function ?report_object?:
report/report.c:916: warning: implicit declaration of function
?dm_report_object?
report/report.c:916: warning: nested extern declaration of
?dm_report_object?
make[1]: *** [report/report.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/root/lvm2/LVM2.2.02.22/lib'
make: *** [lib] Error 2

[Luca Berra]

Post by Tomasz Chmielewski

Post by Tomasz Chmielewski

Post by Alasdair G Kergon

Post by Luca Berra

Post by Tomasz Chmielewski
BTW, the machine is running Debian etch (ARM port).

mmm i don't have an arm maybe this is the issue why lvm does not
understand that sda2 is a md component?!?

I've not followed this thread, but you must use version 2.02.21 or later
for the ARM architecture with md (or backport the patch).

Hmm, I must try that.
I'll let the list know in a week or so.

(...)
gcc -c -I. -I../include -DHAVE_CONFIG_H -fPIC -Wall -Wundef -Wshadow
-Wcast-align -Wwrite-strings -Wmissing-prototypes -Wmissing-declarations
-Wnested-externs -Winline -Wmissing-noreturn -O2 striped/striped.c -o
striped/striped.o
report/report.c:67: warning: ?struct dm_report_field? declared inside
parameter list

...

Did you update device-mapper as well?

Anyway the patch to support both endian versions of the md superblock is
very small and can be applied to lvm earlier than 2.02.21

http://sources.redhat.com/cgi-bin/cvsweb.cgi/LVM2/lib/device/dev-md.c.diff?r1=1.3&r2=1.4&cvsroot=lvm2

L.

[Tomasz Chmielewski]

Post by Luca Berra

Post by Tomasz Chmielewski

Post by Tomasz Chmielewski

Post by Alasdair G Kergon

Post by Luca Berra

Post by Tomasz Chmielewski
BTW, the machine is running Debian etch (ARM port).

mmm i don't have an arm maybe this is the issue why lvm does not
understand that sda2 is a md component?!?

I've not followed this thread, but you must use version 2.02.21 or later
for the ARM architecture with md (or backport the patch).

Hmm, I must try that.
I'll let the list know in a week or so.

(...)
gcc -c -I. -I../include -DHAVE_CONFIG_H -fPIC -Wall -Wundef -Wshadow
-Wcast-align -Wwrite-strings -Wmissing-prototypes
-Wmissing-declarations -Wnested-externs -Winline -Wmissing-noreturn
-O2 striped/striped.c -o striped/striped.o
report/report.c:67: warning: ?struct dm_report_field? declared inside
parameter list

Did you update device-mapper as well?
Anyway the patch to support both endian versions of the md superblock is
very small and can be applied to lvm earlier than 2.02.21
http://sources.redhat.com/cgi-bin/cvsweb.cgi/LVM2/lib/device/dev-md.c.diff?r1=1.3&r2=1.4&cvsroot=lvm2

...

Curiously, I updated lvm to Debian's 2.02.06-3, rebooted (machine was in
a rather weird state), and the issue is gone.

With Debian's libdevmapper-dev 1.02.08-1 it doesn't compile; I didn't
try to upgrade it anymore.

[Tomasz Chmielewski]

Tomasz Chmielewski schrieb:

(...)

Post by Tomasz Chmielewski
Curiously, I updated lvm to Debian's 2.02.06-3, rebooted (machine was in
a rather weird state), and the issue is gone.
With Debian's libdevmapper-dev 1.02.08-1 it doesn't compile; I didn't
try to upgrade it anymore.

As described in another post from today ("moving logical volumes to
another system *remotely* - how?"), perhaps the issue had nothing to do
with lvm tools, but with kernel?

I use 2.6.17.8 on that ARM machine, the kernel oopses after I create a
snapshot, invalidate it (make it full), and try to remove it.

I'll try to upgrade to a newest stable kernel to see if anything changed
(it will take some time, though).