[linux-lvm] Saying goodbye to LVM

[Xen]

You are probably happy about this, but...

I managed to get LVM to corrupt my data on Ubuntu Xenial 3 times now.

All of that is related to:

- LVM not checking or behaving correctly when a duplicate PV appears
- LVM not checking or behaving correctly when a cache volume is out of
sync with its origin.

In addition the thin DM target of kernel 3.x was so buggy I couldn't
compile anything big without the system hanging.

I will probably become a ZFS user.

Goodbye, and thanks for all the fish.

[John Stoffel]

If you think zfs will be the answer... then good luck. So where did a new PV come from that had the same UUID as the existing ones? Basically the more details you give... then we can try to help.

But in any case, good luck.

Sent from my iPhone

Post by Xen
You are probably happy about this, but...
I managed to get LVM to corrupt my data on Ubuntu Xenial 3 times now.
- LVM not checking or behaving correctly when a duplicate PV appears
- LVM not checking or behaving correctly when a cache volume is out of sync with its origin.
In addition the thin DM target of kernel 3.x was so buggy I couldn't compile anything big without the system hanging.
I will probably become a ZFS user.
Goodbye, and thanks for all the fish.
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[Gionatan Danti]

Post by Xen
You are probably happy about this, but...
I managed to get LVM to corrupt my data on Ubuntu Xenial 3 times now.
- LVM not checking or behaving correctly when a duplicate PV appears
- LVM not checking or behaving correctly when a cache volume is out of
sync with its origin.
In addition the thin DM target of kernel 3.x was so buggy I couldn't
compile anything big without the system hanging.
I will probably become a ZFS user.
Goodbye, and thanks for all the fish.

I am both a LVM/ThinLVM and ZFS heavy user, so I hope to be impartial
here...

LVM and its lvmthin counterpart are *rock solid* in my experience,
except in the (very) edge case of full thin pool (this was lengthly
discussed in the past, and both Zednek and Jonathan gave accurate
suggestions on how to avoid that).

However, in my experience, the only distribution which keep updated
version of lvm kernel and user space utilities is RHEL/CentOS. I found
Debian and Ubuntu based distributions particularly *bad* at managing LVM
and device mapper targets in general.

I am not using lvmcache, so I can not speak for it.

That said, ZFS really is outstanding (especially checksum and
compression, albeit is sorely lacks reflinks). I really have high hopes
for stratis (https://github.com/stratis-storage), which plan to provide
ZFS-like feature using stacked device mapper targets (which our beloved
LVM targets on top).

Regards.

[p***@yahoo.com]

When do we run out of turtles?

  Original Message  
From: Gionatan Danti
Sent: Wednesday, February 7, 2018 13:50
To: LVM general discussion and development
Reply To: LVM general discussion and development
Cc: Xen
Subject: Re: [linux-lvm] Saying goodbye to LVM

‎ally have high hopes
for stratis (https://github.com/stratis-storage), which plan to provide
ZFS-like feature using stacked device mapper targets (which our beloved
LVM targets on top)

[Xen]

Post by Gionatan Danti
I am both a LVM/ThinLVM and ZFS heavy user, so I hope to be impartial
here...
LVM and its lvmthin counterpart are *rock solid* in my experience,
except in the (very) edge case of full thin pool (this was lengthly
discussed in the past, and both Zednek and Jonathan gave accurate
suggestions on how to avoid that).
However, in my experience, the only distribution which keep updated
version of lvm kernel and user space utilities is RHEL/CentOS. I found
Debian and Ubuntu based distributions particularly *bad* at managing
LVM and device mapper targets in general.

This is the reason for the problems but LVM released bad products all
the same with the solution being to not use them for very long, or
rather, to upgrade.

Yes Ubuntu runs a long time behind and Debian also.

As a user, I can't help that, upgrading LVM just like that to have less
"there is a pit here but we won't tell you about that" simply seems also
fraught with peril.

For example, upgrading LVM slightly to 160 caused udev problems I didn't
have before.

So you can blame the distributions, you can also blame features being
released first and proper protection only being added much later.

So if you're on Xenial, you are stuck with the features but without the
protection.

In particular there is a quagmire of situations you can end up with wrt
the shielding and dual activation of the same vg, many times of which
you can only get out of the situation with dmsetup remove, but I didn't
know this at first.

Or you end up in the situation where a PV is missing and you cannot edit
the VG, but in order to remove the PV you have to edit the VG.

A missing cache device cannot be removed without the missing cache
device being present.

I meant to say, you can have 2 disks out of sync and resolving it is not
possible other than by editing config on disk and doing vgcfgrestore.

But you can't do vgcfgrestore without removing a missing PV first.

There is a huge amount of chicken and egg problems because physically a
VG sits inside a PV but logically a PV sits inside a VG and this
constantly causes issues.

LVM just has conceptual problems.

Post by Gionatan Danti
That said, ZFS really is outstanding (especially checksum and
compression, albeit is sorely lacks reflinks). I really have high
hopes for stratis (https://github.com/stratis-storage), which plan to
provide ZFS-like feature using stacked device mapper targets (which
our beloved LVM targets on top).

I cannot write more yet because I don't have ZFS setup yet.

I don't like ZFS too much because it's opaque and Linux support seems to
be flimsy (for example boot support) and the only good documentation is
Oracle but it often does not apply.

[Gionatan Danti]

Post by Xen
This is the reason for the problems but LVM released bad products all
the same with the solution being to not use them for very long, or
rather, to upgrade.
Yes Ubuntu runs a long time behind and Debian also.
As a user, I can't help that, upgrading LVM just like that to have
less "there is a pit here but we won't tell you about that" simply
seems also fraught with peril.
For example, upgrading LVM slightly to 160 caused udev problems I
didn't have before.
So you can blame the distributions, you can also blame features being
released first and proper protection only being added much later.
So if you're on Xenial, you are stuck with the features but without
the protection.
In particular there is a quagmire of situations you can end up with
wrt the shielding and dual activation of the same vg, many times of
which you can only get out of the situation with dmsetup remove, but I
didn't know this at first.
Or you end up in the situation where a PV is missing and you cannot
edit the VG, but in order to remove the PV you have to edit the VG.
A missing cache device cannot be removed without the missing cache
device being present.
I meant to say, you can have 2 disks out of sync and resolving it is
not possible other than by editing config on disk and doing
vgcfgrestore.
But you can't do vgcfgrestore without removing a missing PV first.
There is a huge amount of chicken and egg problems because physically
a VG sits inside a PV but logically a PV sits inside a VG and this
constantly causes issues.
LVM just has conceptual problems.

As a CentOS user, I *never* encountered such problems. I really think
these are caused by the lack of proper integration testing from
Debian/Ubuntu. But hey - all key LVM developers are RedHat people, so it
should be expected (for the better/worse).

I can not say anything about lvmcache, though.

Post by Xen
I cannot write more yet because I don't have ZFS setup yet.
I don't like ZFS too much because it's opaque and Linux support seems
to be flimsy (for example boot support) and the only good
documentation is Oracle but it often does not apply.

True. I never use it with boot device. LVM and XFS are, on the other
hand, extremely well integrated into mainline kernel/userspace
utilities.

Hence my great interest in stratis...
Regards.

[Xen]

Post by Gionatan Danti

Post by Xen
LVM just has conceptual problems.

As a CentOS user, I *never* encountered such problems. I really think
these are caused by the lack of proper integration testing from
Debian/Ubuntu.

That would only apply to udev/boot problems, not the tooling issues.

If you never make DD copies, you never run into such issues.

And if you don't use Cache you won't have those missing PV issues
either.

Maybe I am just great at finding missing features but LVM has in the end
cost me a lot more time than it has saved me.

I mean, if I had just stuck to regular partitions I would have been
further ahead in life by now ;-).

Including any lack of LVM expertise I would have had by then. Which, in
the end, I don't think is worth it.

Post by Gionatan Danti
But hey - all key LVM developers are RedHat people, so
it should be expected (for the better/worse).

The denialist nature of Linux people ensures that even if LVM upstream
says UPGRADE, Ubuntu will say "why? everything works fine for me".

Or, "I never ran into such issues" ;-).

Post by Gionatan Danti
True. I never use it with boot device.

Even on Solaris it is limited, for example the root pool cannot have an
external log device (that means SLOG). Then, you have no clue if this is
also going to be the case on Linux or not ;-). And Grub supports booting
from a root dataset but only barely, I don't think anything else (e.g. a
ZVOL) is any kind of realism. The biggest downside is inflexibility in
shrinking pools, and people complain about ZVOL snapshots requiring a
lot of space.

Btrfs, on the other hand, supports removing disks from raid sets and
just reorganizing what's left.

Post by Gionatan Danti
LVM and XFS are, on the other
hand, extremely well integrated into mainline kernel/userspace
utilities.

Except that apparently there are (or were, or can be) extreme
initramfs/udev issues and the Ubuntu support/integration has been flimsy
at best -- what's not flimsy is Grub support, it will even load an
embedded LVM just fine.

I mean you can have an LV on a PV that is an LV on a PV and Grub will be
able to read it, the Ubuntu initramfs will not.

Post by Gionatan Danti
Hence my great interest in stratis...

I don't deny you there but I wonder if I'm not better off sticking to
ordinary partitions ;-).

But my main idea is to use compressed ZVOLs if I can.

You can just stick partition tables on those too. ZFS has a lot of
different "models".

[matthew patton]

Post by Xen
Yes Ubuntu runs a long time behind and Debian also.
As a user, I can't help that, upgrading LVM just like that to have less
... seems also fraught with peril.
So if you're on Xenial, you are stuck with the features but without the protection.

Don't use a distro that is sloppy about looking out for the end-user? I can't speak for how 'stable' LVM is in their releases or if anyone expends effort to identify "golden" releases vs "preview / dangerous" releases...

Post by Xen
particular there is a quagmire of situations ... only get out of the situation with dmsetup remove,
but I didn't know this at first.

Which is part of the problem, really. You were trying to do 'expert' things without the requisite background or knowledge. Now granted we learn by shooting ourselves in the foot from time to time but the blood loss is never pleasant. You either learn quickly or you pick another tool that has spent the time to put proper dependency/protection in place such that 'EXPERTS ONLY NEED APPLY' is not the phrase of the day.

I think LVM does need to step it up in putting proper safeguards in place before calling a feature release-worthy. If you have a feature then it is REQUIRED that it also have ALL of the necessary dependency check logic fully implemented. Otherwise it is a BUG and has no business being published in a manner that careless distros can activate it.

If multiple PVs show up with duplicate ID or other bad characteristics there needs to be a well documented mechanism for rectifying the situation written from the POV of a user/sysadmin and not a developer who happens to know all the inner workings. Are metadata time-stamped or otherwise versioned so even if you see what looks to be a conflict the 'age' of the metadata can give an indication which one is likely to be correct? If not, why not?

Post by Xen
A missing cache device cannot be removed without the missing cache device being present.

That's retarded. Is this a correct characterization? If so, how did that get past the release audit?