[linux-lvm] Identifying useable block devices

[Marius Vollmer]

Hi,

when looking at the udev properties of a device mapper node, how can I
decide whether this is a block device that the user can use for creating
filesystems on, etc?

For example, when creating a thinly provisioned logical volume with these
steps

# vgcreate TEST /dev/loop1
# lvcreate TEST --thinpool pool -L 80
# lvcreate -T TEST/pool -n thin -V 100

I end up with a lot of devices:

# ls /dev/mapper/TEST-*
/dev/mapper/TEST-pool /dev/mapper/TEST-pool_tmeta /dev/mapper/TEST-thin
/dev/mapper/TEST-pool_tdata /dev/mapper/TEST-pool-tpool

How can a program tell that only /dev/mapper/TEST-thin can really be
used as a block device, and the rest should be ignored?

Is there a way to do this by looking at "udevadm info", for example?

(What seems to work is skipping all nodes that have
DM_UDEV_IGNORE_DISK_RULES_FLAG set to true. Is this maybe even
documented somewhere?)

Thanks!

[Alasdair G Kergon]

On Wed, Jan 15, 2014 at 10:19:33AM +0200, Marius Vollmer wrote:
> How can a program tell that only /dev/mapper/TEST-thin can really be
> used as a block device, and the rest should be ignored?

/dev/mapper is meant only for internal use.

Use /dev/$vg/$lv instead (in the case of LVM).
Internal LVs shouldn't appear there.

Alasdair

[Oliver Rath]

Hi Marius!

the /dev/mapper/.. devices are for internal use only. This is needed,
because the thinPOOL is a device logical between vg and lv, which is
represented with help of these special devices.

Normally you should alway use the /dev/vg/lv name scheme for you own
lvm-devices. Nevertheless, if you would persist to use /dev/mapper/
pathes, you can create the path with /dev/mapper/<vg>-<lv>. Note: If you
lv-name includes some "-", they will be doubled in /dev/mapper notation
for differentiating to the "-", which separates vg and lv name. I.e. you
LV is name my-own-lv and your VG is mariusvg, then you can find the
(thintpool-) device under /dev/mapper/mariusvg-my--own--lv and (better)
as link /dev/mariusvg/my-own-lv.

There is a point you have unfortunatly to use the /dev/mapper-notation:
If you need i.e. kpartx for creating subdevices (i.e. the lv includes a
virtual windows disk with partitions), then the temporalily created
names are only accessible via /dev/mapper.

The other created metadevices for thinpool-managment you should never touch.

Hth,
Oliver


On 15.01.2014 09:19, Marius Vollmer wrote:
> Hi,
>
> when looking at the udev properties of a device mapper node, how can I
> decide whether this is a block device that the user can use for creating
> filesystems on, etc?
>
> For example, when creating a thinly provisioned logical volume with these
> steps
>
> # vgcreate TEST /dev/loop1
> # lvcreate TEST --thinpool pool -L 80
> # lvcreate -T TEST/pool -n thin -V 100
>
> I end up with a lot of devices:
>
> # ls /dev/mapper/TEST-*
> /dev/mapper/TEST-pool /dev/mapper/TEST-pool_tmeta /dev/mapper/TEST-thin
> /dev/mapper/TEST-pool_tdata /dev/mapper/TEST-pool-tpool
>
> How can a program tell that only /dev/mapper/TEST-thin can really be
> used as a block device, and the rest should be ignored?
>
> Is there a way to do this by looking at "udevadm info", for example?
>
> (What seems to work is skipping all nodes that have
> DM_UDEV_IGNORE_DISK_RULES_FLAG set to true. Is this maybe even
> documented somewhere?)
>
> Thanks!
>
> _______________________________________________
> linux-lvm mailing list
> linux-***@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[Anatoly Pugachev]

On Wed, Jan 15, 2014 at 8:17 PM, Oliver Rath <***@mglug.de> wrote:
> Hi Marius!
>
> the /dev/mapper/.. devices are for internal use only.

If so, how come most commands use it (df, mount, findmnt, etc..) and
not /dev/vg/vol notation?

Thanks!

[Paul B. Henson]

> Anatoly Pugachev
> Sent: Wednesday, January 15, 2014 12:25 PM
>
> > the /dev/mapper/.. devices are for internal use only.
>
> If so, how come most commands use it (df, mount, findmnt, etc..) and
> not /dev/vg/vol notation?

Because /dev/vg/vol is just a symlink to /dev/mapper/vg-vol, and mount
dereferences it and uses the authoritative name when it actually mounts it.

It can be rather confusing to have an fstab full of /dev/vg/foo but then see
/dev/mapper/vg-foo listed by mount, it does tend to make one think they
should just use /dev/mapper paths directly.

[Peter Rajnoha]

On 01/16/2014 02:32 AM, Paul B. Henson wrote:
>> Anatoly Pugachev
>> Sent: Wednesday, January 15, 2014 12:25 PM
>>
>>> the /dev/mapper/.. devices are for internal use only.
>>
>> If so, how come most commands use it (df, mount, findmnt, etc..) and
>> not /dev/vg/vol notation?
>
> Because /dev/vg/vol is just a symlink to /dev/mapper/vg-vol, and mount
> dereferences it and uses the authoritative name when it actually mounts it.

This applies only if the symlinks/nodes are created directly with
libdevmapper - which is the old classical way. If udev is used to manage /dev
content for dm/lvm devices then the /dev/vg/vol is a symlink to /dev/dm-X
(also /dev/mapper/vg-vol is a symlink to /dev/dm-X). And dm-X is not a
stable name - it can be different across reboots and activation/deactivation
cycle (hence it's never used in scripts etc.). So you have to be very careful
when dereferencing symlinks.

As for mount, findmnt and other utilities from util-linux - they all use
one common code which can handle this situation (e.g. if you call
"findmnt /dev/vg/vol", it shows you /dev/mapper/vg-vol on output, not
the dm-X the /dev/vg/vol is referencing actually).
--
Peter

[Paul B. Henson]

> Peter Rajnoha
> Sent: Wednesday, January 15, 2014 9:43 PM
>
> This applies only if the symlinks/nodes are created directly with
> libdevmapper - which is the old classical way.
> If udev is used to manage /dev
> content for dm/lvm devices then the /dev/vg/vol is a symlink to /dev/dm-X
> (also /dev/mapper/vg-vol is a symlink to /dev/dm-X). And dm-X is not a

Hmm, yah, I see on my RHEL6 boxes that is indeed the case, as opposed to my
older RHEL5 boxes.

> As for mount, findmnt and other utilities from util-linux - they all use
> one common code which can handle this situation (e.g. if you call
> "findmnt /dev/vg/vol", it shows you /dev/mapper/vg-vol on output, not
> the dm-X the /dev/vg/vol is referencing actually).

IIRC, this thread started when somebody was being advised *not* to use the
/dev/mapper path, but rather to use the /dev/vg path. If there is special
case handling in util-linux, why isn't it returning what is being advertised
as the preferable path to use rather than the one that is said to be
internal and not to be referenced?

[Peter Rajnoha]

On 01/16/2014 10:03 PM, Paul B. Henson wrote:
>> Peter Rajnoha
>> Sent: Wednesday, January 15, 2014 9:43 PM
>>
>> This applies only if the symlinks/nodes are created directly with
>> libdevmapper - which is the old classical way.
>> If udev is used to manage /dev
>> content for dm/lvm devices then the /dev/vg/vol is a symlink to /dev/dm-X
>> (also /dev/mapper/vg-vol is a symlink to /dev/dm-X). And dm-X is not a
>
> Hmm, yah, I see on my RHEL6 boxes that is indeed the case, as opposed to my
> older RHEL5 boxes.
>
>> As for mount, findmnt and other utilities from util-linux - they all use
>> one common code which can handle this situation (e.g. if you call
>> "findmnt /dev/vg/vol", it shows you /dev/mapper/vg-vol on output, not
>> the dm-X the /dev/vg/vol is referencing actually).
>
> IIRC, this thread started when somebody was being advised *not* to use the
> /dev/mapper path, but rather to use the /dev/vg path. If there is special
> case handling in util-linux, why isn't it returning what is being advertised
> as the preferable path to use rather than the one that is said to be
> internal and not to be referenced?

Well, that is probably an RFE for util-linux to give priority to
/dev/<vgname>/<lvname> instead of /dev/mapper/<vgname>-<lvname>
if it's available...

Karel, would that be possbible for you to add? (though I expect
you probably don't want to add any additional and specific logic
for dm devices :) )

--
Peter

[Karel Zak]

On Fri, Jan 17, 2014 at 08:54:15AM +0100, Peter Rajnoha wrote:
> On 01/16/2014 10:03 PM, Paul B. Henson wrote:
> >> Peter Rajnoha
> >> Sent: Wednesday, January 15, 2014 9:43 PM
> >>
> >> This applies only if the symlinks/nodes are created directly with
> >> libdevmapper - which is the old classical way.
> >> If udev is used to manage /dev
> >> content for dm/lvm devices then the /dev/vg/vol is a symlink to /dev/dm-X
> >> (also /dev/mapper/vg-vol is a symlink to /dev/dm-X). And dm-X is not a
> >
> > Hmm, yah, I see on my RHEL6 boxes that is indeed the case, as opposed to my
> > older RHEL5 boxes.
> >
> >> As for mount, findmnt and other utilities from util-linux - they all use
> >> one common code which can handle this situation (e.g. if you call
> >> "findmnt /dev/vg/vol", it shows you /dev/mapper/vg-vol on output, not
> >> the dm-X the /dev/vg/vol is referencing actually).
> >
> > IIRC, this thread started when somebody was being advised *not* to use the
> > /dev/mapper path, but rather to use the /dev/vg path. If there is special
> > case handling in util-linux, why isn't it returning what is being advertised
> > as the preferable path to use rather than the one that is said to be
> > internal and not to be referenced?
>
> Well, that is probably an RFE for util-linux to give priority to
> /dev/<vgname>/<lvname> instead of /dev/mapper/<vgname>-<lvname>
> if it's available...
>
> Karel, would that be possbible for you to add? (though I expect
> you probably don't want to add any additional and specific logic
> for dm devices :) )

I have spent years to move from /dev/dm-N to /dev/mapper/<name> ... :-)

Note that /dev/mapper/<name> paths are in many cases composed from
/sys/block/dm-<N>/dm/name and this sysfs file has been added (by
Milan) just to support conversion from random DM paths (or devno) to
the canonicalized /dev/mapper/<name>.

How I can convert maj:min or /dev/dm-N to /dev/vg/vol path? Is the
volume group name (always) in /sys/block/dm-<N>/dm/name and (always)
separated by "-" ?

It also seems that this is LVM specific, for non-LVM devices (for
example LUKS) we want to use /dev/mapper/<name>.

Karel

--
Karel Zak <***@redhat.com>
http://karelzak.blogspot.com

[Peter Rajnoha]

On 01/17/2014 10:29 AM, Karel Zak wrote:
> On Fri, Jan 17, 2014 at 08:54:15AM +0100, Peter Rajnoha wrote:
>> Well, that is probably an RFE for util-linux to give priority to
>> /dev/<vgname>/<lvname> instead of /dev/mapper/<vgname>-<lvname>
>> if it's available...
>>
>> Karel, would that be possbible for you to add? (though I expect
>> you probably don't want to add any additional and specific logic
>> for dm devices :) )
>
> I have spent years to move from /dev/dm-N to /dev/mapper/<name> ... :-)
>
> Note that /dev/mapper/<name> paths are in many cases composed from
> /sys/block/dm-<N>/dm/name and this sysfs file has been added (by
> Milan) just to support conversion from random DM paths (or devno) to
> the canonicalized /dev/mapper/<name>.
>
> How I can convert maj:min or /dev/dm-N to /dev/vg/vol path? Is the
> volume group name (always) in /sys/block/dm-<N>/dm/name and (always)
> separated by "-" ?
>

It would require more steps for LVM:
- you would need to look at the /sys/block/dm-<N>/dm/uuid
- if the UUID has "LVM-" prefix, it's LVM device
- the you would need to translate the /sys/block/dm-<N>/dm/name
into its VG and LV part (it's always separated by "-", if the
VG or LV name itself contains "-", escaped form of this char is used
which is "--").
- then check whether /dev/<vgname>/<lvname> exists and use it if yes

Similarly for any other DM subsystem that has its own naming
logic and priorities with respect to /dev content (not sure at the
moment if mpath or cryptsetup has any, probably not).

> It also seems that this is LVM specific, for non-LVM devices (for
> example LUKS) we want to use /dev/mapper/<name>.

Sure, this is LVM specific only (as well as it would be for any other
DM subsystem). The only thing here is to make people use the more
approriate /dev/<vgname>/<lvname> instead of /dev/mapper because
otherwise if they see /dev/mapper on output, they just use this.

However, frankly, I can imagine that someone somewhere already uses
(incorrectly) the comparison of the path found on output of util-linux
utilities with "mapper" to say that this is the dm device and make
decisions based on that - that can become broken then...

This is more a cosmetic change. The exact same logic can be used
in all those scripts that try to identify which LV is for internal
use only (and so they should avoid processing the device). But in that
case, this logic must be used in all those places around, having that
in util-linux would make it a bit more centric.

As an interface, I can imagine keeping the /dev/mapper as the primary
name and then have a brand new field like "preferred name" or so...
Just a suggestion.

--
Peter

[Peter Rajnoha]

On 01/15/2014 09:19 AM, Marius Vollmer wrote:
> # ls /dev/mapper/TEST-*
> /dev/mapper/TEST-pool /dev/mapper/TEST-pool_tmeta /dev/mapper/TEST-thin
> /dev/mapper/TEST-pool_tdata /dev/mapper/TEST-pool-tpool
>
> How can a program tell that only /dev/mapper/TEST-thin can really be
> used as a block device, and the rest should be ignored?
>
> Is there a way to do this by looking at "udevadm info", for example?
>
> (What seems to work is skipping all nodes that have
> DM_UDEV_IGNORE_DISK_RULES_FLAG set to true. Is this maybe even
> documented somewhere?)

Those flags were primarily targeted to direct udev processing.
But it could be used as DM_UDEV_DISABLE_DISK_RULES_FLAG should
be always set for internal devices...

This flag actually means that udev should skip scanning the device
for metadata and it should skip creating any symlinks in /dev/disk
directory - which is a directory with symlink names based on metadata
found (e.g. filesystem uuid, subsystem names etc.).

However, for your purpose, I'd better use DM_UDEV_DISABLE_OTHER_RULES_FLAG
which just tells that everything else other than DM/LVM related should
skip this device.

For now, these flags are only documented directly in libdevmapper.h
(as they were only meant to direct udev rules and these situations
were all audited directly by communicating with other teams).
I could probably add a few lines to the man page directly though as
others could use this even when reading udev database...

Anyway, the most important rule for avoiding internal devices and
using the right ones is to follow /dev/<vgname>/<lvname> existence,
as already said in this thread.

--
Peter

[Marius Vollmer]

[ I am not subscribed, so please keep me in CC. I'll just reply to
myself, sorry for breaking the threading.
]

Peter Rajnoha wrote:

> For now, these flags are only documented directly in libdevmapper.h
> (as they were only meant to direct udev rules and these situations
> were all audited directly by communicating with other teams). I could
> probably add a few lines to the man page directly though as others
> could use this even when reading udev database...

That would be great!

> However, for your purpose, I'd better use
> DM_UDEV_DISABLE_OTHER_RULES_FLAG which just tells that everything else
> other than DM/LVM related should skip this device.

Hmm, DM_UDEV_DISABLE_OTHER_RULES_FLAG is (now) set for thin volumes, as
far as I can tell. This is what lead me down this rabbit hole in the
first place: UDisks2 _does_ ignore events for nodes with
DM_UDEV_DISABLE_OTHER_RULES_FLAG set, and since Fedora 20, this causes
it to ignore thin volumes.

The use of DM_UDEV_DISABLE_OTHER_RULES_FLAG or any other such flag in
UDisks2 looked like a ugly hack to me, so I started looking for
alternatives.

The best option seemed to be to ignore any DISABLE flag in UDisks, and
to set UDISKS_IGNORE for LVM2 block devices that do not have the
/dev/VG/LV symlink.

Now you say that DM_UDEV_DISABLE_OTHER_RULES_FLAG is actually the Right
Way, but it seems to be buggy re thin volumes. Correct?

(Of course, UDisks2 should not ignore _events_ but should ignore
_nodes_. Otherwise, it will get confused when a node acquires a DISABLE
flag later on, which happens to thin pools. What a mess! :-)

[Marius Vollmer]

Marius Vollmer <***@redhat.com> writes:

> Hmm, DM_UDEV_DISABLE_OTHER_RULES_FLAG is (now) set for thin volumes, as
> far as I can tell.

For what it's worth, the DM_UDEV_DISABLE_OTHER_RULES_FLAG disappears
when cycling the volume to inactive and back to active.

[Peter Rajnoha]

On 01/17/2014 02:35 PM, Marius Vollmer wrote:
> Marius Vollmer <***@redhat.com> writes:
>
>> Hmm, DM_UDEV_DISABLE_OTHER_RULES_FLAG is (now) set for thin volumes, as
>> far as I can tell.
>
> For what it's worth, the DM_UDEV_DISABLE_OTHER_RULES_FLAG disappears
> when cycling the volume to inactive and back to active.
>

Yes, exactly - that's because the flag is set only during LV creation,
not on further deactivation/activation cycle - it's just about avoiding
any scanning before the LV is properly initialized during its
creation as described in my previous post...

> _______________________________________________
> linux-lvm mailing list
> linux-***@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>


--
Peter

[Peter Rajnoha]

On 01/17/2014 11:02 AM, Marius Vollmer wrote:
> [ I am not subscribed, so please keep me in CC. I'll just reply to
> myself, sorry for breaking the threading.
> ]
>
> Peter Rajnoha wrote:
>
>> For now, these flags are only documented directly in libdevmapper.h
>> (as they were only meant to direct udev rules and these situations
>> were all audited directly by communicating with other teams). I could
>> probably add a few lines to the man page directly though as others
>> could use this even when reading udev database...
>
> That would be great!
>
>> However, for your purpose, I'd better use
>> DM_UDEV_DISABLE_OTHER_RULES_FLAG which just tells that everything else
>> other than DM/LVM related should skip this device.
>
> Hmm, DM_UDEV_DISABLE_OTHER_RULES_FLAG is (now) set for thin volumes, as
> far as I can tell. This is what lead me down this rabbit hole in the
> first place: UDisks2 _does_ ignore events for nodes with
> DM_UDEV_DISABLE_OTHER_RULES_FLAG set, and since Fedora 20, this causes
> it to ignore thin volumes.
>
> The use of DM_UDEV_DISABLE_OTHER_RULES_FLAG or any other such flag in
> UDisks2 looked like a ugly hack to me, so I started looking for
> alternatives.
>
> The best option seemed to be to ignore any DISABLE flag in UDisks, and
> to set UDISKS_IGNORE for LVM2 block devices that do not have the
> /dev/VG/LV symlink.
>
> Now you say that DM_UDEV_DISABLE_OTHER_RULES_FLAG is actually the Right
> Way, but it seems to be buggy re thin volumes. Correct?
>

Thing here is that when LVs are created then at first they have this flag
set until proper initialization is finished - meaning zeroing of any existing
signatures found on the volume before this LV can be used cleanly (otherwise,
it could happen that some scanning done outside LVM could find stale metadata
on just created LV, like FS labels, MD signatures.. whatever that might pose
a confusion about what is layed on top of the LV). Only after the signature
wiping is done, the flag is dropped and so others are free to use it as the
LV is clean now.

However, you're right that in case of thin LVs, this is set incorrectly.
The DM_UDEV_DISABLE_OTHER_RULES flag should not be there. I've fixed that:

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=91b26b63b408b7d7b4f79f0754d190783874e4cc

Thanks for spotting this!
--
Peter

[Peter Rajnoha]

On 01/20/2014 12:49 PM, Peter Rajnoha wrote:
> On 01/17/2014 11:02 AM, Marius Vollmer wrote:
>> [ I am not subscribed, so please keep me in CC. I'll just reply to
>> myself, sorry for breaking the threading.
>> ]
>>
>> Peter Rajnoha wrote:
>>
>>> For now, these flags are only documented directly in libdevmapper.h
>>> (as they were only meant to direct udev rules and these situations
>>> were all audited directly by communicating with other teams). I could
>>> probably add a few lines to the man page directly though as others
>>> could use this even when reading udev database...
>>
>> That would be great!
>>
>>> However, for your purpose, I'd better use
>>> DM_UDEV_DISABLE_OTHER_RULES_FLAG which just tells that everything else
>>> other than DM/LVM related should skip this device.
>>
>> Hmm, DM_UDEV_DISABLE_OTHER_RULES_FLAG is (now) set for thin volumes, as
>> far as I can tell. This is what lead me down this rabbit hole in the
>> first place: UDisks2 _does_ ignore events for nodes with
>> DM_UDEV_DISABLE_OTHER_RULES_FLAG set, and since Fedora 20, this causes
>> it to ignore thin volumes.
>>
>> The use of DM_UDEV_DISABLE_OTHER_RULES_FLAG or any other such flag in
>> UDisks2 looked like a ugly hack to me, so I started looking for
>> alternatives.
>>
>> The best option seemed to be to ignore any DISABLE flag in UDisks, and
>> to set UDISKS_IGNORE for LVM2 block devices that do not have the
>> /dev/VG/LV symlink.
>>
>> Now you say that DM_UDEV_DISABLE_OTHER_RULES_FLAG is actually the Right
>> Way, but it seems to be buggy re thin volumes. Correct?
>>
>
> Thing here is that when LVs are created then at first they have this flag
> set until proper initialization is finished - meaning zeroing of any existing
> signatures found on the volume before this LV can be used cleanly (otherwise,
> it could happen that some scanning done outside LVM could find stale metadata
> on just created LV, like FS labels, MD signatures.. whatever that might pose
> a confusion about what is layed on top of the LV). Only after the signature
> wiping is done, the flag is dropped and so others are free to use it as the
> LV is clean now.
>
> However, you're right that in case of thin LVs, this is set incorrectly.
> The DM_UDEV_DISABLE_OTHER_RULES flag should not be there. I've fixed that:

...the flag would be gone if you opened the LV for read-write and then
close it - you don't even need to write anything to the LV, just open
and close - this would fire an event that would have dropped the flag.
It was (incorrectly) supposed to be the LVM itself that would open the
new thin LV for signature wiping - that would be exactly the "open for
RW"/"close" sequence that would have dropped the flag.
Though in this exact case the wiping is not done - the thin LV is an
exception here and the code handled this incorrectly...

--
Peter

[Marius Vollmer]

Peter Rajnoha <***@redhat.com> writes:

>> Thing here is that when LVs are created then at first they have this flag
>> set until proper initialization is finished - [...]

Is it guaranteed (modulo bugs) that the DM_UDEV_DISABLE_*_RULES flags
are only ever removed from a node, and are never added to it over it's
lifetime between add/remove events?

This isn't true right now, and UDisks fails to handle it correctly when
a flag is added in a "change" event. I am asking to figure out where
the fix should go.

For example, this is what happens when creating a pool for thinly
provisioned volumes:

UDEV [2081.714175] add /devices/virtual/block/dm-1 (block)
[...]
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
[...]

UDEV [2081.771737] change /devices/virtual/block/dm-1 (block)
[...]
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
[...]

UDEV [2081.779997] change /devices/virtual/block/dm-1 (block)
[...]
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
[...]

UDEV [2081.943224] change /devices/virtual/block/dm-1 (block)
[...]
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
[...]

I.e., the flags are set, then are removed, then set again. UDisks
ignores the change event with the flags set, and gets into an
inconsistent state.

(With my limited understanding, I would say that LVM2 should make the
guarantee and fix the above scenario.)

> ...the flag would be gone if you opened the LV for read-write and then
> close it - you don't even need to write anything to the LV, just open
> and close - this would fire an event that would have dropped the flag.

Thanks, that sounds like a good workaround until your fix is available
where we need it.

[Peter Rajnoha]

On 01/22/2014 10:23 AM, Marius Vollmer wrote:
> Peter Rajnoha <***@redhat.com> writes:
>
>>> Thing here is that when LVs are created then at first they have this flag
>>> set until proper initialization is finished - [...]
>
> Is it guaranteed (modulo bugs) that the DM_UDEV_DISABLE_*_RULES flags
> are only ever removed from a node, and are never added to it over it's
> lifetime between add/remove events?

No, we don't have this restriction generally - when LVM command processes
a device, it can suspend or resume it or do it in a cycle while modifying
these flags based on what's actually needed to be done - whether we need
to avoid any scanning on the device in udev or not etc (but I'm not sure
at the moment we have such a sequence used anywhere, but it's possible,
there's no restriction).

>
> This isn't true right now, and UDisks fails to handle it correctly when
> a flag is added in a "change" event. I am asking to figure out where
> the fix should go.

Well, udisks should always check the DM_UDEV_DISABLE_OTHER_RULES_FLAG
and if it's set, skip its processing. It already has:

# honor the flag that device-mapper sets if the device should be ignored
ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG}=="1", GOTO="udisks_end"

..in 80-udisks.rules. So it should be already following this.

>
> For example, this is what happens when creating a pool for thinly
> provisioned volumes:
>
> UDEV [2081.714175] add /devices/virtual/block/dm-1 (block)
> [...]
> DM_UDEV_DISABLE_DISK_RULES_FLAG=1
> DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
> DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
> [...]
>
> UDEV [2081.771737] change /devices/virtual/block/dm-1 (block)
> [...]
> DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
> [...]
>
> UDEV [2081.779997] change /devices/virtual/block/dm-1 (block)
> [...]
> DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
> [...]
>
> UDEV [2081.943224] change /devices/virtual/block/dm-1 (block)
> [...]
> DM_UDEV_DISABLE_DISK_RULES_FLAG=1
> DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
> DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
> DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
> [...]
>
> I.e., the flags are set, then are removed, then set again. UDisks
> ignores the change event with the flags set, and gets into an
> inconsistent state.
>

Hmm, could you please send the whole log. This is what I see on my machine:


Creating temporary and internal LV to clean up pool metadata spare LV
- one LV marked as internal with creating the /dev/vg/lvol0 symlink (dm-2)
(DM_UDEV_DISABLE_DISK_RULES and DM_UDEV_DISABLE_OTHER_RULES set)
======================================================================

UDEV [170.702190] add /devices/virtual/block/dm-2 (block)
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1

UDEV [170.713968] change /devices/virtual/block/dm-2 (block)
DM_NAME=vg-lvol0
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.730282] remove /devices/virtual/block/dm-2 (block)
DM_NAME=vg-lvol0
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

(dm-2 removed!)
=======================================================================

Creating the pool LV itself
- one top-level and public LV to represent the pool as a whole (dm-2 here)
(no DM_UDEV_DISABLE_* set as that's the device to be used publicly)

- one internal LV for metadata device (dm-3 here)
(DM_UDEV_DISABLE_{DISK, OTHER, SUBSYSTEM}_RULES_FLAG set

- one internal LV for data device (dm-4 here)
(DM_UDEV_DISABLE_{DISK, OTHER, SUBSYSTEM}_RULES_FLAG set

- one internal LV for pool device (dm-5 here)
(DM_UDEV_DISABLE_{DISK, OTHER, SUBSYSTEM}_RULES_FLAG set
============================
UDEV [170.743992] add /devices/virtual/block/dm-2 (block)
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1

UDEV [170.771393] change /devices/virtual/block/dm-2 (block)
DM_NAME=vg-pool
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1

UDEV [170.789168] add /devices/virtual/block/dm-3 (block)
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1

UDEV [170.793567] add /devices/virtual/block/dm-4 (block)
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1

UDEV [170.799868] add /devices/virtual/block/dm-5 (block)
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1

UDEV [170.809403] change /devices/virtual/block/dm-2 (block)
DM_NAME=vg-pool
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.812167] change /devices/virtual/block/dm-3 (block)
DM_NAME=vg-pool_tmeta
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.814378] change /devices/virtual/block/dm-4 (block)
DM_NAME=vg-pool_tdata
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.840436] change /devices/virtual/block/dm-5 (block)
DM_NAME=vg-pool-tpool
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1

UDEV [170.870652] change /devices/virtual/block/dm-3 (block)
DM_NAME=vg-pool_tmeta
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.871265] change /devices/virtual/block/dm-4 (block)
DM_NAME=vg-pool_tdata
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.871863] change /devices/virtual/block/dm-2 (block)
DM_NAME=vg-pool
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1

UDEV [170.872784] change /devices/virtual/block/dm-5 (block)
DM_NAME=vg-pool-tpool
DM_UDEV_DISABLE_DISK_RULES_FLAG=1
DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
DM_UDEV_PRIMARY_SOURCE_FLAG=1


Seems OK to me. But I need your full udevadm monitor log to compare...
--
Peter

[Marius Vollmer]

Peter Rajnoha <***@redhat.com> writes:

> On 01/22/2014 10:23 AM, Marius Vollmer wrote:
>
>> Is it guaranteed (modulo bugs) that the DM_UDEV_DISABLE_*_RULES flags
>> are only ever removed from a node, and are never added to it over it's
>> lifetime between add/remove events?
>
> No, we don't have this restriction generally

Ok.

>> This isn't true right now, and UDisks fails to handle it correctly
>> when a flag is added in a "change" event. I am asking to figure out
>> where the fix should go.
>
> Well, udisks should always check the DM_UDEV_DISABLE_OTHER_RULES_FLAG
> and if it's set, skip its processing. It already has:
>
> # honor the flag that device-mapper sets if the device should be ignored
> ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG}=="1", GOTO="udisks_end"
>
> ..in 80-udisks.rules. So it should be already following this.

That's from UDisks 1, I am concerned with UDisks2, which is a quite
different beast, I think. Sorry for not making this clear.

The problem with UDisks2, as I see it, is that it ignores a "change" or
"add" event that has DM_UDEV_DISABLE_OTHER_RULES_FLAG set, while I think
it should treat it as a "remove" event.

I have proposed this patch:

https://bugs.freedesktop.org/attachment.cgi?id=92577&action=edit

> Hmm, could you please send the whole log.

Sure, attached.

[Peter Rajnoha]

On 01/23/2014 01:35 PM, Marius Vollmer wrote:
> Peter Rajnoha <***@redhat.com> writes:
>
>> On 01/22/2014 10:23 AM, Marius Vollmer wrote:
>>
>>> Is it guaranteed (modulo bugs) that the DM_UDEV_DISABLE_*_RULES flags
>>> are only ever removed from a node, and are never added to it over it's
>>> lifetime between add/remove events?
>>
>> No, we don't have this restriction generally
>
> Ok.
>
>>> This isn't true right now, and UDisks fails to handle it correctly
>>> when a flag is added in a "change" event. I am asking to figure out
>>> where the fix should go.
>>
>> Well, udisks should always check the DM_UDEV_DISABLE_OTHER_RULES_FLAG
>> and if it's set, skip its processing. It already has:
>>
>> # honor the flag that device-mapper sets if the device should be ignored
>> ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG}=="1", GOTO="udisks_end"
>>
>> ..in 80-udisks.rules. So it should be already following this.
>
> That's from UDisks 1, I am concerned with UDisks2, which is a quite
> different beast, I think. Sorry for not making this clear.
>
> The problem with UDisks2, as I see it, is that it ignores a "change" or
> "add" event that has DM_UDEV_DISABLE_OTHER_RULES_FLAG set, while I think
> it should treat it as a "remove" event.
>
> I have proposed this patch:
>
> https://bugs.freedesktop.org/attachment.cgi?id=92577&action=edit

Well, I don't quite agree with this statement from the patch:
"We treat the uevent as "remove" if the device-mapper layer
requests that other rules ignore this uevent".

The flag (DM_UDEV_DISABLE_OTHER_RULES_FLAG) is here to direct
udev processing to skip any scans - it's not actually saying
everyone else should remove this device now. It's just saying
"don't access/touch it" when this flag is set. If there was a
situation where we really need to remove (deactivate) the device,
we'd do that in lvm2 directly within processing of the device.

Also:
"It's somewhat nasty to do this but it avoids all kinds of
race-conditions caused by the design of device-mapper
(such as temporary-cryptsetup nodes and cleartext devices
without ID_FS properties properly set)."

The only reason we have these flags is that there's no way in
udev to declare the device as being private. The ID_FS_*
properties are the result of the blkid scan. And that's
exactly what we need to avoid! (...one of the reasons is
that such a private device could contain garbage since it's
not yet initialized fully). So it's actually the other way
round...

>
>> Hmm, could you please send the whole log.
>
> Sure, attached.
>

Thanks! Well, sorry for that, I've finally noticed the thing,
that was another bug, unfortunately. Should be solved now with
this git head in lvm2 upstream:
89d77326170d020ebba6ae1c717c08ac4b07996a
(git.fedorahosted.org/git/lvm2.git)

Thing is that the pool volume *should always* be marked
as private which also means DM_UDEV_DISABLE_OTHER_RULES_FLAG
is set.
--
Peter

[Peter Rajnoha]

On 01/24/2014 02:24 PM, Peter Rajnoha wrote:
> On 01/23/2014 01:35 PM, Marius Vollmer wrote:
>> Peter Rajnoha <***@redhat.com> writes:
>>
>>> On 01/22/2014 10:23 AM, Marius Vollmer wrote:
>>>
>>>> Is it guaranteed (modulo bugs) that the DM_UDEV_DISABLE_*_RULES flags
>>>> are only ever removed from a node, and are never added to it over it's
>>>> lifetime between add/remove events?
>>>
>>> No, we don't have this restriction generally
>>
>> Ok.
>>
>>>> This isn't true right now, and UDisks fails to handle it correctly
>>>> when a flag is added in a "change" event. I am asking to figure out
>>>> where the fix should go.
>>>
>>> Well, udisks should always check the DM_UDEV_DISABLE_OTHER_RULES_FLAG
>>> and if it's set, skip its processing. It already has:
>>>
>>> # honor the flag that device-mapper sets if the device should be ignored
>>> ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG}=="1", GOTO="udisks_end"
>>>
>>> ..in 80-udisks.rules. So it should be already following this.
>>
>> That's from UDisks 1, I am concerned with UDisks2, which is a quite
>> different beast, I think. Sorry for not making this clear.
>>
>> The problem with UDisks2, as I see it, is that it ignores a "change" or
>> "add" event that has DM_UDEV_DISABLE_OTHER_RULES_FLAG set, while I think
>> it should treat it as a "remove" event.
>>
>> I have proposed this patch:
>>
>> https://bugs.freedesktop.org/attachment.cgi?id=92577&action=edit
>
> Well, I don't quite agree with this statement from the patch:
> "We treat the uevent as "remove" if the device-mapper layer
> requests that other rules ignore this uevent".
>
> The flag (DM_UDEV_DISABLE_OTHER_RULES_FLAG) is here to direct
> udev processing to skip any scans - it's not actually saying
> everyone else should remove this device now. It's just saying
> "don't access/touch it" when this flag is set. If there was a
> situation where we really need to remove (deactivate) the device,
> we'd do that in lvm2 directly within processing of the device.

...simply, the event listener that gets the event with this flag
set should just consider this dm device as "private".

--
Peter

[Marius Vollmer]

Peter Rajnoha <***@redhat.com> writes:

>> The flag (DM_UDEV_DISABLE_OTHER_RULES_FLAG) is here to direct
>> udev processing to skip any scans - it's not actually saying
>> everyone else should remove this device now. It's just saying
>> "don't access/touch it" when this flag is set. If there was a
>> situation where we really need to remove (deactivate) the device,
>> we'd do that in lvm2 directly within processing of the device.
>
> ...simply, the event listener that gets the event with this flag
> set should just consider this dm device as "private".

Yeah, that's what treating events with the flag as a "remove" event is
meant to accomplish.

UDisks2 maintains objects on D-Bus corresponding to public udev block
devices. When a device changes from public to private, we should remove
the corresponding object from D-Bus. The code for that is the same as
when UDisks2 receives a "remove" event for the device AFAICS, so we just
jump into that code path by changing the event action early on.

The alternative is to also create D-Bus objects for private udev block
devices, but set UDISKS_IGNORE for them (and rely on its clients to dtrt
with that). I think that is what UDisks 1 used to do, but David has
choosen not to do this for UDisks2. I can't really judge which approach
is better. Do you have an opinion?

[Peter Rajnoha]

On 01/24/2014 03:39 PM, Marius Vollmer wrote:
> Peter Rajnoha <***@redhat.com> writes:
>
>>> The flag (DM_UDEV_DISABLE_OTHER_RULES_FLAG) is here to direct
>>> udev processing to skip any scans - it's not actually saying
>>> everyone else should remove this device now. It's just saying
>>> "don't access/touch it" when this flag is set. If there was a
>>> situation where we really need to remove (deactivate) the device,
>>> we'd do that in lvm2 directly within processing of the device.
>>
>> ...simply, the event listener that gets the event with this flag
>> set should just consider this dm device as "private".
>
> Yeah, that's what treating events with the flag as a "remove" event is
> meant to accomplish.
>
> UDisks2 maintains objects on D-Bus corresponding to public udev block
> devices. When a device changes from public to private, we should remove
> the corresponding object from D-Bus. The code for that is the same as
> when UDisks2 receives a "remove" event for the device AFAICS, so we just
> jump into that code path by changing the event action early on.

Well, in that case I think it's OK to do that.

>
> The alternative is to also create D-Bus objects for private udev block
> devices, but set UDISKS_IGNORE for them (and rely on its clients to dtrt
> with that). I think that is what UDisks 1 used to do, but David has
> choosen not to do this for UDisks2. I can't really judge which approach
> is better. Do you have an opinion?
>

To be honest, I don't like the idea with setting these variables, but
there's nothing else at the moment we could use in udev...

Thing with setting variables is that we need to rely on others to properly
check this variable - and each variable coming from different subsystems has
a different name. Would be really great if we have a more decent way
how to mark devices as private where with the "private" I mean that it can
be processed only by the subsystem that claims it. This would require a
solution hardwired directly to the way the devices are presented in the system,
maybe a common sysfs variable that would be used for all... Since there
may be numerous devices for which the first ADD/CHANGE event doesn't actually
mean the device is ready for use - some have extra configuration that follows
for it to be usable (the same applies for MD, loop devices) or they just
need to be marked as private for whatever reason.

For example, even systemd tries to gather this information from various
sources and sets SYSTEMD_READY=0/1 (see also https://github.com/systemd/systemd/blob/master/rules/99-systemd.rules.in)
(unfortunately, systemd is not the only one in the world, there are alternatives,
that's why having this info directly in sysfs would make more sense as it's
global).

This change is possible, but requires everybody to gather up and agree on
common solution...

--
Peter

[Marius Vollmer]

Peter Rajnoha <***@redhat.com> writes:

> Thing with setting variables is that we need to rely on others to properly
> check this variable - and each variable coming from different subsystems has
> a different name. Would be really great if we have a more decent way
> how to mark devices as private where with the "private" I mean that it can
> be processed only by the subsystem that claims it.

I don't know whether it is at all feasible, but what about using
something like SUBSYSTEM="proto-block" for devices that are not general
purpose block devices (yet). For example, a device mapper node would
start out as a proto-block, and could be changed to a block via dmsetup.

That would be a fundamental change, though, and might cause more
breakage than it helps to clean things up.

> For example, even systemd tries to gather this information from
> various sources and sets SYSTEMD_READY=0/1 (see also
> https://github.com/systemd/systemd/blob/master/rules/99-systemd.rules.in)
> (unfortunately, systemd is not the only one in the world, there are
> alternatives, that's why having this info directly in sysfs would make
> more sense as it's global).

Yeah, but on the other hand, everone might have their own slight
variation on what to ignore exactly. For example, systemd seems to
ignore encrypted but unformatted block devices for some reason, while
the guy who is actually going to format them (maybe UDisks2) probably
doesn't want to ignore them.

But, yes, it would be good if everyone could start from a sane setup and
then add exceptions to it.

[Marius Vollmer]

Peter Rajnoha <***@redhat.com> writes:

> Thanks! Well, sorry for that, I've finally noticed the thing,
> that was another bug, unfortunately. Should be solved now with
> this git head in lvm2 upstream:
> 89d77326170d020ebba6ae1c717c08ac4b07996a
> (git.fedorahosted.org/git/lvm2.git)
>
> Thing is that the pool volume *should always* be marked
> as private which also means DM_UDEV_DISABLE_OTHER_RULES_FLAG
> is set.

Nice, thanks!

With this fixed, I have to ask again: Is _every_ situation where a block
device goes from public to private with a "change" event a bug?

I would say "yes", simply because I can't think of a situation where
LVM2 doesn't know from the start whether the device is creates is gonna
be public or private.

If so, we just keep UDisks2 as it is, I'd say, and I file bugs when I
find another public->private transition.

[Peter Rajnoha]

On 01/24/2014 03:50 PM, Marius Vollmer wrote:
> Peter Rajnoha <***@redhat.com> writes:
>
>> Thanks! Well, sorry for that, I've finally noticed the thing,
>> that was another bug, unfortunately. Should be solved now with
>> this git head in lvm2 upstream:
>> 89d77326170d020ebba6ae1c717c08ac4b07996a
>> (git.fedorahosted.org/git/lvm2.git)
>>
>> Thing is that the pool volume *should always* be marked
>> as private which also means DM_UDEV_DISABLE_OTHER_RULES_FLAG
>> is set.
>
> Nice, thanks!
>
> With this fixed, I have to ask again: Is _every_ situation where a block
> device goes from public to private with a "change" event a bug?
>

Hmm, and if we can detect a device going from public to private, would
it be a problem to detect the other way round - a device going from
private to public? IMHO it would be exactly the same situation - just
in case it gets public, the device would be presented to any event
listeners again with all the context it currently has.

--
Peter

[Zdenek Kabelac]

Dne 24.1.2014 15:50, Marius Vollmer napsal(a):
> Peter Rajnoha <***@redhat.com> writes:
>
>> Thanks! Well, sorry for that, I've finally noticed the thing,
>> that was another bug, unfortunately. Should be solved now with
>> this git head in lvm2 upstream:
>> 89d77326170d020ebba6ae1c717c08ac4b07996a
>> (git.fedorahosted.org/git/lvm2.git)
>>
>> Thing is that the pool volume *should always* be marked
>> as private which also means DM_UDEV_DISABLE_OTHER_RULES_FLAG
>> is set.
>
> Nice, thanks!
>
> With this fixed, I have to ask again: Is _every_ situation where a block
> device goes from public to private with a "change" event a bug?
>
> I would say "yes", simply because I can't think of a situation where
> LVM2 doesn't know from the start whether the device is creates is gonna
> be public or private.
>
> If so, we just keep UDisks2 as it is, I'd say, and I file bugs when I
> find another public->private transition.


This is probably worth to put here this comment:

When lvm2 creates a complex device - it starts to build it from pieces.
However each individual piece is initially created as a visible plain LV.

The reason behind this is - whenever lvm2 fails to finish the command
(or someone just press Power-Off button) - it should leave metadata in the
state, you can recover from with normal lvm2 command.

This means - even if the lvm2 fails to build complex targets - it should never
leave metadata filled with 'private' LVs, which user cannot delete with
lvremove command. Of course this is currently just a 'target' we try to reach
and you are encourage to report bugs if you notice ordering problems here
(i.e. raids are not compatible with this style).

So while TEMPORARY flag fixes the 'OK-path' here - it actually may introduce a
problem of having a device with unknown content for udev in the case of lvm2
command problem - but the assumption here is - the system has far more serious
troubles in such failures than you should care about udev-db correctness...

And there is more scary part behind this - the current NOSCAN and TEMPORARY
flags do work only non-clustered case.

Regards

Zdenek

[Peter Rajnoha]

On 01/24/2014 04:17 PM, Zdenek Kabelac wrote:
> Dne 24.1.2014 15:50, Marius Vollmer napsal(a):
>> Peter Rajnoha <***@redhat.com> writes:
>>
>>> Thanks! Well, sorry for that, I've finally noticed the thing,
>>> that was another bug, unfortunately. Should be solved now with
>>> this git head in lvm2 upstream:
>>> 89d77326170d020ebba6ae1c717c08ac4b07996a
>>> (git.fedorahosted.org/git/lvm2.git)
>>>
>>> Thing is that the pool volume *should always* be marked
>>> as private which also means DM_UDEV_DISABLE_OTHER_RULES_FLAG
>>> is set.
>>
>> Nice, thanks!
>>
>> With this fixed, I have to ask again: Is _every_ situation where a block
>> device goes from public to private with a "change" event a bug?
>>
>> I would say "yes", simply because I can't think of a situation where
>> LVM2 doesn't know from the start whether the device is creates is gonna
>> be public or private.
>>
>> If so, we just keep UDisks2 as it is, I'd say, and I file bugs when I
>> find another public->private transition.
>
>
> This is probably worth to put here this comment:
>
> When lvm2 creates a complex device - it starts to build it from pieces.
> However each individual piece is initially created as a visible plain LV.
>
> The reason behind this is - whenever lvm2 fails to finish the command
> (or someone just press Power-Off button) - it should leave metadata in
> the state, you can recover from with normal lvm2 command.
>
> This means - even if the lvm2 fails to build complex targets - it should
> never leave metadata filled with 'private' LVs, which user cannot
> delete with lvremove command. Of course this is currently just a
> 'target' we try to reach and you are encourage to report bugs if you
> notice ordering problems here (i.e. raids are not compatible with this
> style).
>
> So while TEMPORARY flag fixes the 'OK-path' here - it actually may
> introduce a problem of having a device with unknown content for udev in
> the case of lvm2 command problem - but the assumption here is - the
> system has far more serious troubles in such failures than you should
> care about udev-db correctness...
>
> And there is more scary part behind this - the current NOSCAN and
> TEMPORARY flags do work only non-clustered case.

...yes, using udev in cluster is a bit of scary thing to do, I have to
admit :)

--
Peter

[Marius Vollmer]

Peter Rajnoha <***@redhat.com> writes:

> However, you're right that in case of thin LVs, this is set incorrectly.
> The DM_UDEV_DISABLE_OTHER_RULES flag should not be there. I've fixed that:
>
> https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=91b26b63b408b7d7b4f79f0754d190783874e4cc

Great! And just in time for the release, too! :-)