[linux-lvm] determining snapshot overhead

Discussion:

Bram Klein Gunnewiek

2015-01-14 07:24:30 UTC

Hello,

I am trying to create snapshots of thick volumes that can't break
because they overflow (thin volumes are not an option right now). I
can't find information on how to calculate (in advance) the metadata
overhead of a snapshot when all blocks in the origin volume are changed
(e.g. the snapshot is 100% full). Is there a way to do this?

-Bram

Zdenek Kabelac

2015-01-14 08:34:20 UTC

Permalink

Post by Bram Klein Gunnewiek
Hello,
I am trying to create snapshots of thick volumes that can't break because they
overflow (thin volumes are not an option right now). I can't find information
on how to calculate (in advance) the metadata overhead of a snapshot when all
blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is
there a way to do this?

Recent versions of lvm2 (let's say >= 111) are able to determine this size
automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'

Regards

Zdenek

Bram Klein Gunnewiek

2015-01-14 10:18:10 UTC

Permalink

Thanks for your reply. I tested it out and it does not seem to work
(lvm2 version used is 2.02.98-6ubuntu2):

lvcreate -n testvolume3 -L 1G vg
lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1

After this the snapshot is invalid:

testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00

I checked out the lvm2 source code and found the methods cow_max_extents
and _cow_max_size in snapshot_manip.c. Do these methods calculate the
exact max size of the snapshot meta data?

Post by Zdenek Kabelac

Recent versions of lvm2 (let's say >= 111) are able to determine this
size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Zdenek Kabelac

2015-01-14 11:42:09 UTC

Permalink

Thanks for your reply. I tested it out and it does not seem to work (lvm2

yep - too old for this feature being supported.

lvcreate -n testvolume3 -L 1G vg
lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
I checked out the lvm2 source code and found the methods cow_max_extents and
_cow_max_size in snapshot_manip.c. Do these methods calculate the exact max
size of the snapshot meta data?

yes.

It's probably worth to note there has been some related kernel fixes for old
snaps - otherwise there has been theoretical path to create much bigger
snapshots then the origin size was - though it's nearly impossible to reach
such state.

Zdenek

Post by Zdenek Kabelac

Recent versions of lvm2 (let's say >= 111) are able to determine this size
automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Bram Klein Gunnewiek

2015-01-14 12:38:42 UTC

Permalink

Post by Zdenek Kabelac

Thanks for your reply. I tested it out and it does not seem to work (lvm2

yep - too old for this feature being supported.

Ok, then we have to do it ourself since we don't want to go with
backports / custom versions on our systems

Post by Zdenek Kabelac

lvcreate -n testvolume3 -L 1G vg
lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
I checked out the lvm2 source code and found the methods
cow_max_extents and
_cow_max_size in snapshot_manip.c. Do these methods calculate the exact max
size of the snapshot meta data?

yes.
It's probably worth to note there has been some related kernel fixes
for old snaps - otherwise there has been theoretical path to create
much bigger snapshots then the origin size was - though it's nearly
impossible to reach such state.

Thanks again. I did some tests and calculations and it seems that the
overhead is ~0.39% of the origin size when chunk sizes of 4KiB are used.
Bigger chunk size means less overhead. If we use 0.5% of the origin
space as meta data overhead we should be on the safe side I think?

Post by Zdenek Kabelac
Zdenek

Post by Zdenek Kabelac

Post by Bram Klein Gunnewiek
Hello,
I am trying to create snapshots of thick volumes that can't break because they
overflow (thin volumes are not an option right now). I can't find information
on how to calculate (in advance) the metadata overhead of a
snapshot when all
blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is
there a way to do this?

Recent versions of lvm2 (let's say >= 111) are able to determine this size
automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Zdenek Kabelac

2015-01-14 14:26:06 UTC

Permalink

Post by Zdenek Kabelac

Thanks for your reply. I tested it out and it does not seem to work (lvm2

yep - too old for this feature being supported.

Ok, then we have to do it ourself since we don't want to go with backports /
custom versions on our systems

Post by Zdenek Kabelac

yes.
It's probably worth to note there has been some related kernel fixes for old
snaps - otherwise there has been theoretical path to create much bigger
snapshots then the origin size was - though it's nearly impossible to reach
such state.

Thanks again. I did some tests and calculations and it seems that the overhead
is ~0.39% of the origin size when chunk sizes of 4KiB are used. Bigger chunk
size means less overhead. If we use 0.5% of the origin space as meta data
overhead we should be on the safe side I think?

It cannot be defined as some 'percentage' value - as there are even
combinations of LV sizes and extent size where the percentage might be over
50% (thought those are not really usable in real life).

lvm2 code is the only valid calculation how to estimate full max size of
snapshot for various combinations of those parameters.

Zdenek

Post by Zdenek Kabelac
Zdenek

Post by Zdenek Kabelac

Recent versions of lvm2 (let's say >= 111) are able to determine this size
automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/