[linux-lvm] thin pool and discards

[Vasiliy Tolstov]

Hello. I'm try to check that discard are working. I have thin volume
1031 (used by virtual machine)
I'm fill the disc and delete data, (fs ext4 with discard option) after
that i manually run fstrim -v /
but data on thin volume not reclaimed.

LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
tp0 vg0 twi-a-tz-- 445.00g 0.00
373_153 vg1 Vwi---tz-k 20.00g tp1
375_151 vg1 Vwi---tz-k 40.00g tp1
529_217 vg1 Vwi---tz-k 40.00g tp1
531_221 vg1 Vwi---tz-k 20.00g tp1
533_219 vg1 Vwi---tz-k 40.00g tp1
577_305 vg1 Vwi---tz-k 40.00g tp1
581_309 vg1 Vwi---tz-k 40.00g tp1
583_307 vg1 Vwi---tz-k 20.00g tp1
tp1 vg1 twi-a-tz-- 445.00g 0.50
377_159 vg2 Vwi---tz-k 20.00g tp2
445_197 vg2 Vwi---tz-k 20.00g tp2
tp2 vg2 twi-a-tz-- 445.00g 0.25
1031 vg3 Vwi-aotz-- 20.00g tp3 56.23
tp3 vg3 twi-a-tz-- 445.00g 2.53
tp4 vg4 twi-a-tz-- 445.00g 0.00
111 vg5 Vwi-aotz-- 20.00g tp5 5.63
111_23 vg5 Vwi---tz-k 20.00g tp5 111
111_71 vg5 Vwi---tz-k 20.00g tp5 111
13 vg5 Vwi-aotz-- 30.00g tp5 4.70
17 vg5 Vwi-aotz-- 20.00g tp5 25.58
29 vg5 Vwi-a-tz-- 20.00g tp5 0.00
31 vg5 Vwi-a-tz-- 20.00g tp5 0.00
33 vg5 Vwi-a-tz-- 20.00g tp5 0.00
35 vg5 Vwi-a-tz-- 20.00g tp5 0.00
75 vg5 Vwi-aotz-- 20.00g tp5 25.58
tp5 vg5 twi-a-tz-- 445.00g 2.92

lvm - 2.02.106
kernel 3.10

[Zdenek Kabelac]

Post by Vasiliy Tolstov
Hello. I'm try to check that discard are working. I have thin volume
1031 (used by virtual machine)
I'm fill the disc and delete data, (fs ext4 with discard option) after
that i manually run fstrim -v /
but data on thin volume not reclaimed.
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
tp0 vg0 twi-a-tz-- 445.00g 0.00
373_153 vg1 Vwi---tz-k 20.00g tp1
375_151 vg1 Vwi---tz-k 40.00g tp1
529_217 vg1 Vwi---tz-k 40.00g tp1
531_221 vg1 Vwi---tz-k 20.00g tp1
533_219 vg1 Vwi---tz-k 40.00g tp1
577_305 vg1 Vwi---tz-k 40.00g tp1
581_309 vg1 Vwi---tz-k 40.00g tp1
583_307 vg1 Vwi---tz-k 20.00g tp1
tp1 vg1 twi-a-tz-- 445.00g 0.50
377_159 vg2 Vwi---tz-k 20.00g tp2
445_197 vg2 Vwi---tz-k 20.00g tp2
tp2 vg2 twi-a-tz-- 445.00g 0.25
1031 vg3 Vwi-aotz-- 20.00g tp3 56.23
tp3 vg3 twi-a-tz-- 445.00g 2.53
tp4 vg4 twi-a-tz-- 445.00g 0.00
111 vg5 Vwi-aotz-- 20.00g tp5 5.63
111_23 vg5 Vwi---tz-k 20.00g tp5 111
111_71 vg5 Vwi---tz-k 20.00g tp5 111
13 vg5 Vwi-aotz-- 30.00g tp5 4.70
17 vg5 Vwi-aotz-- 20.00g tp5 25.58
29 vg5 Vwi-a-tz-- 20.00g tp5 0.00
31 vg5 Vwi-a-tz-- 20.00g tp5 0.00
33 vg5 Vwi-a-tz-- 20.00g tp5 0.00
35 vg5 Vwi-a-tz-- 20.00g tp5 0.00
75 vg5 Vwi-aotz-- 20.00g tp5 25.58
tp5 vg5 twi-a-tz-- 445.00g 2.92
lvm - 2.02.106
kernel 3.10

Kernel 3.10 is too old,
Please repeat the test with at least 3.16


Zdenek

[Vasiliy Tolstov]

Post by Zdenek Kabelac
Kernel 3.10 is too old,
Please repeat the test with at least 3.16

Why? As i understand 3.10 lts... Can you say me what patches i need to
cherry pick? Or maybe redhat kernel have this feature...?

[Mike Snitzer]

On Tue, Sep 30 2014 at 10:57am -0400,

Post by Vasiliy Tolstov

Post by Zdenek Kabelac
Kernel 3.10 is too old,
Please repeat the test with at least 3.16

Why? As i understand 3.10 lts... Can you say me what patches i need to
cherry pick? Or maybe redhat kernel have this feature...?

You likely need this upstream commit:
http://git.kernel.org/linus/19fa1a6756ed9e92daa9537c03b47d6b55cc2316

Though that commit was tagged for stable. So not sure.

In general, upstream dm-thinp saw quite a few improvements that in
combination results in a much more stable dm-thinp (particularly error
handling from faults and out of space conditions).

RHEL7 does include these changes.

[Vasiliy Tolstov]

Post by Mike Snitzer
http://git.kernel.org/linus/19fa1a6756ed9e92daa9537c03b47d6b55cc2316
Though that commit was tagged for stable. So not sure.
In general, upstream dm-thinp saw quite a few improvements that in
combination results in a much more stable dm-thinp (particularly error
handling from faults and out of space conditions).
RHEL7 does include these changes.

Thanks! Also i'm try to check rhel 7 kernel sources...

[Vasiliy Tolstov]

Post by Mike Snitzer
http://git.kernel.org/linus/19fa1a6756ed9e92daa9537c03b47d6b55cc2316
Though that commit was tagged for stable. So not sure.
In general, upstream dm-thinp saw quite a few improvements that in
combination results in a much more stable dm-thinp (particularly error
handling from faults and out of space conditions).
RHEL7 does include these changes.

Thanks! Rhel kernel works fine for me. Does it possible to send needed
patches to stable for all other users ?=) As i see this commit does
not in 3.10 stable now.

[Vasiliy Tolstov]

Post by Vasiliy Tolstov
Thanks! Rhel kernel works fine for me. Does it possible to send needed
patches to stable for all other users ?=) As i see this commit does
not in 3.10 stable now.

AS i see i have another issue. I don't know how created some thinp
volumes, but on some of them i'm able to get back space after discard,
but on some not. How can i debug this ?

on vg0 not works:
vgdisplay vg0/tp0
--- Volume group ---
VG Name vg0
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 359
VG Access read/write
VG Status resizable
MAX LV 10000
Cur LV 6
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 449.70 GiB
PE Size 4.00 MiB
Total PE 115124
Alloc PE / Size 114230 / 446.21 GiB
Free PE / Size 894 / 3.49 GiB
VG UUID 7iCwdw-NTqg-W8dJ-4hTE-8D8J-r9Mv-8y4NVM

lvdisplay vg0/tp0
--- Logical volume ---
LV Name tp0
VG Name vg0
LV UUID R4ioiK-PWnr-SLUp-8g0l-CTU0-nRn1-bKebI3
LV Write Access read/write
LV Creation host, time cn07, 2014-08-13 19:54:22 +0400
LV Pool transaction ID 173
LV Pool metadata tp0_tmeta
LV Pool data tp0_tdata
LV Pool chunk size 64.00 KiB
LV Zero new blocks yes
LV Status available
# open 0
LV Size 445.00 GiB
Allocated pool data 2.97%
Allocated metadata 1.34%
Current LE 113920
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:39




on vg1 all works fine:

vgdisplay vg1
--- Volume group ---
VG Name vg1
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 227
VG Access read/write
VG Status resizable
MAX LV 10000
Cur LV 9
Open LV 3
Max PV 0
Cur PV 1
Act PV 1
VG Size 449.70 GiB
PE Size 4.00 MiB
Total PE 115124
Alloc PE / Size 114230 / 446.21 GiB
Free PE / Size 894 / 3.49 GiB
VG UUID zbDIA4-731p-3uCm-oHx1-txYu-YMd7-YLMCVj

lvdisplay vg1/tp1
--- Logical volume ---
LV Name tp1
VG Name vg1
LV UUID aVJpg3-xDYo-0Tuz-fImd-kA3I-Qd3g-e3xRSR
LV Write Access read/write
LV Creation host, time cn07, 2014-08-13 19:54:22 +0400
LV Pool transaction ID 110
LV Pool metadata tp1_tmeta
LV Pool data tp1_tdata
LV Pool chunk size 64.00 KiB
LV Zero new blocks yes
LV Status available
# open 0
LV Size 445.00 GiB
Allocated pool data 6.65%
Allocated metadata 2.68%
Current LE 113920
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:32

[Zdenek Kabelac]

Post by Vasiliy Tolstov

Post by Vasiliy Tolstov
Thanks! Rhel kernel works fine for me. Does it possible to send needed
patches to stable for all other users ?=) As i see this commit does
not in 3.10 stable now.

AS i see i have another issue. I don't know how created some thinp
volumes, but on some of them i'm able to get back space after discard,
but on some not. How can i debug this ?

Please mention which kernel do you use - there have been patches
related to discard on thin volumes (pretty resent - kernel 3.14 I think).

If you use old stable 3.10 from vanilla kernel it simply cannot be compared
with 3.10 RHEL kernel - they are very different (RHEL has ton of stuff
backported even from the latest kernels)

So - please try first with upstream to see if you problem is not already fixed.


Zdenek

[Vasiliy Tolstov]

Post by Zdenek Kabelac
Please mention which kernel do you use - there have been patches
related to discard on thin volumes (pretty resent - kernel 3.14 I think).
If you use old stable 3.10 from vanilla kernel it simply cannot be compared
with 3.10 RHEL kernel - they are very different (RHEL has ton of stuff
backported even from the latest kernels)
So - please try first with upstream to see if you problem is not already fixed.

I'm recreate all devices and now all works fine in 3.10.55 and in
rhel. I think my hands bad =(.