Discussion:
[linux-lvm] lvm partition on ramdisk
Larry Dickson
2008-05-12 15:23:50 UTC
Permalink
Hi all,

I'm trying to set up a logical volume over a 256MB ramdisk and a 1TB raid
array partition. My ramdisk (set up in grub.conf with ramdisk_size=262144
ramdisk_blocksize=4096) fails to make a type 8e partition either in fdisk or
sfdisk. The error in fdisk is
WARNING: Re-reading the partition table failed with error 22: Invalid
argument.
The kernel still uses the old table.
The new table will be used at the next reboot.

And, of course, after the next reboot, my ramdisk data will be gone.

Can anyone give me some guidance here? Is it choking on the 4K blocksize? I
wanted that for efficiency, as suggested by MrMichaelWill on his blog,
but his base case seems to be a file system mounted directly on the
unpartitioned device, and I need LVM.

TIA,

Larry Dickson
Cutting Edge Networked Storage
Larry Dickson
2008-05-12 20:05:37 UTC
Permalink
Continuing this frustration: I read in the pvcreate doc that you could tell
it to use a whole disk (instead of a partition). So I tried "pvcreate
/dev/ram0". With or without zeroing the disk, and with or without "--force",
and with any blocksize (1024, 2048, or 4096), the result is always the same:

Device /dev/ram0 not found (or ignored by filtering).

I know /dev/ram0 is there; fdisk sees it. Is there any way to turn off the
filtering?

Larry Dickson
Post by Larry Dickson
Hi all,
I'm trying to set up a logical volume over a 256MB ramdisk and a 1TB raid
array partition. My ramdisk (set up in grub.conf with ramdisk_size=262144
ramdisk_blocksize=4096) fails to make a type 8e partition either in fdisk or
sfdisk. The error in fdisk is
WARNING: Re-reading the partition table failed with error 22: Invalid
argument.
The kernel still uses the old table.
The new table will be used at the next reboot.
And, of course, after the next reboot, my ramdisk data will be gone.
Can anyone give me some guidance here? Is it choking on the 4K blocksize?
I wanted that for efficiency, as suggested by MrMichaelWill on his blog,
but his base case seems to be a file system mounted directly on the
unpartitioned device, and I need LVM.
TIA,
Larry Dickson
Cutting Edge Networked Storage
Stuart D. Gathman
2008-05-12 20:10:56 UTC
Permalink
Post by Larry Dickson
I'm trying to set up a logical volume over a 256MB ramdisk and a 1TB raid
array partition. My ramdisk (set up in grub.conf with ramdisk_size=262144
ramdisk_blocksize=4096) fails to make a type 8e partition either in fdisk or
sfdisk. The error in fdisk is
You can't and don't need to set a partition type for a ramdisk. It
already is a partition.
Post by Larry Dickson
Can anyone give me some guidance here? Is it choking on the 4K blocksize? I
wanted that for efficiency, as suggested by MrMichaelWill on his blog,
but his base case seems to be a file system mounted directly on the
unpartitioned device, and I need LVM.
No, just as you can make a disk device like /dev/sda a PV without any
partition table, you don't need a partition table for a ramdisk.

HOWEVER: you REALLY REALLY don't want to mix volatile and non-volatile
PVs in the same volume group. When you reboot, and the ramdisk PV is
missing, the volume group won't activate automatically. Furthermore,
you could accidentally end up with real data on the ramdisk - and it goes
bye bye on reboot.

Let me guess - you wanted to use the ramdisk for a snapshot? Yes, some
other people have tried that (see the thread here a week or so ago). You will
be very unhappy when the system reboots and the ramdisk is missing.

The real way to put snapshots on a ramdisk is a "temporary snapshot" -
a feature which could in theory be done via device-mapper, but needs
utility support.

If your root fs is not on the VG you are adding the ramdisk to, you
could probably get away with it. Add the ramdisk, snapshot, make your
backup, delete snapshot, reduce ramdisk from VG. If the system crashes
while the ramdisk is still in the VG metadata, then you'll need to manually
recover the VG to activate - but hopefully that will be rare.

There is also an option you could add to initrd to activate a VG even
with missing PVs. But that makes all LVs read-only until the VG is repaired.
--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Larry Dickson
2008-05-12 20:26:03 UTC
Permalink
The purpose of the ramdisk is to emulate a battery-backed solid-state drive
which is intended to do high speed, seek-free metadata IO in a file system
(for instance FAT32) in which all the metadata is at the beginning. We can't
afford the SSD unless we know it is going to work.

We don't care if the test data survives a shutdown, only that the mapping be
right.

I did try making the whole ramdisk a PV, and it choked on that too, because
of "filtering". A way around all this overprotectiveness would be highly
appreciated.

TIA,

Larry
Post by Larry Dickson
Post by Larry Dickson
I'm trying to set up a logical volume over a 256MB ramdisk and a 1TB
raid
Post by Larry Dickson
array partition. My ramdisk (set up in grub.conf with
ramdisk_size=262144
Post by Larry Dickson
ramdisk_blocksize=4096) fails to make a type 8e partition either in
fdisk or
Post by Larry Dickson
sfdisk. The error in fdisk is
You can't and don't need to set a partition type for a ramdisk. It
already is a partition.
Post by Larry Dickson
Can anyone give me some guidance here? Is it choking on the 4K
blocksize? I
Post by Larry Dickson
wanted that for efficiency, as suggested by MrMichaelWill on his blog,
but his base case seems to be a file system mounted directly on the
unpartitioned device, and I need LVM.
No, just as you can make a disk device like /dev/sda a PV without any
partition table, you don't need a partition table for a ramdisk.
HOWEVER: you REALLY REALLY don't want to mix volatile and non-volatile
PVs in the same volume group. When you reboot, and the ramdisk PV is
missing, the volume group won't activate automatically. Furthermore,
you could accidentally end up with real data on the ramdisk - and it goes
bye bye on reboot.
Let me guess - you wanted to use the ramdisk for a snapshot? Yes, some
other people have tried that (see the thread here a week or so ago). You will
be very unhappy when the system reboots and the ramdisk is missing.
The real way to put snapshots on a ramdisk is a "temporary snapshot" -
a feature which could in theory be done via device-mapper, but needs
utility support.
If your root fs is not on the VG you are adding the ramdisk to, you
could probably get away with it. Add the ramdisk, snapshot, make your
backup, delete snapshot, reduce ramdisk from VG. If the system crashes
while the ramdisk is still in the VG metadata, then you'll need to manually
recover the VG to activate - but hopefully that will be rare.
There is also an option you could add to initrd to activate a VG even
with missing PVs. But that makes all LVs read-only until the VG is repaired.
--
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Charles Marcus
2008-05-12 20:28:30 UTC
Permalink
A way around all this overprotectiveness would be highly appreciated.
It is not 'over-protectiveness'... I would call it 'highly *desired*
protectiveness'.

A workaround has already been provided...
--
Best regards,

Charles
Larry Dickson
2008-05-12 20:47:30 UTC
Permalink
I got the workaround - it crossed in the mail.

However, let me follow up your (and Stuart's) point. Are you saying that an
unmounted LVM volume will mess up the boot, even if the volume in question
is not mapped to boot or /? I was proceeding under the assumption that LVM
would be happy to sew the pieces together again later, even if the data in
them is trashed.

Larry
Post by Charles Marcus
A way around all this overprotectiveness would be highly appreciated.
It is not 'over-protectiveness'... I would call it 'highly *desired*
protectiveness'.
A workaround has already been provided...
--
Best regards,
Charles
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Larry Dickson
2008-05-12 22:07:53 UTC
Permalink
OK, I got it to work using your workaround. Thanks, Stuart.

NEXT QUESTION...

How do I get LVM to put all the extents of the ramdisk BEFORE any of the
extents of the raid array? Even if I list the pvs, ramdisk first, in the
lvcreate command and/or start with the ramdisk in a vgcreate followed by
adding the raid array in a vgextend... it always ignores the ramdisk and
only assigns extents from the raid array. (I suppose it is putting the
ramdisk at the end, where it is useless for my purposes.)

TIA,

Larry
Post by Larry Dickson
I got the workaround - it crossed in the mail.
However, let me follow up your (and Stuart's) point. Are you saying that
an unmounted LVM volume will mess up the boot, even if the volume in
question is not mapped to boot or /? I was proceeding under the assumption
that LVM would be happy to sew the pieces together again later, even if the
data in them is trashed.
Larry
Post by Charles Marcus
A way around all this overprotectiveness would be highly appreciated.
It is not 'over-protectiveness'... I would call it 'highly *desired*
protectiveness'.
A workaround has already been provided...
--
Best regards,
Charles
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Stuart D. Gathman
2008-05-12 20:18:46 UTC
Permalink
Post by Larry Dickson
Device /dev/ram0 not found (or ignored by filtering).
I know /dev/ram0 is there; fdisk sees it. Is there any way to turn off the
filtering?
It is filtered by default to prevent shooting yourself in the foot.
You can change /etc/lvm/lvm.conf (on EL5) to allow a ramdisk as PV.
However, you really need to understand the problems that will result from
a crash/reboot while a ramdisk is part of the VG before you do this.
--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Stuart D. Gathman
2008-05-13 01:12:16 UTC
Permalink
Post by Larry Dickson
OK, I got it to work using your workaround. Thanks, Stuart.
NEXT QUESTION...
How do I get LVM to put all the extents of the ramdisk BEFORE any of the
extents of the raid array? Even if I list the pvs, ramdisk first, in the
lvcreate command and/or start with the ramdisk in a vgcreate followed by
adding the raid array in a vgextend... it always ignores the ramdisk and
only assigns extents from the raid array. (I suppose it is putting the
ramdisk at the end, where it is useless for my purposes.)
You can specify which PV to use in lvcreate.

Let me repeat the warning - if the system goes down while the ramdisk
is part of the volume group, LVM will be unhappy and require recovery
of the volume group. (I.e. your system could be unbootable.) You had better
not have your root fs on the VG.
--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Stuart D. Gathman
2008-05-13 01:29:22 UTC
Permalink
Post by Larry Dickson
However, let me follow up your (and Stuart's) point. Are you saying that an
unmounted LVM volume will mess up the boot, even if the volume in question
is not mapped to boot or /? I was proceeding under the assumption that LVM
would be happy to sew the pieces together again later, even if the data in
them is trashed.
As long as the VG is not needed in initrd (e.g. a test VG), you should
be ok. You will simply have to go through the procedure of removing the
"failed" PV and adding it back after a reboot. As long as your root fs
(and /usr and other stuff needed at startup) are not on the test VG, you
should be fine. The problem is that the VG will not activate automatically
with a missing PV. Even with --partial, it will activate the VG
as readonly metadata. Yes, AIX handles this better, IMO. But Linux LVM is
getting there.

For your application, you should make a separate "testvg" VG for your test
that does not have your system. At boot, activate the VG with --partial,
then use "pvcreate -u " to set the UUID on the ramdisk to match the UUID
originally on the ramdisk, followed by vgcfgrestore.
--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Karl Wagner
2008-05-13 08:00:30 UTC
Permalink
If this is what you are looking to do, there are two more approaches you
could consider. I have used both to varying degrees of success.

The first is to just keep the first part of the disk in cache. Ie. Set
up the device as you want, then run the following either as a cron job
or inside a 'while true' loop in the background:
cat /dev/yourdevice | dd bs=1048576 count=$N of=/dev/null
(replacing $N with the number of MB you want from the beginning of the
drive to remain in the block device cache)
This will simply periodicaly read $N MB from the beginning of your
device, which keeps it in the buffer. I did this on a drive exported
over ATA-over-ethernet to my windows machine as it's boot drive, and had
sub-millisecond access times, very fast boot and app loading times...
Although I was keeping the entire C: drive (6GB) in the buffer on my
home server with 8GB ram :)

The second way I have used (and replaced with the above for simplicity)
is to set up an mdraid1 between the ramdisk and an equally sized
partition, using the write-mostly and write-behind options, then use
dmsetup (or LVM, I went for the direct approach) to concatenate it with
the rest of the disk.

My first way works brilliantly for me, as it is so simple yet so safe. I
have even had a power cut in the past and no data was lost (although of
course it forced windows to do checkdisk on boot, but it would have
anyway).

Hope this is helpful
Mouse

-----Original Message-----
From: linux-lvm-***@redhat.com [mailto:linux-lvm-***@redhat.com]
On Behalf Of Stuart D. Gathman
Sent: 13 May 2008 02:29
To: LVM general discussion and development
Subject: Re: [linux-lvm] lvm partition on ramdisk
Post by Larry Dickson
However, let me follow up your (and Stuart's) point. Are you saying
that an
Post by Larry Dickson
unmounted LVM volume will mess up the boot, even if the volume in
question
Post by Larry Dickson
is not mapped to boot or /? I was proceeding under the assumption that
LVM
Post by Larry Dickson
would be happy to sew the pieces together again later, even if the
data in
Post by Larry Dickson
them is trashed.
As long as the VG is not needed in initrd (e.g. a test VG), you should
be ok. You will simply have to go through the procedure of removing the
"failed" PV and adding it back after a reboot. As long as your root fs
(and /usr and other stuff needed at startup) are not on the test VG, you
should be fine. The problem is that the VG will not activate
automatically
with a missing PV. Even with --partial, it will activate the VG
as readonly metadata. Yes, AIX handles this better, IMO. But Linux LVM
is
getting there.

For your application, you should make a separate "testvg" VG for your
test
that does not have your system. At boot, activate the VG with
--partial,
then use "pvcreate -u " to set the UUID on the ramdisk to match the UUID
originally on the ramdisk, followed by vgcfgrestore.
--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703
591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


_______________________________________________
linux-lvm mailing list
linux-***@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


This has been checked by www.blackspider.com


_________________________________________________________

Zetex Semiconductors - Solutions for an analog world.

http://www.zetex.com
http://www.zetex.cn

E-MAILS are susceptible to interference. You should not assume that
the contents originated from the sender or the Zetex Group or that they
have been accurately reproduced from their original form.
Zetex accepts no responsibility for information, errors or omissions in
this e-mail nor for its use or misuse nor for any act committed or
omitted in connection with this communication.
If in doubt, please verify the authenticity with the sender.
_________________________________________________________
Larry Dickson
2008-05-13 14:16:20 UTC
Permalink
Hi all,

I appreciate and will take note of all your suggestions. Thank you for
taking the time! Here is how I achieved victory on this, in case anyone else
has a similar need. (And it was victory - 98.8% of the speed of a raw
device.)

I did pvcreate on the full (256MB) ramdisk and the full RAID array, without
partitioning. Then I did vgcreate and lvcreate on the ramdisk only (it
allowed me only 252MB, which was enough). Then vgextend to add the RAID
array, and lvresize to extend the logical volume to the full terabyte. This,
as I checked, put the ramdisk at the front of the lv.

Because I had created the ramdisk, for efficiency, with
ramdisk_blocksize=4096, I had trouble getting my file system to mount. The
magic was to make it with

mkfs.vfat -S 4096 -I -F 32

on the full, unpartitioned lv. Then it mounted, with the entire FAT on
ramdisk, and wrote very fast because FAT32 (like DOS) lays down data in
order from the start of a disk and does not skip around (I'd be interested
if anyone knows any other file systems with that property).

Larry
Post by Karl Wagner
If this is what you are looking to do, there are two more approaches you
could consider. I have used both to varying degrees of success.
The first is to just keep the first part of the disk in cache. Ie. Set
up the device as you want, then run the following either as a cron job
cat /dev/yourdevice | dd bs=1048576 count=$N of=/dev/null
(replacing $N with the number of MB you want from the beginning of the
drive to remain in the block device cache)
This will simply periodicaly read $N MB from the beginning of your
device, which keeps it in the buffer. I did this on a drive exported
over ATA-over-ethernet to my windows machine as it's boot drive, and had
sub-millisecond access times, very fast boot and app loading times...
Although I was keeping the entire C: drive (6GB) in the buffer on my
home server with 8GB ram :)
The second way I have used (and replaced with the above for simplicity)
is to set up an mdraid1 between the ramdisk and an equally sized
partition, using the write-mostly and write-behind options, then use
dmsetup (or LVM, I went for the direct approach) to concatenate it with
the rest of the disk.
My first way works brilliantly for me, as it is so simple yet so safe. I
have even had a power cut in the past and no data was lost (although of
course it forced windows to do checkdisk on boot, but it would have
anyway).
Hope this is helpful
Mouse
-----Original Message-----
On Behalf Of Stuart D. Gathman
Sent: 13 May 2008 02:29
To: LVM general discussion and development
Subject: Re: [linux-lvm] lvm partition on ramdisk
Post by Larry Dickson
However, let me follow up your (and Stuart's) point. Are you saying
that an
Post by Larry Dickson
unmounted LVM volume will mess up the boot, even if the volume in
question
Post by Larry Dickson
is not mapped to boot or /? I was proceeding under the assumption that
LVM
Post by Larry Dickson
would be happy to sew the pieces together again later, even if the
data in
Post by Larry Dickson
them is trashed.
As long as the VG is not needed in initrd (e.g. a test VG), you should
be ok. You will simply have to go through the procedure of removing the
"failed" PV and adding it back after a reboot. As long as your root fs
(and /usr and other stuff needed at startup) are not on the test VG, you
should be fine. The problem is that the VG will not activate
automatically
with a missing PV. Even with --partial, it will activate the VG
as readonly metadata. Yes, AIX handles this better, IMO. But Linux LVM
is
getting there.
For your application, you should make a separate "testvg" VG for your
test
that does not have your system. At boot, activate the VG with
--partial,
then use "pvcreate -u " to set the UUID on the ramdisk to match the UUID
originally on the ramdisk, followed by vgcfgrestore.
--
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703
591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
This has been checked by www.blackspider.com
_________________________________________________________
Zetex Semiconductors - Solutions for an analog world.
http://www.zetex.com
http://www.zetex.cn
E-MAILS are susceptible to interference. You should not assume that
the contents originated from the sender or the Zetex Group or that they
have been accurately reproduced from their original form.
Zetex accepts no responsibility for information, errors or omissions in
this e-mail nor for its use or misuse nor for any act committed or
omitted in connection with this communication.
If in doubt, please verify the authenticity with the sender.
_________________________________________________________
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Stuart D. Gathman
2008-05-13 17:21:06 UTC
Permalink
Post by Larry Dickson
on the full, unpartitioned lv. Then it mounted, with the entire FAT on
ramdisk, and wrote very fast because FAT32 (like DOS) lays down data in
order from the start of a disk and does not skip around (I'd be interested
if anyone knows any other file systems with that property).
The SysV filesystem put a fixed size inode table at the beginning of a
partition. More modern filesystems from ext to reiser try to distribute
the meta-data to keep it closer to the data. This is, of course, counter
productive when the beginning of a disk is significantly faster and seek-free
as in your setup.

Since ext3 inode placement is table driven (with the table in a magic inode),
there is probably a simple patch to mke2fs to create only one inode table at
the beginning of a drive. In fact, I wonder if there is already an option...
looks like -g blocks_per_group might do the trick - assuming inodes are
at the beginning of a block group, rather than the middle. If not,
a patch to mke2fs is needed to do what you want.
--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Larry Dickson
2008-05-19 14:37:13 UTC
Permalink
Final results FYI in this investigation: A recent Ubuntu release gets past
the 1TB limit. I found binaries for dosfstools_2.11-2.1ubuntu3 at

http://packages.ubuntu.com/feisty/otherosfs/dosfstools

and unpacked them with ar -x. The source is in

https://launchpad.net/ubuntu/+source/dosfstools/2.11-2.1ubuntu3

With 4KB sectors, it defaults to 128KB clusters and reaches over 97% write
speed on an 8TB volume. The ramdisk area needed is a little over 512MB, so
if you use 768MB you get quite a bit of room for directories also on
ramdisk, and with a little finesse you can even make the subdirectories lay
themselves down on ramdisk. To be "Windows-legal" you could use 32KB
clusters and a little over 2GB ramdisk (or a little over 1GB with one FAT).
Linux is happy with the big clusters, and according to the design should
actually be willing to go to 16TB.

Larry
Post by Larry Dickson
Post by Larry Dickson
on the full, unpartitioned lv. Then it mounted, with the entire FAT on
ramdisk, and wrote very fast because FAT32 (like DOS) lays down data in
order from the start of a disk and does not skip around (I'd be
interested
Post by Larry Dickson
if anyone knows any other file systems with that property).
The SysV filesystem put a fixed size inode table at the beginning of a
partition. More modern filesystems from ext to reiser try to distribute
the meta-data to keep it closer to the data. This is, of course, counter
productive when the beginning of a disk is significantly faster and seek-free
as in your setup.
Since ext3 inode placement is table driven (with the table in a magic inode),
there is probably a simple patch to mke2fs to create only one inode table at
the beginning of a drive. In fact, I wonder if there is already an option...
looks like -g blocks_per_group might do the trick - assuming inodes are
at the beginning of a block group, rather than the middle. If not,
a patch to mke2fs is needed to do what you want.
--
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Gerrard Geldenhuis
2008-05-19 15:01:08 UTC
Permalink
Thanks for the update. I have not followed the thread closely... did you
see any significant measured peformance increase in using lvm on a
ramdisk?



Regards



________________________________

From: linux-lvm-***@redhat.com [mailto:linux-lvm-***@redhat.com]
On Behalf Of Larry Dickson
Sent: 19 May 2008 15:37
To: LVM general discussion and development
Subject: Re: [linux-lvm] lvm partition on ramdisk



Final results FYI in this investigation: A recent Ubuntu release gets
past the 1TB limit. I found binaries for dosfstools_2.11-2.1ubuntu3 at

http://packages.ubuntu.com/feisty/otherosfs/dosfstools

and unpacked them with ar -x. The source is in

https://launchpad.net/ubuntu/+source/dosfstools/2.11-2.1ubuntu3


With 4KB sectors, it defaults to 128KB clusters and reaches over 97%
write speed on an 8TB volume. The ramdisk area needed is a little over
512MB, so if you use 768MB you get quite a bit of room for directories
also on ramdisk, and with a little finesse you can even make the
subdirectories lay themselves down on ramdisk. To be "Windows-legal" you
could use 32KB clusters and a little over 2GB ramdisk (or a little over
1GB with one FAT). Linux is happy with the big clusters, and according
to the design should actually be willing to go to 16TB.

Larry
Post by Larry Dickson
on the full, unpartitioned lv. Then it mounted, with the entire FAT on
ramdisk, and wrote very fast because FAT32 (like DOS) lays down data in
order from the start of a disk and does not skip around (I'd be
interested
Post by Larry Dickson
if anyone knows any other file systems with that property).
The SysV filesystem put a fixed size inode table at the beginning of a
partition. More modern filesystems from ext to reiser try to distribute
the meta-data to keep it closer to the data. This is, of course,
counter
productive when the beginning of a disk is significantly faster and
seek-free
as in your setup.

Since ext3 inode placement is table driven (with the table in a magic
inode),
there is probably a simple patch to mke2fs to create only one inode
table at
the beginning of a drive. In fact, I wonder if there is already an
option...
looks like -g blocks_per_group might do the trick - assuming inodes are
at the beginning of a block group, rather than the middle. If not,
a patch to mke2fs is needed to do what you want.

--
Stuart D. Gathman <***@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703
591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Larry Dickson
2008-05-19 15:23:01 UTC
Permalink
The purpose was to move seeking operations onto the ramdisk (ultimately
NVRAM or battery-backed), and use a file system that allocates disk sectors
in sequential order for scientific (no-delete) data streaming writes. It
allows near full speed writing to the main (big RAID) storage, and yet a
standard file system structure appears on the recorded data at little cost.
Log-structured file systems serve a similar purpose, but the structure of
the data and metadata is quite complex, and an additional step of "capping"
is needed if random-access reads are to be reasonably fast.

Larry
Thanks for the update. I have not followed the thread closely… did you
see any significant measured peformance increase in using lvm on a ramdisk?
Regards
------------------------------
*On Behalf Of *Larry Dickson
*Sent:* 19 May 2008 15:37
*To:* LVM general discussion and development
*Subject:* Re: [linux-lvm] lvm partition on ramdisk
Final results FYI in this investigation: A recent Ubuntu release gets past
the 1TB limit. I found binaries for dosfstools_2.11-2.1ubuntu3 at
http://packages.ubuntu.com/feisty/otherosfs/dosfstools
and unpacked them with ar -x. The source is in
https://launchpad.net/ubuntu/+source/dosfstools/2.11-2.1ubuntu3
With 4KB sectors, it defaults to 128KB clusters and reaches over 97%
write speed on an 8TB volume. The ramdisk area needed is a little over
512MB, so if you use 768MB you get quite a bit of room for directories also
on ramdisk, and with a little finesse you can even make the subdirectories
lay themselves down on ramdisk. To be "Windows-legal" you could use 32KB
clusters and a little over 2GB ramdisk (or a little over 1GB with one FAT).
Linux is happy with the big clusters, and according to the design should
actually be willing to go to 16TB.
Larry
Post by Larry Dickson
on the full, unpartitioned lv. Then it mounted, with the entire FAT on
ramdisk, and wrote very fast because FAT32 (like DOS) lays down data in
order from the start of a disk and does not skip around (I'd be
interested
Post by Larry Dickson
if anyone knows any other file systems with that property).
The SysV filesystem put a fixed size inode table at the beginning of a
partition. More modern filesystems from ext to reiser try to distribute
the meta-data to keep it closer to the data. This is, of course, counter
productive when the beginning of a disk is significantly faster and seek-free
as in your setup.
Since ext3 inode placement is table driven (with the table in a magic inode),
there is probably a simple patch to mke2fs to create only one inode table at
the beginning of a drive. In fact, I wonder if there is already an option...
looks like -g blocks_per_group might do the trick - assuming inodes are
at the beginning of a block group, rather than the middle. If not,
a patch to mke2fs is needed to do what you want.
--
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Larry Dickson
2008-05-19 15:41:38 UTC
Permalink
Rereading your question more precisely, the performances were (write speeds,
as percentage of long sequential writes to raw device, 64GB timed, with
pre-filling of buffers in all tests):

97% ramdisk-assisted FAT32
79% ext2 (fastest non-ramdisk-assisted FS)
59% FAT32
58% xfs

So, in respect of performance loss, the ramdisk results in at least a
tenfold improvement. In the test, not only the FAT but the directory was on
ramdisk - and this is realistic.

Larry
Post by Larry Dickson
The purpose was to move seeking operations onto the ramdisk (ultimately
NVRAM or battery-backed), and use a file system that allocates disk sectors
in sequential order for scientific (no-delete) data streaming writes. It
allows near full speed writing to the main (big RAID) storage, and yet a
standard file system structure appears on the recorded data at little cost.
Log-structured file systems serve a similar purpose, but the structure of
the data and metadata is quite complex, and an additional step of "capping"
is needed if random-access reads are to be reasonably fast.
Larry
Thanks for the update. I have not followed the thread closely… did you
see any significant measured peformance increase in using lvm on a ramdisk?
Regards
------------------------------
*On Behalf Of *Larry Dickson
*Sent:* 19 May 2008 15:37
*To:* LVM general discussion and development
*Subject:* Re: [linux-lvm] lvm partition on ramdisk
Final results FYI in this investigation: A recent Ubuntu release gets past
the 1TB limit. I found binaries for dosfstools_2.11-2.1ubuntu3 at
http://packages.ubuntu.com/feisty/otherosfs/dosfstools
and unpacked them with ar -x. The source is in
https://launchpad.net/ubuntu/+source/dosfstools/2.11-2.1ubuntu3
With 4KB sectors, it defaults to 128KB clusters and reaches over 97%
write speed on an 8TB volume. The ramdisk area needed is a little over
512MB, so if you use 768MB you get quite a bit of room for directories also
on ramdisk, and with a little finesse you can even make the subdirectories
lay themselves down on ramdisk. To be "Windows-legal" you could use 32KB
clusters and a little over 2GB ramdisk (or a little over 1GB with one FAT).
Linux is happy with the big clusters, and according to the design should
actually be willing to go to 16TB.
Larry
Post by Larry Dickson
on the full, unpartitioned lv. Then it mounted, with the entire FAT on
ramdisk, and wrote very fast because FAT32 (like DOS) lays down data in
order from the start of a disk and does not skip around (I'd be
interested
Post by Larry Dickson
if anyone knows any other file systems with that property).
The SysV filesystem put a fixed size inode table at the beginning of a
partition. More modern filesystems from ext to reiser try to distribute
the meta-data to keep it closer to the data. This is, of course, counter
productive when the beginning of a disk is significantly faster and seek-free
as in your setup.
Since ext3 inode placement is table driven (with the table in a magic inode),
there is probably a simple patch to mke2fs to create only one inode table at
the beginning of a drive. In fact, I wonder if there is already an option...
looks like -g blocks_per_group might do the trick - assuming inodes are
at the beginning of a block group, rather than the middle. If not,
a patch to mke2fs is needed to do what you want.
--
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
_______________________________________________
linux-lvm mailing list
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Continue reading on narkive:
Search results for '[linux-lvm] lvm partition on ramdisk' (Questions and Answers)
10
replies
What is AIX Box?
started 2006-05-08 15:58:44 UTC
hardware
Loading...