Welcome to the Reiser4 Wiki, the Wiki for users and developers of the ReiserFS and Reiser4 filesystems.

For now, most of the documentation is just a snapshot of the old Namesys site (archive.org, 2007-09-29).

There was also a Reiser4 Wiki (archive.org, 2007-07-06) once on pub.namesys.com.

Logical Volumes Administration

From Reiser4 FS Wiki
(Difference between revisions)
Jump to: navigation, search
(Adding a data brick to LV)
 
(44 intermediate revisions by one user not shown)
Line 1: Line 1:
 +
'''WARNING! FSCK doesn't work correctly on volumes with scaling-out abilities. Any attempts to repair such volumes will result in data loss'''
 +
 +
Before working with logical volumes you need to understand some basic [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Background principles].
 +
 
Logical volume (LV) can be composed of any number of block devices,
 
Logical volume (LV) can be composed of any number of block devices,
 
different in physical and geometric parameters. However the optimal
 
different in physical and geometric parameters. However the optimal
Line 24: Line 28:
  
 
The item #4 is to handle incomplete operations interrupted by various
 
The item #4 is to handle incomplete operations interrupted by various
reasons (system crash, hard reset, etc).
+
reasons (system crash, hard reset, etc) when bringing logical volumes
 +
on-line.
  
 
For each volume its configuration should be stored somewhere (but not
 
For each volume its configuration should be stored somewhere (but not
Line 31: Line 36:
 
this. Volume configuration is needed to facilitate deploying a volume.
 
this. Volume configuration is needed to facilitate deploying a volume.
  
'''Abstract capacity''' (or simply capacity) of a brick is a positive
+
'''Capacity of a brick''' (or abstract capacity) is a positive
 
integer number. Capacity is a brick's property defined by user. Don't
 
integer number. Capacity is a brick's property defined by user. Don't
 
confuse it with the size of block device. Think of it as of brick's
 
confuse it with the size of block device. Think of it as of brick's
Line 61: Line 66:
 
volume partitioning. Obviously, C1 + C2 + ... = 1.
 
volume partitioning. Obviously, C1 + C2 + ... = 1.
  
'''(Real) data space usage''' on a brick is number of data blocks, stored
+
'''Real data space usage''' on a brick is number of data blocks, stored
 
on that brick.
 
on that brick.
  
'''Ideal (or expected) data space usage''' on a brick is T*C, where T is
+
'''Ideal data space usage''' on a brick is defined as T*C, where T
total number of data blocks stored in the volume. C is relative
+
is total number of data blocks stored in the volume. C is relative
 
capacity of the brick.
 
capacity of the brick.
  
Line 82: Line 87:
 
the total volume's capacity.
 
the total volume's capacity.
  
Most volume operations are accompanied by rebalancing, which keeps
+
In contrast with regular file operations, volume operations break
fairness of distribution. For example, adding a brick to a logical
+
fairness of data distribution on your logical volume. To restore
volume changes its partitioning, and hence, breaks fairness of the
+
fairness of distribution, a special balancing procedure should be run
distribution, so we need to move some data stripes to the new brick to
+
on the volume. For example, after adding a brick to a logical volume,
make distribution fair. Also you can not simply remove a brick from
+
the balancing procedure will populate the new brick with data, moved
a logical volume - all data stripes should be moved from that brick
+
from other bricks.
to other bricks of the logical volume.
+
  
Every time when user performs a volume operation, Reiser5 marks LV as
+
All volume operations except brick removal are fast, atomic and leave
"not balanced". After successful balancing the status of LV is changed
+
the volume in unbalanced state.
to "balanced". If balancing procedure fails for some reasons, it
+
should be resumed manually (with volume.reiser4 utility).
+
  
It is allowed to perform regular file operations on not balanced LV.
+
Operation of brick removal always includes balancing, which moves
However, in this case:
+
data from the brick you want to remove to other bricks of the volume.
a) we don't guarantee a good quality of data distribution on your LV.
+
If that data migration is interrupted for some reason, then the volume
b) you won't be able to perform volume operations on your LV except balancing - any other volume operation will return error (EBUSY).
+
is marked as a "volume with incomplete brick removal".
  
So, don't forget to bring your LV to the balanced state as soon as
+
It is allowed to perform regular file and volume operations on a not
possible!
+
balanced LV (assuming, it was not incomplete removal). However, in this
 +
case we don't guarantee a good quality of data distribution on your LV.
 +
In addition, on a volume with incomplete removal you won't be able to
 +
perform regular volume operations - first you will need to complete the
 +
removal by running a special removal completion procedure on your volume.
  
 
= Prepare Software and Hardware =
 
= Prepare Software and Hardware =
Line 108: Line 114:
 
Build, install and boot kernel with Reiser4 of software framework
 
Build, install and boot kernel with Reiser4 of software framework
 
release number 5.X.Y. Kernel patches can be found [https://sourceforge.net/projects/reiser4/files/v5-unstable/ here].  
 
release number 5.X.Y. Kernel patches can be found [https://sourceforge.net/projects/reiser4/files/v5-unstable/ here].  
 +
When building the kernel, make sure that Reiser4-specific configuration option "Enable Plan-A key allocation scheme" is '''disabled''',
 +
or check that the .config file contains the following line:
 +
 +
# CONFIG_REISER4_OLD is not set
  
 
Note that by Linux kernel and GNU utilities the testing stuff is
 
Note that by Linux kernel and GNU utilities the testing stuff is
Line 129: Line 139:
 
Start from choosing a unique ID (uuid) of your volume. By default it
 
Start from choosing a unique ID (uuid) of your volume. By default it
 
is generated by mkfs utility. However, user can generate it himself by
 
is generated by mkfs utility. However, user can generate it himself by
proper tools (e.g. uuid(1)) and store in an environment variable for
+
proper tools (e.g. uuidgen(1)) and store in an environment variable for
 
convenience:
 
convenience:
  
  # VOL_ID=`uuid -v4`
+
  # VOL_ID=`uuidgen`
 
  # echo "Using uuid $VOL_ID"
 
  # echo "Using uuid $VOL_ID"
  
Line 139: Line 149:
 
volume size. On the other hand, too small stripes will increase space
 
volume size. On the other hand, too small stripes will increase space
 
consumption on your meta-data brick. In our example we choose stripe
 
consumption on your meta-data brick. In our example we choose stripe
size 256K:
+
size 512K:
  
  # STRIPE=256K
+
  # STRIPE=512K
 
  # echo "Using stripe size $STRIPE"
 
  # echo "Using stripe size $STRIPE"
  
Line 181: Line 191:
 
At any time you are able to add a data brick to your LV. You can do it
 
At any time you are able to add a data brick to your LV. You can do it
 
in parallel with regular file operations executing on this volume.
 
in parallel with regular file operations executing on this volume.
Make sure, however, that there is no other volume operations (e.g.
+
Make sure, however, that your volume is not marked as "volume with
removing a brick) over your volume in progress, otherwise your
+
incomplete brick removal", or there is no other volume operations
operation will fail with EBUSY.
+
over your volume in progress. Otherwise your operation will fail with
 +
EBUSY.
  
 
Obviously, adding a brick will increase capacity of your volume.
 
Obviously, adding a brick will increase capacity of your volume.
Line 199: Line 210:
 
  # mkfs.reiser4 -U $VOL_ID -t $STRIPE -a /dev/vdb2
 
  # mkfs.reiser4 -U $VOL_ID -t $STRIPE -a /dev/vdb2
  
Important: it is important that data brick is formatted with the same
+
It is important that data brick is formatted with the same volume ID
volume ID and stripe size, as the meta-data brick of your logical volume.
+
and stripe size, as the meta-data brick of your logical volume.
 
Otherwise, operation of adding a data brick will fail.
 
Otherwise, operation of adding a data brick will fail.
  
Line 210: Line 221:
 
  # volume.reiser4 -a /dev/vdb2 /mnt
 
  # volume.reiser4 -a /dev/vdb2 /mnt
  
The procedure of adding a brick automatically invokes re-balancing,
+
By default operation of adding a brick is fast and atomic and leaves
which moves a portion of data stripes to the newly added brick (so
+
the volume in unbalanced state, so after adding a brick you might want
that the resulted distribution will fair).
+
to run a balancing procedure, which will move a portion of data to the
 +
new brick from other bricks of the logical volume, which will make data
 +
distribution on your volume fair:
  
Portion of data blocks moved during such rebalancing is equal to the
+
# volume.reiser4 -b /mnt
 +
 
 +
Portion of data blocks, being moved during such rebalancing, is equal to
 
relative capacity of the new brick, that is to the portion of capacity
 
relative capacity of the new brick, that is to the portion of capacity
that the new brick adds to updated LV's capacity. This important
+
that the new brick adds to updated LV's capacity. This important property
property defines the cost of balancing procedure. If the portion of
+
defines the cost of balancing procedure. If the portion of capacity added
capacity added by a brick is small, then number of stripes moved
+
by a brick is small, then number of stripes moved during balancing is also small.
during balancing is also small.
+
  
Like other user-space utilities, the operation of adding a brick can
+
Using this operation in conjunction with the option -B (--with-balance)
return error, even in the assumption that the brick you wanted to add
+
will automatically trigger the balancing procedure:
is properly formatted. In this case check the status of your LV:
+
  
  # volume.reiser4 /mnt
+
  # volume.reiser4 -Ba /dev/vdb2 /mnt
 
+
 
If the volume is unbalanced, then simply complete balancing manually:
+
Upon successful completion update your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration]. That is, increment (#2), add info about the new brick
 
+
to (#3) and remove records at (#4).
# volume.reiser4 -b /mnt
+
 
+
Otherwise, check number of bricks in your LV. Most likely that it is
+
the same as it was before the failed operation. In this case simply
+
repeat the operation of adding a brick from scratch.
+
  
Upon successful completion update your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration]. That is, increment (#2), add info about the new brick to (#3) and remove
+
When adding more than one brick at once, call volume.reiser4 with option -a
records at (#4).
+
for each brick individually in any order. It will be reasonable to not complete
 +
each call with balancing. Run balancing only after adding the last brick.
  
 
= Removing a data brick from LV =
 
= Removing a data brick from LV =
  
  
At any time you are able to remove a data brick from your LV. You can
+
At any time you are able to remove any data brick from your LV.
do it in parallel with regular file operations executing on this
+
You can do it in parallel with regular file operations executing on that
volume. Make sure, however, that there is no other volume operations
+
volume. Make sure, however, that your volume is not marked as "volume
(e.g. adding a brick) over your volume in progress, otherwise your
+
with incomplete brick removal", or there is no other volume operations
operation will fail with EBUSY.
+
over your volume in progress. Otherwise your operation will fail with
 +
EBUSY.  
  
Obviously, removing a brick will decrease abstract capacity of your
+
Obviously, the removal operation will decrease abstract capacity of your LV.
LV. Note that other bricks should have enough space to store all data
+
Note that other bricks should have enough space to store all data blocks of
blocks of the brick you want to remove, otherwise, the removal
+
the brick you want to remove, otherwise, the removal operation will return
operation will return error (ENOSPC).
+
error (ENOSPC).
  
Suppose you want to remove brick /dev/vdb2 from your LV mounted at
+
Suppose you want to remove brick /dev/vdb2 from your LV mounted at /mnt.
/mnt.
+
  
Update your volume configuration with the UUID and name of the brick
+
Update your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration]
you want to remove (#item #4).
+
with the UUID and name of the brick you want to remove (#item #4).
  
 
To remove a brick simply pass its name as an argument for option "-r"
 
To remove a brick simply pass its name as an argument for option "-r"
Line 263: Line 272:
 
  # volume.reiser4 -r /dev/vdb2 /mnt
 
  # volume.reiser4 -r /dev/vdb2 /mnt
  
The procedure of brick removal automatically invokes re-balancing,
+
The procedure of brick removal starts from moving all data from the
which distributes data of the brick to be removed among other bricks,
+
brick you want to remove to other bricks of your volume, so that
so that resulted distribution is also fair. Portion of data stripes
+
resulted data distribution among the rest of bricks will be also fair.
moved during such rebalancing is equal to the relative capacity of the
+
 
brick to be removed (that it to the portion of capacity that the brick
+
Portion of data stripes being moved during such migration is equal
added to LV's capacity).
+
to the relative capacity of the brick to be removed (that it to the
 +
portion of capacity that the brick added to LV's capacity).
 +
Successful brick removal always leaves the volume is balanced state.
 +
 
 +
So, in contrast with the operation of adding a brick, removing a brick
 +
is a rather long operation, which can be interrupted for various reasons.
 +
In this case volume will be marked as a "volume with incomplete brick
 +
removal".
  
It can happen, that the command above completes with error (like other
+
To check removal status of your LV simply run
user-space applications). In this case check the status of your LV:
+
  
 
  # volume.reiser4 /mnt
 
  # volume.reiser4 /mnt
  
If volume is not balanced, then simply complete balancing manually:
+
and check the field "health".
  
# volume.reiser4 -b /mnt
+
To complete brick removal in the current mount session simply run
  
Otherwise, check the number of the bricks in your logical volume - it
+
# volume.reiser4 -R /mnt
should be the same as before the failed operation. The error -ENOSPC
+
indicates that free space on other bricks is not enough to fit all the
+
data of the brick you want to remove.
+
  
On success update your volume configuration: remove the information about
+
Note, that the option -R (--finish-removal) doesn't accept any arguments.
the brick /dev/vdb2 at #3 and #4. Check your kernel logs: it should contain
+
 
a message that brick /dev/vdb2 has been unregistered. Now device /dev/vdb2
+
On success update your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration]:
doesn't belong to the logical volume any more, and you can reuse it for
+
remove the information about the brick /dev/vdb2 at #3 and #4.
other purposes (re-format, etc).
+
Check your kernel logs: it should contain a message that brick /dev/vdb2
 +
has been unregistered. Now device /dev/vdb2 doesn't belong to the logical
 +
volume any more, and you can reuse it for other purposes (re-format, etc).
  
 
= Changing brick's capacity =
 
= Changing brick's capacity =
  
  
At any time (in the assumption that no other volume operation is in
+
At any time you can change abstract capacity of any brick to some new
progress) you can change abstract capacity of any brick to some new
+
value, different from 0. You can do it in parallel with regular file
value, different from 0. Changing capacity always changes volume
+
operations executing on that volume. Make sure, however, that your volume
partitioning, and therefore, breaks fairness of distribution, so
+
is not marked as "volume with incomplete brick removal", or there is no
Reiser5 automatically launches rebalancing to make sure that resulted
+
other volume operations over your volume in progress. Otherwise your
distribution is fair for the new set of capacities.
+
operation will fail with EBUSY.
  
In particular, increasing bricks capacity will move some data from
+
Changing capacity always changes volume partitioning, and therefore,
other bricks to the brick, whose capacity was increased. Decreasing
+
breaks fairness of data distribution on the volume. By default operation
bricks capacity will move some data from the brick, whose capacity was
+
of changing brick capacity leaves the volume in unbalanced state, so
decreased, to other bricks.
+
after changing brick capacity you might want to run a balancing procedure
 +
to make data distribution on your volume fair. In particular, after
 +
increasing brick capacity the balancing procedure will move some data
 +
from other bricks to the brick, whose capacity was increased. After
 +
decreasing bricks capacity the balancing procedure will move some data
 +
from the brick, whose capacity was decreased, to other bricks.
  
 
To change abstract capacity of a brick /dev/vdb1 to a new value (e.g.
 
To change abstract capacity of a brick /dev/vdb1 to a new value (e.g.
Line 313: Line 332:
 
mounted at /mnt".
 
mounted at /mnt".
  
The operation of changing capacity can return error. Most likely, it
+
By default the operation of changing capacity is fast, atomic and leave
is -ENOSPC, which is a side effect of concurrent regular file writes.
+
the volume in unbalanced state. To automatically invoke balancing,
In this case check the status of your LV. If it is unbalanced, then
+
use this operation in conjunction with the option -B (--with-balance).
consider removing some files from your LV and complete balancing by
+
Also you can run a balancing procedure later at any time by executing
running
+
  
 
  # volume.reiser4 -b /mnt
 
  # volume.reiser4 -b /mnt
  
Otherwise, repeat the operation from scratch.
+
When changing capacities of more than one brick at once, call the
 +
volume.reiser4 utility for each brick individually in any order.
 +
It will be reasonable to not complete each call with balancing. Run
 +
balancing after changing capacity of the last brick.
  
 
Comment. Changing bricks capacity to 0 is undefined and will return
 
Comment. Changing bricks capacity to 0 is undefined and will return
 
error. Consider brick removal operation instead.
 
error. Consider brick removal operation instead.
 
  
 
= Operations with meta-data brick =
 
= Operations with meta-data brick =
  
  
Meta-data brick can also contain data stripes and participate in data
+
Meta-data brick also can contain data stripes and participate in data
distribution like other data bricks. So that all the volume operations
+
distribution like other data bricks. All the volume operations described
described above are also applicable to meta-data brick. Note, however,
+
above are also applicable to meta-data brick. Note, however, that it is
that it is impossible to completely remove meta-data brick from the
+
impossible to completely remove meta-data brick from the logical volume
logical volume for obvious reasons (meta-data need to be stored
+
for obvious reasons (meta-data need to be stored somewhere), so brick
somewhere), so brick removal operation applied to the meta-data brick
+
removal operation applied to the meta-data brick actually removes it only
actually removes it from Data-Storage Array (DSA), not from the
+
from Data-Storage Array (DSA), which is a subset of LV consisting of bricks,
logical volume. DSA is a subset of LV consisting of bricks,
+
participating in regular data distribution, corresponding to their abstract
participating in data distribution. Once you remove meta-data brick
+
capacities.
from DSA, that brick will be used only to store meta-data. Operation
+
 
of adding a brick, being applied to a meta-data brick, returns the
+
Once you remove meta-data brick from DSA, that brick will be used only
last one back to DSA.
+
to store meta-data. Operation of adding a brick, being applied to a
 +
meta-data brick, returns the last one back to DSA.
  
 
Important: Reiser5 doesn't count busy data and meta-data blocks
 
Important: Reiser5 doesn't count busy data and meta-data blocks
Line 348: Line 369:
 
meta-data brick - Reiser5 knows only total space occupied.
 
meta-data brick - Reiser5 knows only total space occupied.
  
To check the status of meta-data brick simply run
+
To check the status of meta-data brick of the volume mounted at /mnt
 +
simply run
  
  # volume.reiser4 /mnt
+
  # volume.reiser4 -p0 /mnt
 
+
and compare values of "bricks total" and "bricks in DSA". If they are
+
equal, then meta-data brick participates in data distribution.
+
Otherwise, "bricks total" should be 1 more than "bricks in DSA" -
+
it indicates that meta-data brick doesn't participate in data
+
distribution (and therefore, doesn't contain data blocks). Note that
+
other cases are impossible: for data bricks participation in LV and
+
DSA is always equivalent.
+
  
 +
and check the field "in DSA".
  
 
= Unmounting a logical volume =
 
= Unmounting a logical volume =
  
  
To terminate a mount session just issue usual umount command with the
+
To terminate a mount session just issue usual umount against the mount point:
mount point specified.
+
 
 +
# umount /mnt
  
 
Note that after unmounting the volume all bricks by default remain to
 
Note that after unmounting the volume all bricks by default remain to
Line 373: Line 389:
  
 
  # volume.reiser4 -u BRICK_NAME
 
  # volume.reiser4 -u BRICK_NAME
 
  
 
= Deploying a logical volume after correct unmount =
 
= Deploying a logical volume after correct unmount =
  
  
Make sure (by checking your volume configuration) that all bricks of
+
After unmounting a logical volume all its bricks remain to be
the volume are registered in the system. To register a brick issue the following command:
+
registered in the system. So, if you want to mount the volume again,
 
+
simply issue the mount command against some its brick. is recommended
# volume.reiser4 -g BRICK_NAME
+
to issue it against meta-data brick.
 
+
The list of all volumes and bricks registered in the system can be found in the output of the
+
following command:
+
 
+
# volume.reisrer4 -l
+
 
+
Issue usual mount(8) command against one of the bricks of your volume. It
+
is recommended to issue it against meta-data brick.
+
  
 
NOTE: Reiser5 will refuse to mount a logical volume, in the case, when
 
NOTE: Reiser5 will refuse to mount a logical volume, in the case, when
Line 399: Line 406:
  
  
To mount your LV, first, make sure that all its bricks (data and
+
First of all, check [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing configuration]
meta-data) are registered in the system. Important: Reiser5 will refuse
+
of your volume and make sure that all its bricks (data and meta-data ones)
to mount a logical volume, in the case, when a wrong (incomplete or
+
are registered in the system. The list of registered bricks can be printed by
 +
 
 +
# volume.reiser4 -l
 +
 
 +
Also make sure that the set of registered per volume bricks doesn't contain
 +
bricks not mentioned in the [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration].
 +
 
 +
Important: Reiser5 will refuse to mount a logical volume, in the case, when a wrong (incomplete or
 
redundant) set of bricks is registered in the system. Redundant set of
 
redundant) set of bricks is registered in the system. Redundant set of
 
bricks appears, for example, when you mistakenly register a brick that
 
bricks appears, for example, when you mistakenly register a brick that
 
was removed from the logical volume. For this reasons we strongly
 
was removed from the logical volume. For this reasons we strongly
recommend for user to keep a track of his LV - store its configuration
+
recommend for user to keep a track of his LV - store its [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing configuration]
 
somewhere, but not in this volume! And don't forget to update that
 
somewhere, but not in this volume! And don't forget to update that
configuration after _every_ volume operation. If you lost
+
configuration after '''every''' volume operation. If you lost
 
configuration of your LV and don't remember it (wich is most likely
 
configuration of your LV and don't remember it (wich is most likely
 
for large volumes), then it will be rather painful to restore it:
 
for large volumes), then it will be rather painful to restore it:
currently there is no tools for to manage offline logical volumes.
+
currently there is no tools for to manage logical volumes off-line.
 
So that, users are prompted to do this on their own. It is not at all
 
So that, users are prompted to do this on their own. It is not at all
 
difficult.
 
difficult.
Line 422: Line 436:
 
  # volume.reiser4 -l
 
  # volume.reiser4 -l
  
To mount your LV simply issue a mount(8) command against one of the bricks of your
+
Now mount your LV, simply issuing a mount(8) command against one of the bricks of your
 
LV. We recommend to issue it against meta-data brick.
 
LV. We recommend to issue it against meta-data brick.
  
Line 445: Line 459:
  
  
== Adding a brick was interrupted ==
+
== Volume balancing was interrupted ==
  
  
Check your volume configuration. Register the old set of bricks (that
+
Check your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration].
is, the set of brick that the volume had before applying the
+
operation) and try to mount. In the case of error register also the
+
brick you wanted to add and try to mount again.
+
  
Check the status of your LV by running
+
Register the complete set of bricks and mount the volume
 +
by issuing the mount command against some its brick.
 +
 
 +
Check the balanced status of your LV by running
  
 
  # volume.reiser4 /mnt
 
  # volume.reiser4 /mnt
  
In the volume is unbalanced, then complete balancing manually by
+
and checking "balanced" value. If the volume is unbalanced, then
running
+
complete balancing by running
  
 
  # volume.reiser4 -b /mnt
 
  # volume.reiser4 -b /mnt
 
Check "bricks total" of your LV in the output of
 
 
# volume.reiser4 /mnt
 
 
Compare it with the old number of bricks in the configuration. The new
 
value should be an increment of the old one. If the number of bricks
 
is the same, then your operation of adding a brick was completely
 
rolled back by the transaction manager, so that you need to repeat it
 
from scratch. Otherwise, your operation was successfully completed -
 
update your volume configuration respectively.
 
 
  
 
== Brick removal was interrupted ==
 
== Brick removal was interrupted ==
  
  
Check your volume configuration. Register the old set of bricks (that
+
Check your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration]. Register the new set of bricks (that is, the set of bricks without the brick you wanted to remove). Try to mount the volume. In the case of error register also the brick you wanted to remove and try to mount again.
is, the set of brick that volume had before applying the interrupted
+
operation) except the brick you wanted to remove. Try to mount the
+
volume. In the case of error register also the brick you wanted to
+
remove and try to mount again.
+
  
Check the status of your LV:
+
Check the status of your LV by running
  
 
  # volume.reiser4 /mnt
 
  # volume.reiser4 /mnt
  
If the volume is unbalanced then complete balancing manually by
+
and checking the value of "health". If required, complete brick removal by running
running
+
  
  # volume.reiser4 -b /mnt
+
  # volume.reiser4 -R /mnt
  
Comment. After sucessful balancing completion the brick will be
+
Note, that the option -R doesn't accept any arguments.
automatically removed form the volume. Make sure of it by checking
+
After successful removal completion the brick will be automatically
status of your LV:
+
removed from the volume and unregistered. Make sure of it by checking
 +
status of your LV and the list of registered bricks:
  
 
  # volume.reiser4 /mnt
 
  # volume.reiser4 /mnt
  
Update your volume configuration respectively.
+
# volume.reiser4 -l
  
 
+
Upon successful completion update your [https://reiser4.wiki.kernel.org/index.php/Logical_Volumes_Administration#Basic_definitions._Volume_configuration._Brick.27s_capacity._Partitioning._Fair_distribution._Balancing volume configuration] respectively.
== Another volume operation was interrupted ==
+
 
+
 
+
Using the volume configuration, register the new set of bricks and
+
try to mount the volume. The mount should be successful.
+
 
+
Check the status of your LV:
+
 
+
# volume.reiser4 /mnt
+
 
+
If the volume is unbalanced then complete balancing manually by
+
running
+
 
+
# volume.reiser4 -b /mnt
+
  
 
= LV monitoring =
 
= LV monitoring =
Line 531: Line 515:
 
  bricks in DSA: Number of bricks participating in data distribution
 
  bricks in DSA: Number of bricks participating in data distribution
 
  balanced: Balanced status of the volume
 
  balanced: Balanced status of the volume
 +
health: Brick removal completion status
  
  
Line 718: Line 703:
 
= FAQ =
 
= FAQ =
  
Q. What happens if I lose a device-component (due to a breakdown, etc) of my logical volume?
+
'''What happens if I lose a brick (due to a breakdown, etc) of my logical volume?'''
  
A. Bodies of some your regular files will become "punched" in random places.
+
If you lose a meta-data brick, then you lose the whole volume. If you loose
 +
a data brick, then you'll be able to mount the volume, but bodies of some
 +
your regular files will become "punched" in random places.
 
Portion of such files depends on the relative capacity of the lost brick, on
 
Portion of such files depends on the relative capacity of the lost brick, on
 
the number of bricks in the logical volume, and on other factors. Fsck will be
 
the number of bricks in the logical volume, and on other factors. Fsck will be
Line 726: Line 713:
 
recommend to consider mirroring your bricks (e.g. by software, or hardware RAID-1)
 
recommend to consider mirroring your bricks (e.g. by software, or hardware RAID-1)
 
to avoid such highly unpleasant situations.
 
to avoid such highly unpleasant situations.
 +
 +
'''I am not able to complete brick removal because of "no space left on device". What should I do?'''
 +
 +
At the very beginning of removal operation the file system calculates precise
 +
amount of space on other bricks, needed to accommodate all the data of the brick
 +
you are going to remove. It could happen, however that free space on the bricks
 +
was near the calculated amount, and during the removal, exceeded it due to
 +
intensive regular file operations (caused e.g. by torrents activity) going in
 +
parallel. So, to complete the operation consider removing a part of regular files
 +
on your partition. In the future it will be possible to add a brick to a volume
 +
marked as a "volume with incomplete brick removal". It will also allow to complete
 +
removal operation.
 +
 +
 +
[[category:Reiser4]]

Latest revision as of 07:10, 13 July 2022

WARNING! FSCK doesn't work correctly on volumes with scaling-out abilities. Any attempts to repair such volumes will result in data loss

Before working with logical volumes you need to understand some basic principles.

Logical volume (LV) can be composed of any number of block devices, different in physical and geometric parameters. However the optimal configuration (true parallelism) imposes some restrictions and dependencies on the size of such devices.

WARNING: The stuff is not stable. Don't put important data to logical volumes managed by software of release number 5.X.Y. Also don't mount your old partitions in kernels with Reiser4 of SFRN 5.X.Y before its stabilization

IMPORTANT: Currently there is no tools to manage Reiser5 logical volumes off-line, so it it strongly recommended to save/update configurations of your LV in a file, which doesn't belong to that volume.


Contents

[edit] Basic definitions. Volume configuration. Brick's capacity. Partitioning. Fair distribution. Balancing

Basic configuration of a logical volume is the following information:

1) Volume UUID;
2) Number of bricks in the volume;
3) List of brick names or UUIDs in the volume;
4) UUID or name of the brick to be added/removed (if any). That brick is not counted in (2) and (3).

The item #4 is to handle incomplete operations interrupted by various reasons (system crash, hard reset, etc) when bringing logical volumes on-line.

For each volume its configuration should be stored somewhere (but not on that volume!) and properly updated before and after each volume operation performed on that volume. We make the user responsible for this. Volume configuration is needed to facilitate deploying a volume.

Capacity of a brick (or abstract capacity) is a positive integer number. Capacity is a brick's property defined by user. Don't confuse it with the size of block device. Think of it as of brick's "weight" in some units. And this is the user, who decides, which property of the brick to assign as its abstract capacity and in which units. In particular, it can be size of the block device in kilobytes, or its size in megabytes, or its throughput in M/sec, or other geometric or physical parameter of the device, associated with the brick. It is important that capacities of all bricks of the same logical volume are measured in the same units. Also, it would be utterly pointless to assign different properties as abstract capacities for bricks of the same LV. For example, size of block device for one brick, and disk bandwidth for another one.

Capacity of each brick gets initialized by mkfs utility. By default it is calculated as number of free blocks on the device at the very end of the formatting procedure. For meta-data brick it is calculated as 70% of such amount. Capacity of any brick can be changed on-line by user.

Capacity of a logical volume is defined as a sum of capacities of its bricks-components.

Relative capacity of a brick is the ratio of brick's capacity to volume's capacity. Relative capacity defines a portion of IO-requests that will be issued against that brick.

Array of relative capacities (C1, C2, ...) of all bricks is called volume partitioning. Obviously, C1 + C2 + ... = 1.

Real data space usage on a brick is number of data blocks, stored on that brick.

Ideal data space usage on a brick is defined as T*C, where T is total number of data blocks stored in the volume. C is relative capacity of the brick.

It is recommended to compose volumes in the way so that space-based partitioning coincides with throughput-based one - it would be the optimal volume configuration, which provides true parallelism. If it is impossible for some reason, then choose a preferred partitioning method (space-based, or throughput-based). Note that space-based partitioning saves volume space, whereas throughput based one saves volume throughput.

When performing regular file operations, Reiser5 distributes data stripes throughout the volume evenly and fairly. It means that portion of IO-requests issued against each brick is equal to its relative capacity, that is, to the portion of capacity that the brick adds to the total volume's capacity.

In contrast with regular file operations, volume operations break fairness of data distribution on your logical volume. To restore fairness of distribution, a special balancing procedure should be run on the volume. For example, after adding a brick to a logical volume, the balancing procedure will populate the new brick with data, moved from other bricks.

All volume operations except brick removal are fast, atomic and leave the volume in unbalanced state.

Operation of brick removal always includes balancing, which moves data from the brick you want to remove to other bricks of the volume. If that data migration is interrupted for some reason, then the volume is marked as a "volume with incomplete brick removal".

It is allowed to perform regular file and volume operations on a not balanced LV (assuming, it was not incomplete removal). However, in this case we don't guarantee a good quality of data distribution on your LV. In addition, on a volume with incomplete removal you won't be able to perform regular volume operations - first you will need to complete the removal by running a special removal completion procedure on your volume.

[edit] Prepare Software and Hardware

Build, install and boot kernel with Reiser4 of software framework release number 5.X.Y. Kernel patches can be found here. When building the kernel, make sure that Reiser4-specific configuration option "Enable Plan-A key allocation scheme" is disabled, or check that the .config file contains the following line:

# CONFIG_REISER4_OLD is not set

Note that by Linux kernel and GNU utilities the testing stuff is still recognized as "Reiser4". Make sure there is the following message in kernel logs:

"Loading Reiser4 (Software Framework Release: 5.X.Y)"

Build and install the latest libaal

Download, build and install the latest version 2.A.B of Reiser4progs package.

Make sure that utility for managing logical volumes is installed (as a part of reiser4progs package) on your machine:

# volume.reiser4 -?

[edit] Creating a logical volume

Start from choosing a unique ID (uuid) of your volume. By default it is generated by mkfs utility. However, user can generate it himself by proper tools (e.g. uuidgen(1)) and store in an environment variable for convenience:

# VOL_ID=`uuidgen`
# echo "Using uuid $VOL_ID"

Choose a stripe size for your logical volume. For a good quality of distribution it is recommended that stripe doesn't exceed 1/10000 of volume size. On the other hand, too small stripes will increase space consumption on your meta-data brick. In our example we choose stripe size 512K:

# STRIPE=512K
# echo "Using stripe size $STRIPE"

Start from creating the first brick of your volume - meta-data brick, passing volume-ID and stripe size to mkfs.reiser4 utility:

# mkfs.reiser4 -U $VOL_ID -t $STRIPE /dev/vdb1

Currently only one meta-data brick per volume is supported, so it is recommended that size of block device for meta-data brick in not too small. In most cases it will be enough, if your meta-data brick is not smaller than 1/200 of maximal volume size. For example, 100G meta-data brick will be able to service ~20T logical volume.

Data and meta-data bricks don't differ from the standpoint of disk format, and there is no special option to inform mkfs utility that we want to create exactly meta-data brick: the first brick in the volume automatically becomes a meta-data brick, and other bricks are interpreted as data bricks.

Mount your initial logical volume consisting of one meta-data brick:

# mount /dev/vdb1 /mnt

Find a record about your volume in the output of the following command:

# volume.reiser4 -l

Create configuration of your logical volume (its definition is above) and store it somewhere, but not on that volume!

Your logical volume is now on-line and ready to use. You can perform regular file operations and volume operations (e.g. add a data brick to your LV).

[edit] Adding a data brick to LV

At any time you are able to add a data brick to your LV. You can do it in parallel with regular file operations executing on this volume. Make sure, however, that your volume is not marked as "volume with incomplete brick removal", or there is no other volume operations over your volume in progress. Otherwise your operation will fail with EBUSY.

Obviously, adding a brick will increase capacity of your volume.

Choose a block device for the new data brick. Make sure that it is not too large, or too small. Capacities of any 2 bricks of the same logical volume can not differ more than 2^19 (~1 million) times. E.g. your logical volume can not contain both, 1M and 2T bricks. Any attempts to add a brick of improper capacity will fail with error.

Format it with the same volume ID and stripe size, as you used for meta-data brick, but specify also "-a" option (to not restrict data capacity).

# mkfs.reiser4 -U $VOL_ID -t $STRIPE -a /dev/vdb2

It is important that data brick is formatted with the same volume ID and stripe size, as the meta-data brick of your logical volume. Otherwise, operation of adding a data brick will fail.

Update item #4 of your volume configuration with UUID or name of the brick you want to add.

To add a brick simply pass its name as an argument for the option "-a" and specify your LV via its mount point:

# volume.reiser4 -a /dev/vdb2 /mnt

By default operation of adding a brick is fast and atomic and leaves the volume in unbalanced state, so after adding a brick you might want to run a balancing procedure, which will move a portion of data to the new brick from other bricks of the logical volume, which will make data distribution on your volume fair:

# volume.reiser4 -b /mnt

Portion of data blocks, being moved during such rebalancing, is equal to relative capacity of the new brick, that is to the portion of capacity that the new brick adds to updated LV's capacity. This important property defines the cost of balancing procedure. If the portion of capacity added by a brick is small, then number of stripes moved during balancing is also small.

Using this operation in conjunction with the option -B (--with-balance) will automatically trigger the balancing procedure:

# volume.reiser4 -Ba /dev/vdb2 /mnt
 

Upon successful completion update your volume configuration. That is, increment (#2), add info about the new brick to (#3) and remove records at (#4).

When adding more than one brick at once, call volume.reiser4 with option -a for each brick individually in any order. It will be reasonable to not complete each call with balancing. Run balancing only after adding the last brick.

[edit] Removing a data brick from LV

At any time you are able to remove any data brick from your LV. You can do it in parallel with regular file operations executing on that volume. Make sure, however, that your volume is not marked as "volume with incomplete brick removal", or there is no other volume operations over your volume in progress. Otherwise your operation will fail with EBUSY.

Obviously, the removal operation will decrease abstract capacity of your LV. Note that other bricks should have enough space to store all data blocks of the brick you want to remove, otherwise, the removal operation will return error (ENOSPC).

Suppose you want to remove brick /dev/vdb2 from your LV mounted at /mnt.

Update your volume configuration with the UUID and name of the brick you want to remove (#item #4).

To remove a brick simply pass its name as an argument for option "-r" and specify the logical volume by its mount point:

# volume.reiser4 -r /dev/vdb2 /mnt

The procedure of brick removal starts from moving all data from the brick you want to remove to other bricks of your volume, so that resulted data distribution among the rest of bricks will be also fair.

Portion of data stripes being moved during such migration is equal to the relative capacity of the brick to be removed (that it to the portion of capacity that the brick added to LV's capacity). Successful brick removal always leaves the volume is balanced state.

So, in contrast with the operation of adding a brick, removing a brick is a rather long operation, which can be interrupted for various reasons. In this case volume will be marked as a "volume with incomplete brick removal".

To check removal status of your LV simply run

# volume.reiser4 /mnt

and check the field "health".

To complete brick removal in the current mount session simply run

# volume.reiser4 -R /mnt

Note, that the option -R (--finish-removal) doesn't accept any arguments.

On success update your volume configuration: remove the information about the brick /dev/vdb2 at #3 and #4. Check your kernel logs: it should contain a message that brick /dev/vdb2 has been unregistered. Now device /dev/vdb2 doesn't belong to the logical volume any more, and you can reuse it for other purposes (re-format, etc).

[edit] Changing brick's capacity

At any time you can change abstract capacity of any brick to some new value, different from 0. You can do it in parallel with regular file operations executing on that volume. Make sure, however, that your volume is not marked as "volume with incomplete brick removal", or there is no other volume operations over your volume in progress. Otherwise your operation will fail with EBUSY.

Changing capacity always changes volume partitioning, and therefore, breaks fairness of data distribution on the volume. By default operation of changing brick capacity leaves the volume in unbalanced state, so after changing brick capacity you might want to run a balancing procedure to make data distribution on your volume fair. In particular, after increasing brick capacity the balancing procedure will move some data from other bricks to the brick, whose capacity was increased. After decreasing bricks capacity the balancing procedure will move some data from the brick, whose capacity was decreased, to other bricks.

To change abstract capacity of a brick /dev/vdb1 to a new value (e.g. 200000), simply run

# volume.reiser4 -z /dev/vdb1 -c 200000 /mnt

Pronounced as "resize brick /dev/vdb1 to new capacity 200000 in volume mounted at /mnt".

By default the operation of changing capacity is fast, atomic and leave the volume in unbalanced state. To automatically invoke balancing, use this operation in conjunction with the option -B (--with-balance). Also you can run a balancing procedure later at any time by executing

# volume.reiser4 -b /mnt

When changing capacities of more than one brick at once, call the volume.reiser4 utility for each brick individually in any order. It will be reasonable to not complete each call with balancing. Run balancing after changing capacity of the last brick.

Comment. Changing bricks capacity to 0 is undefined and will return error. Consider brick removal operation instead.

[edit] Operations with meta-data brick

Meta-data brick also can contain data stripes and participate in data distribution like other data bricks. All the volume operations described above are also applicable to meta-data brick. Note, however, that it is impossible to completely remove meta-data brick from the logical volume for obvious reasons (meta-data need to be stored somewhere), so brick removal operation applied to the meta-data brick actually removes it only from Data-Storage Array (DSA), which is a subset of LV consisting of bricks, participating in regular data distribution, corresponding to their abstract capacities.

Once you remove meta-data brick from DSA, that brick will be used only to store meta-data. Operation of adding a brick, being applied to a meta-data brick, returns the last one back to DSA.

Important: Reiser5 doesn't count busy data and meta-data blocks separately. So in contrast with data bricks (which contain only data) you are not able to find out real space occupied by data blocks on the meta-data brick - Reiser5 knows only total space occupied.

To check the status of meta-data brick of the volume mounted at /mnt simply run

# volume.reiser4 -p0 /mnt

and check the field "in DSA".

[edit] Unmounting a logical volume

To terminate a mount session just issue usual umount against the mount point:

# umount /mnt

Note that after unmounting the volume all bricks by default remain to be registered in the system till system shutdown. If you want to unregister a brick before system shutdown, then simply issue the following command:

# volume.reiser4 -u BRICK_NAME

[edit] Deploying a logical volume after correct unmount

After unmounting a logical volume all its bricks remain to be registered in the system. So, if you want to mount the volume again, simply issue the mount command against some its brick. is recommended to issue it against meta-data brick.

NOTE: Reiser5 will refuse to mount a logical volume, in the case, when a wrong (incomplete or redundant) set of bricks is registered in the system. Redundant set of bricks appears, for example, when you mistakenly register a brick that was earlier removed from the logical volume.

[edit] Deploying a logical volume after correct shutdown

First of all, check configuration of your volume and make sure that all its bricks (data and meta-data ones) are registered in the system. The list of registered bricks can be printed by

# volume.reiser4 -l

Also make sure that the set of registered per volume bricks doesn't contain bricks not mentioned in the volume configuration.

Important: Reiser5 will refuse to mount a logical volume, in the case, when a wrong (incomplete or redundant) set of bricks is registered in the system. Redundant set of bricks appears, for example, when you mistakenly register a brick that was removed from the logical volume. For this reasons we strongly recommend for user to keep a track of his LV - store its configuration somewhere, but not in this volume! And don't forget to update that configuration after every volume operation. If you lost configuration of your LV and don't remember it (wich is most likely for large volumes), then it will be rather painful to restore it: currently there is no tools for to manage logical volumes off-line. So that, users are prompted to do this on their own. It is not at all difficult.

To register a brick in the system use the following command:

# volume.reiser4 -g BRICK_NAME

To print a list of all registered bricks use

# volume.reiser4 -l

Now mount your LV, simply issuing a mount(8) command against one of the bricks of your LV. We recommend to issue it against meta-data brick.

Comment. Reiser5 always tries to register the brick which is passed to the mount command as an argument, so it is not necessarily to preregister the brick you want to issue a mount command against.

[edit] Deploying a logical volume after hard reset or system crash

If no volume operations were interrupted by hard reset or system crash, then just follow the instructions in this section.

In Reiser5 only restricted number of bricks participate in every transaction. Maximal number of such bricks can be specified by user. At mount time a transaction replay procedure will be launched on each such brick independently in parallel.

Depending on a kind of interrupted volume operation, perform one of the following actions:


[edit] Volume balancing was interrupted

Check your volume configuration.

Register the complete set of bricks and mount the volume by issuing the mount command against some its brick.

Check the balanced status of your LV by running

# volume.reiser4 /mnt

and checking "balanced" value. If the volume is unbalanced, then complete balancing by running

# volume.reiser4 -b /mnt

[edit] Brick removal was interrupted

Check your volume configuration. Register the new set of bricks (that is, the set of bricks without the brick you wanted to remove). Try to mount the volume. In the case of error register also the brick you wanted to remove and try to mount again.

Check the status of your LV by running

# volume.reiser4 /mnt

and checking the value of "health". If required, complete brick removal by running

# volume.reiser4 -R /mnt

Note, that the option -R doesn't accept any arguments. After successful removal completion the brick will be automatically removed from the volume and unregistered. Make sure of it by checking status of your LV and the list of registered bricks:

# volume.reiser4 /mnt
# volume.reiser4 -l

Upon successful completion update your volume configuration respectively.

[edit] LV monitoring

Common info about LV mounted at /mnt

# volume.reiser4 /mnt
ID:		Volume UUID
volume:		ID of plugin managing the volume
distribution:	ID of distribution plugin
stripe:		Stripe size in bytes
segments:	Number of hash space segments (for distribution)
bricks total:	Total number of bricks in the volume
bricks in DSA:	Number of bricks participating in data distribution
balanced:	Balanced status of the volume
health:		Brick removal completion status


Info about any its brick of index J

# volume.reiser4 -p J /mnt
internal ID:	Brick's "internal ID" and its status in the volume
external ID:	Brick's UUID
device name:	Name of the block device associated with the brick
block count:	Size of the block device in blocks
blocks used:	Total number of occupied blocks on the device
system blocks:	Minimal possible number of busy blocks on that device
data capacity:	Abstract capacity of the brick
space usage:	Portion of occupied blocks on the device
in DSA:		Participation in regular data distribution
is proxy:	Participation in data tiering (Burst Buffers, etc)


Comment. When retrieving brick's info make sure that no volume operations over that volume are in progress. Otherwise the command above will return error (EBUSY).

WARNING. Bricks info provided by such way is not necessarily the most recent one. To get an actual info run sync(1) and make sure that no regular file operations are in progress.

[edit] Checking free space

To check number of available free blocks on a volume mounted at /mnt, make sure that no regular file operations, as well as volume operations, are in progress on that volume, then run

# sync
# df --block-size=4K /mnt

To check number of free blocks on the brick of index J run

# volume.reiser4 -p J /mnt

Then calculate the difference between block count and blocks used

Comment. Not all free blocks on a brick/volume are available for use. Number of available free blocks is always ~95% of total number of free blocks (Reiser4 reserves 5% to make sure that regular file truncate operations won't fail).

NOTE: volume.reiser4 shows total number of free blocks, whereas df(1) shows number of available free blocks.

"Space usage" statistics shows a portion of busy blocks on individual brick. For the reasons explained above "space usage" on any brick can not be more than 0.95


[edit] Checking quality of data distribution

Quality of data distribution is a measure of deviation of the real data space usage from the ideal one defined by volume partitioning. The smaller the deviation, the better the distribution quality.

Checking quality of distribution makes sense only in the case when your volume partitioning is space-based, or if it coincides with the space-based one.

If your partitioning is throughput-based, and it doesn't coincide with the space-based one, then quality of actual data distribution can be rather bad, as in this case the file system is worried for low-performance devices to not become a bottleneck, and effective space usage in this case is not a high priority.

Checking quality of data distribution is based on the free blocks accounting, provided by the file system. Note that file system doesn't count busy data and meta-data blocks separately, so you are not able to find real data space usage, and hence to check quality of distribution in the case when meta-data brick contains data blocks.

To check quality of distribution

  • make sure that meta-data brick doesn't contain data blocks;
  • make sure that no regular file and volume operations are currently in progress;
  • find "blocks used", "system blocks" and "data capacity" statistics for each data brick:
# sync
# volume.reiser4 -p 1 /mnt
...
# volume.reiser4 -p N /mnt
  • find real data space usage on each brick;
  • calculate partitioning and ideal data space usage on each data brick;
  • find deviation of (4) from (5).

Example.

Let' build a LV of 3 bricks (one 10G meta-data brick sdb1, and two data bricks: sdc1 (10G), sdd1(5G)) with space-based partitioning:

# VOL_ID=`uuid -v4`
# echo "Using uuid $VOL_ID"

# mkfs.reiser4 -U $VOL_ID -y    -t 256K /dev/vdb1
# mkfs.reiser4 -U $VOL_ID -y -a -t 256K /dev/vdc1
# mkfs.reiser4 -U $VOL_ID -y -a -t 256K /dev/vdd1

# mount /dev/vdb1 /mnt

Fill the meta-data brick with data:

# dd if=/dev/zero of=/mnt/myfile bs=256K

No space left on device...

Add data-bricks /dev/sdc1 and dev/sdd1 to the volume:

# volume.reiser4 -a /dev/vdc1 /mnt
# volume.reiser4 -a /dev/vdd1 /mnt

Move all data blocks to the newly added bricks:

# volume.reiser4 -r /dev/vdb1 /mnt
# sync

Now meta-data brick doesn't contain data blocks (only meta-data ones), so that we can calculate quality of data distribution

# volume.reiser4 /mnt -p0
blocks used: 503
# volume.reiser4 /mnt -p1
blocks used: 1657203
system blocks: 115
data capacity: 2621069
# volume.reiser4 /mnt -p2
blocks used: 833001
system blocks: 73
data capacity: 1310391

Basing on the statistics above calculate quality of distribution.

Total data capacity of the volume:

C = 2621069 + 1310391 = 3931460

Relative capacities of data bricks:

C1 = 2621069 /(2621069 + 1310391) = 0.6667
C2 = 1310464 /(2621069 + 1310391) = 0.3333

Real space usage on data bricks (blocks used - system blocks):

R1 = 1657203 - 115 = 1657088
R2 = 833001 - 73 = 832928

Space usage on the volume:

R = R1 + R2 = 1657088 + 832928 = 2490016

Ideal data space usage on data bricks:

I1 = C1 * R = 0.6667 * 2490016 = 1660094
I2 = C2 * R = 0.3333 * 2490016 = 829922

Deviation:

D = (R1, R2) - (I1, I2) = (3006, -3006)

Relative deviation:

D/R = (-0.0012, 0.0012)

Quality of distribution:

Q = 1 - max(|D1|, |D1|) = 1 - 0.0012 = 0.9988

Comment. For any specified number of bricks N and quality of distribution Q it is possible to find a configuration of a logical volume composed of N bricks, so that quality of distribution on that volume will be better than Q.

Comment. Quality of distribution Q doesn't depend on the number of bricks in the logical volume. This is a theorem, which can be strictly proven.

[edit] FAQ

What happens if I lose a brick (due to a breakdown, etc) of my logical volume?

If you lose a meta-data brick, then you lose the whole volume. If you loose a data brick, then you'll be able to mount the volume, but bodies of some your regular files will become "punched" in random places. Portion of such files depends on the relative capacity of the lost brick, on the number of bricks in the logical volume, and on other factors. Fsck will be able to detect and remove such files with corrupted bodies. Nevertheless, we recommend to consider mirroring your bricks (e.g. by software, or hardware RAID-1) to avoid such highly unpleasant situations.

I am not able to complete brick removal because of "no space left on device". What should I do?

At the very beginning of removal operation the file system calculates precise amount of space on other bricks, needed to accommodate all the data of the brick you are going to remove. It could happen, however that free space on the bricks was near the calculated amount, and during the removal, exceeded it due to intensive regular file operations (caused e.g. by torrents activity) going in parallel. So, to complete the operation consider removing a part of regular files on your partition. In the future it will be possible to add a brick to a volume marked as a "volume with incomplete brick removal". It will also allow to complete removal operation.

Personal tools