... | ... | @@ -123,7 +123,15 @@ ln -s /z/groups /groups |
|
|
|
|
|
### Using ZFS
|
|
|
|
|
|
If you have multiple physical disks available for Emulab storage and they are not combined using hardware RAID, then you can use ZFS to create RAID1, RAID10, RAID5, RAID50, RAID6 or RAID60 style configurations. For example, if you have two extra disks, /dev/ada1 and /dev/ada2, you can create a mirrored zpool with:
|
|
|
If you have multiple physical disks available for Emulab storage and they are not combined using hardware RAID, then you can use ZFS to create RAID1, RAID10, RAID5, RAID50, RAID6 or RAID60 style configurations. Using ZFS permits other nice features as well, including easy creation and resizing of filesystems and the ability to create per-filesystem snapshots for backup purposes.
|
|
|
|
|
|
In the simplest form, ZFS can be a drop in replacement for UFS by creating single filesystems for the various Emulab file hierarchies. It should be noted however that Emulab software does not currently have the ability to manage UFS-style per-user quotas within single ZFS filesystems, so it cannot currently be used for `/users`, `/proj`, or `/groups` when such quotas are desired. It does however work fine for the `/usr/testbed` and `/share` hierarchies.
|
|
|
|
|
|
The primary use of ZFS in Emulab is to implement individual per-user and per-project filesystems. This enables disk quotas by limiting the amount of storage each filesystem can use, regardless of what user is doing the writing. This style of quota avoids one of the biggest problems with UFS quotas--the inability to place a quota on the "root" user, thereby allowing for example, a user running as root to fill up the entire `/users` filesystem.
|
|
|
|
|
|
ZFS can still be used in a configuration with only a single extra disk, built on top of a HW RAID volume for example. In this way you still gain all the advantages of the ZFS filesystem except for its providing redundancy. It should be noted however that "stacking" volume managers in this way can mask errors and lead to bad performance.
|
|
|
|
|
|
To use ZFS for your install, you first create a zpool. For example, if you have two extra disks, /dev/ada1 and /dev/ada2, you can create a mirrored zpool with:
|
|
|
```
|
|
|
zpool create -m none z mirror /dev/ada1 /dev/ada2
|
|
|
```
|
... | ... | @@ -135,9 +143,26 @@ or a RAID10 striped mirror: |
|
|
```
|
|
|
zpool create -m none z mirror /dev/ada1 /dev/ada2 mirror /dev/ada3 /dev/ada4
|
|
|
```
|
|
|
To create a single disk zpool you would:
|
|
|
```
|
|
|
zpool create -m none z /dev/ada1
|
|
|
```
|
|
|
Once the zpool is created, you create filesystems for the various hierarchies. For example, for a zpool `z` composed of a mirror of two 1TB drives:
|
|
|
```
|
|
|
# 50G for /usr/testbed
|
|
|
zfs create -o mountpoint=/usr/testbed -o quota=50G z/testbed
|
|
|
# 10G for /share
|
|
|
zfs create -o mountpoint=/share -o quota=10G -o setuid=off z/share
|
|
|
# 200G for /users
|
|
|
zfs create -o mountpoint=/users -o quota=200G -o setuid=off z/users
|
|
|
# 600G for /proj
|
|
|
zfs create -o mountpoint=/proj -o quota=500G -o setuid=off z/proj
|
|
|
# 100G for /groups
|
|
|
zfs create -o mountpoint=/groups -o quota=100G -o setuid=off z/groups
|
|
|
```
|
|
|
A couple of important points about these newly created filesystems. First note that we appear to be creating a single filesystem for `/users`, etc., but these filesystem will actually be the parents of the per-user and per-project filesystems that will be created underneath. These parent filesystems serve as a source of inherited filesystem attributes such as the mount point and the disabling of setuid files. In this way the attributes don't have to be specified for every child filesystem. Second we are setting a quota on the parent filesystems. This limits the total disk space that can be used by all child filesystems and will thus prevent, e.g., `/users` filesystems from filling up all the space in the zpool. These quotas can be changed dynamically as needed or removed entirely if you are not worried about limiting the distribution of space between the file hierarchies.
|
|
|
|
|
|
Using ZFS permits a number of nice features, Most notably highly reliable, large scale filesystems and the ability to create per-filesystem snapshots. Emulab can leverage ZFS to implement per-user and per-project filesystems. This enables fine grained disk quotas and avoids one of the biggest problems with UFS quotas--the inability to place a quota on the "root" user.
|
|
|
|
|
|
The Emulab software is responsible for setting the quotas (if any) on individual user or project filesystems. Those limits are specified in the definitions file as described in the next section.
|
|
|
|
|
|
## Updating the Definitions file
|
|
|
|
... | ... | |