... | ... | @@ -6,57 +6,58 @@ Boot the FreeBSD installation CD on the machine you will use as your `ops` serve |
|
|
|
|
|
Next it will ask you about disks and disk partitioning. There are a variety of ways to partition disk space depending on how many disks the server has and whether you might want to enforce disk space quotas. For now we just choose the first disk, probably /dev/ada0, and create a single partition in which to install the base system; other partitions will be created as needed later.
|
|
|
|
|
|
From the menu choose "Auto (UFS)", and then "Entire Disk", confirm erasing the disk and then choose either MBR or GPT. If your disk is over 2TB and the server BIOS supports UEFI boot, select GPT. Otherwise it is probably easiest to stick with MBR and a "legacy boot" BIOS setting. At this point it will want you to review the partitioning. If you are dedicating the entire disk to the OS, then just select "Finish". If you are using a single disk for everything, then you will need to shrink the FreeBSD partition to leave for Emulab. Unfortunately, doing this requires you "Delete" the existing "freebsd-ufs" and "freebsd-swap" partitions and "Create" new ones. The "freebsd-ufs" partition should be 50-100GB and should have a mountpoint of "/". The swap partition you can make 16GB or smaller. Once you have recreated these partitions select "Finish".
|
|
|
From the menu choose "Auto (UFS)", and then "Entire Disk", confirm erasing the disk and then choose either MBR or GPT. If your disk is over 2TB and the server BIOS supports UEFI boot, select GPT. Otherwise it is probably easiest to stick with MBR and a "legacy boot" BIOS setting.
|
|
|
|
|
|
Next it will put you in the partition editor and want you to review the partitioning. If you are dedicating the entire disk to the OS, then just select "Finish". If you are using a single disk for OS and the Emulab install, then you will need to shrink the FreeBSD partition to leave room for Emulab. Unfortunately, doing this requires you "Delete" the existing "freebsd-ufs" and "freebsd-swap" partitions and "Create" new ones. They should be created under the "BSD" partition (probably ada0s1). The "freebsd-ufs" partition should be 50-100GB and should have a mountpoint of "/". The swap partition you can make 16GB or smaller. Once you have recreated these partitions select "Finish".
|
|
|
|
|
|
Now it will prompt you to "Commit" the changes. After doing so, it will begin the install.
|
|
|
Near the end of the installation, you will be asked if you wish to install or configure several optional items. Apart from setting the root password, configuring your network devices, configuring DNS, setting your time and time zone, and enabling SSH logins (if desired), you should skip additional configuration.
|
|
|
Near the end of the installation, you will be asked if you wish to install or configure several optional items. Do set the root password, configure your network device, configure DNS, set your time and time zone, and enable SSH logins (if desired). You should skip any other configuration.
|
|
|
|
|
|
Do **not** create any user accounts yet, and just log in as root for the time being. Our software will create user accounts later, once you get boss set up correctly. If you already created any users, delete them with the "pw" command and make sure their home directories are removed as well!
|
|
|
|
|
|
After exiting the sysinstall tool and rebooting, you should login as root and run freebsd-update to pick up security patches:
|
|
|
Installation will complete and it will ask if you want to reboot the machine. Do so, and after rebooting, login as root and run `freebsd-update` to pick up security patches (**NOTE:** If you are installing FreeBSD 11.3 skip this step since 11.3 has reached end-of-life and there are no updates. This step will be needed when we transition to FreeBSD 12.2.):
|
|
|
```
|
|
|
freebsd-update fetch
|
|
|
freebsd-update install
|
|
|
```
|
|
|
**NOTE:** When installing FreeBSD 11.3 it will tell you that it has reached end-of-life and
|
|
|
do nothing. This step will be needed when we transition to FreeBSD 12.2.
|
|
|
|
|
|
|
|
|
## Create Emulab Partitions
|
|
|
|
|
|
Now you need to create the needed partitions and filesystem for the Emulab-required directory hierarchies listed below. Note that the example sizes in square brackets are for a 1TB single disk.
|
|
|
Now you need to create the partitions and filesystem for the Emulab-required directory hierarchies described below. Note that the example sizes in square brackets are for a 1TB single disk, you can adjust accordingly based on your available space.
|
|
|
|
|
|
* **/usr/testbed/** Space for testbed software and logs, at least 10GB. [50GB]
|
|
|
* **/usr/testbed/** Space for testbed software and logs, allocate at least 10GB but no larger than 100GB. [50GB]
|
|
|
|
|
|
* **/share/** Exported read-only to all nodes. We use it for providing experimenters with the source for the FreeBSD and Linux versions we run as well as common packages. This could require anything from 1GB to 100s of GBs depending on what you want to make available. This directory must exist as a distinct filesystem, but most sites don't use it and you can likely keep it small. [10GB]
|
|
|
* **/share/** Space for globally shared files. Exported read-only to all nodes where it is mounted as `/share`. We use this for providing experimenters with the source for the FreeBSD and Linux versions we run as well as common packages. This could require anything from 1GB to 100s of GBs depending on what you want to make available. This directory must exist as a distinct filesystem, but most sites don't use it and you can likely keep it small. [10GB]
|
|
|
|
|
|
* **/users/** Needs space for user home directories. The amount of space required depends on how many users you expect to have. Generally, though, we suggest that users store large files related to their projects in the `/proj` directory. This could range from 10GB to 100s of GB. [200GB]
|
|
|
* **/users/** Space for user home directories. The amount of space required depends on how many users you expect to have. Generally, we encourage users to only put user configuration related files here and suggest they use `/proj` space for storing large files related to their experimentation. This could range from 10GB to 100s of GB. [200GB]
|
|
|
|
|
|
* **/proj/** Needs space for project files. We recommend that this be larger than `/users`, to encourage people to store files here, which aids per-project accountability. The bulk of your available space should go here. [500GB]
|
|
|
* **/proj/** Space for project-related files. We recommend that this be significantly larger than `/users`, to encourage people to store files here, which aids per-project accountability. The bulk of your available space should go here. [500GB]
|
|
|
|
|
|
* **/groups/** Needs enough space for files shared by the sub-groups of projects. Subgroups allow for private storage within a project and are primarily used for group projects in classes. Thus not much space, maybe 1GB, is needed unless you plan to support such subgroups. [100GB]
|
|
|
* **/groups/** Space for files shared by the sub-groups of projects. Subgroups allow for private storage within a project and are primarily used for "group projects" in classes. If you plan to use this mechanism, you should again allocate from 10GB to 100s of GB. If not, maybe 1-10GB. [100GB]
|
|
|
|
|
|
Often `/users`, `/proj`, and `/groups` are just subdirectories on the same filesystem. This allows you to avoid decisions about how much space to put where. Just put all remaining space in one filesystem and use it for all three.
|
|
|
Because of the vague storage requirements, `/users`, `/proj`, and `/groups` are often just subdirectories on the same filesystem. This allows you to avoid hard decisions about how much space to allocate to each. Just put all remaining space in one filesystem and use it for all three.
|
|
|
|
|
|
In fact, the only reasons you might want to make them separate filesystems would be to prevent one of the hierarchies from filling up the entire disk or if you want to enforce distinct quotas for a user in the `users` and `proj` filesystems.
|
|
|
|
|
|
Note that since `/share` is exported read-only, FreeBSD requires that it be on a separate filesystem from anything that is exported read-write. So while `/users, /proj` and `/groups` can be on the same filesystem, `/share` cannot.
|
|
|
**Note:** since `/share` is exported read-only, FreeBSD requires that it be on a separate filesystem from anything that is exported read-write. So while `/users, /proj` and `/groups` can be on the same filesystem, `/share` cannot.
|
|
|
|
|
|
### Using UFS
|
|
|
|
|
|
The traditional UFS filesystem is best used with a single redundant underlying volume, either
|
|
|
a hardware RAID provided volume, or a virtual disk provided by the VM host. While you can use `gvinum` or `graid` to implement RAID or interact with a software RAID controller, it is recommended that you [use ZFS instead](#using-zfs) if you need to build a multi-disk redundant configuration.
|
|
|
a hardware RAID provided volume, or a virtual disk provided by the VM host. Otherwise you are taking your chances with a disk failure. While you can use FreeBSD's `gvinum` or `graid` to implement RAID or interact with a software RAID controller, it is recommended that you [use ZFS instead](#using-zfs) if you need to build a multi-disk redundant configuration.
|
|
|
|
|
|
Here are two examples of configuring partitions and filesystems for a system with a large single disk, or with a second disk dedicated to Emulab bits.
|
|
|
Here are two examples of configuring partitions and filesystems on a single disk, either for a system with a large single disk shared by both the OS and Emulab, or with a second disk dedicated to Emulab bits.
|
|
|
|
|
|
#### Single Disk
|
|
|
#### Shared Single Disk
|
|
|
|
|
|
This example creates separate filesystems for `/usr/testbed` and `/share`, and a `/z` filesystem for the remaining directory hierarchies. It assumes you have modified the FreeBSD install and swap partitions during installation to leave space for these filesystems.
|
|
|
This example creates separate filesystems for `/usr/testbed` and `/share`, and a `/z` filesystem for the remaining directory hierarchies. It assumes you have modified the FreeBSD install and swap partitions during installation to leave space for these filesystems. You can use `gpart show` to determine what device ("geom") to specify, use the one with the most free space. It will most likely be "ada0s1" as is shown in the following.
|
|
|
```
|
|
|
# 50G for /usr/testbed
|
|
|
gpart add -t freebsd-ufs -s 50G ada0s1
|
|
|
# 10G for /share
|
|
|
gpart add -t freebsd-ufs -s 10G ada0s1
|
|
|
# the rest for /z (/users, /proj, /groups)
|
|
|
# the rest (implied by no size argument) for /z (/users, /proj, /groups)
|
|
|
gpart add -t freebsd-ufs ada0s1
|
|
|
|
|
|
# create the filesystems and mountpoints
|
... | ... | @@ -83,13 +84,13 @@ ln -s /z/groups /groups |
|
|
```
|
|
|
If you wanted distinct filesystems for each of `/users`, `/proj`, and `/groups`, then instead of the commands to create `/z`, you would have analogous commands for those three filesystems. You would obviously not need to create the symlinks as shown in the last command group.
|
|
|
|
|
|
#### Second Disk
|
|
|
#### Second Dedicated Disk
|
|
|
|
|
|
If you have a second disk (/dev/ada1) available for just Emulab, then the process is similar, but the disk first has to be prepared. We use GPT partitioning in this case to avoid the need for extended-DOS or FreeBSD partition tables inside of MBR tables (MBR supports only four primary partitions).
|
|
|
If you have a second disk (/dev/ada1) available for just Emulab, then the process is similar, but the disk first has to be prepared. We use GPT partitioning in this case to avoid the need for extended-DOS or FreeBSD partition tables inside of MBR tables (MBR supports only four primary partitions and we need more).
|
|
|
```
|
|
|
gpart create -s GPT /dev/ada1
|
|
|
```
|
|
|
The remaining steps are very similar to the above, but note the change in device names.
|
|
|
The remaining steps are very similar to the above, but note the change in device names from the previous example.
|
|
|
```
|
|
|
# 50G for /usr/testbed
|
|
|
gpart add -t freebsd-ufs -s 50G ada1
|
... | ... | |