ZFS 基础教程
Usage
ZFS includes already all programs to manage the hardware and the file systems, there are no additional tools needed.
Preparation
ZFS supports the use of either block devices or files. Administration is the same in both cases, but for production use, the ZFS developers recommend the use of block devices (preferably whole disks). To take full advantage of block devices on Advanced Format disks, it is highly recommended to read the ZFS on Linux FAQ before creating your pool. To go through the different commands and scenarios we can use files in place of block devices.
The following commands create 2GB sparse image files in /var/lib/zfs_img/ that we use as our hard drives. This uses at most 8GB disk space, but in practice will use very little because only written areas are allocated:
root #mkdir /var/lib/zfs_imgroot #truncate -s 2G /var/lib/zfs_img/zfs0.imgroot #truncate -s 2G /var/lib/zfs_img/zfs1.imgroot #truncate -s 2G /var/lib/zfs_img/zfs2.imgroot #truncate -s 2G /var/lib/zfs_img/zfs3.img
Now we check which loopback devices are in use:
Note
On pool export, all of the files will be released and the folder /var/lib/zfs_img can be deleted
zfs /dev/disk/by-id 一定要用这个，要不改换数据线时sda1，sdb1的位置会调换，这时zfs就会找不到磁盘Zpools
The program /usr/sbin/zpool is used with any operation regarding zpools.
import/export Zpool
To export (unmount) an existing zpool named zfs_test into the file system, you can use the following command:
root #zpool export zfs_test
To import (mount) the zpool named zfs_test use this command:
root #zpool import zfs_test
The root mountpoint of zfs_test is a property and can be changed the same way as for volumes. To import (mount) the zpool named zfs_test root on /mnt/gentoo, use this command:
root #zpool import -R /mnt/gentoo zfs_test
Note
ZFS will automatically search on the hard drives for the zpool named zfs_test
ZFS will automatically search on the hard drives for the zpool named zfs_test
To search for all zpools available in the system issue the command:
root #zpool importOne hard drive
Create a new zpool named zfs_test with one hard drive:
root #zpool create zfs_test /var/lib/zfs_img/zfs0.img
The zpool will automatically be mounted, default is the root file system aka /zfs_test
root #zpool status
To delete a zpool use this command:
root #zpool destroy zfs_test
Important
ZFS will not ask if you are sure.
ZFS will not ask if you are sure.
Two hard drives (MIRROR)
In ZFS you can have several hard drives in a MIRROR, where equal copies exist on each storage. This increases the performance and redundancy. To create a new zpool named zfs_test with two hard drives as MIRROR:
root #zpool create zfs_test mirror /var/lib/zfs_img/zfs0.img /var/lib/zfs_img/zfs1.img
Note
of the two hard drives only 2GB are effective useable so total_space * 1/n
of the two hard drives only 2GB are effective useable so total_space * 1/n
root #zpool status
To delete the zpool:
root #zpool destroy zfs_testThree hard drives (RAIDZ1)
RAIDZ1 is the equivalent to RAID5, where data is written to the first two drives and a parity onto the third. You need at least three hard drives, one can fail and the zpool is still ONLINE but the faulty drive should be replaced as soon as possible.
To create a pool with RAIDZ1 and three hard drives:
root #zpool create zfs_test raidz1 /var/lib/zfs_img/zfs0.img /var/lib/zfs_img/zfs1.img /var/lib/zfs_img/zfs2.img
Note
of the three hard drives only 4GB are effective useable so total_space * (1-1/n)
of the three hard drives only 4GB are effective useable so total_space * (1-1/n)
root #zpool status
To delete the zpool:
root #zpool destroy zfs_testFour hard drives (RAIDZ2)
RAIDZ2 is the equivalent to RAID6, where data is written to the first two drives and a parity onto the next two. You need at least four hard drives, two can fail and the zpool is still ONLINE but the faulty drives should be replaced as soon as possible.
To create a pool with RAIDZ2 and four hard drives:
root #zpool create zfs_test raidz2 /var/lib/zfs_img/zfs0.img /var/lib/zfs_img/zfs1.img /var/lib/zfs_img/zfs2.img /var/lib/zfs_img/zfs3.img
Note
of the four hard drives only 4GB are effective useable so total_space * (1-2/n)
of the four hard drives only 4GB are effective useable so total_space * (1-2/n)
root #zpool status
To delete the zpool:
root #zpool destroy zfs_testFour hard drives (STRIPED MIRROR)
A STRIPED MIRROR is the equivalent to RAID10, where data is striped across sets of disks then the striped data is mirrored. You need at least four hard drives; this configuration provides redundancy and an increase in read speed. You can lose all disks but one per mirror.
To create a STRIPED MIRRORED pool with four hard drives:
root #zpool create zfs_test mirror /var/lib/zfs_img/zfs0.img /var/lib/zfs_img/zfs1.img mirror /var/lib/zfs_img/zfs2.img /var/lib/zfs_img/zfs3.img
Note
of the four hard drives only 4GB are useable so total_space * (1-2/n)
of the four hard drives only 4GB are useable so total_space * (1-2/n)
root #zpool status pool: zfs_test
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
zfs_test ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
/var/lib/zfs_img/zfs0.img ONLINE 0 0 0
/var/lib/zfs_img/zfs1.img ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
/var/lib/zfs_img/zfs2.img ONLINE 0 0 0
/var/lib/zfs_img/zfs3.img ONLINE 0 0 0
errors: No known data errors
To delete the zpool:
root #zpool destroy zfs_testSpares/Replace vdev
You can add hot-spares into your zpool. In case a failure, those are already installed and available to replace faulty vdevs.
In this example, we use RAIDZ1 with three hard drives and a zpool named zfs_test:
root #zpool add zfs_test spare /var/lib/zfs_img/zfs3.imgroot #zpool status
The status of /dev/loop3 will stay AVAIL until it is set to be online, now we let /var/lib/zfs_img/zfs0.img fail:
root #zpool offline zfs_test /var/lib/zfs_img/zfs0.imgroot #zpool status pool: zfs_test
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
zfs_test ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
/var/lib/zfs_img/zfs0.img ONLINE 0 0 0
/var/lib/zfs_img/zfs1.img ONLINE 0 0 0
/var/lib/zfs_img/zfs2.img ONLINE 0 0 0
spares
/var/lib/zfs_img/zfs3.img
errors: No known data errors
We replace /var/lib/zfs_img/zfs0.img with our spare /var/lib/zfs_img/zfs3.img:
root #zpool replace zfs_test /var/lib/zfs_img/zfs0.img /var/lib/zfs_img/zfs3.imgroot #zpool status pool: zfs_test
state: ONLINE
scan: resilvered 62K in 0h0m with 0 errors on Sun Sep 1 15:41:41 2013
config:
NAME STATE READ WRITE CKSUM
zfs_test ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
spare-0 ONLINE 0 0 0
/var/lib/zfs_img/zfs0.img ONLINE 0 0 0
/var/lib/zfs_img/zfs3.img ONLINE 0 0 0
/var/lib/zfs_img/zfs1.img ONLINE 0 0 0
/var/lib/zfs_img/zfs2.img ONLINE 0 0 0
spares
/var/lib/zfs_img/zfs3.img INUSE currently in use
errors: No known data errors
The original vdev will automatically get removed asynchronously. If this is not the case, the old vdev may need to be detached with the "zpool detach" command. Later you will see it leave the zpool status output:
root #zpool status pool: zfs_test
state: ONLINE
scan: resilvered 62K in 0h0m with 0 errors on Sun Sep 1 15:41:41 2013
config:
NAME STATE READ WRITE CKSUM
zfs_test ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
/var/lib/zfs_img/zfs3.img ONLINE 0 0 0
/var/lib/zfs_img/zfs1.img ONLINE 0 0 0
/var/lib/zfs_img/zfs2.img ONLINE 0 0 0
errors: No known data errors
Note
ZFS automatically resilvered onto /var/lib/zfs_img/zfs0.img and the zpool had no downtime
ZFS automatically resilvered onto /var/lib/zfs_img/zfs0.img and the zpool had no downtime
Now we start a manual scrub:
root #zpool scrub zfs_testroot #zpool status pool: zfs_test
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Sun Sep 1 15:57:31 2013
config:
NAME STATE READ WRITE CKSUM
zfs_test ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
/var/lib/zfs_img/zfs3.img ONLINE 0 0 0
/var/lib/zfs_img/zfs1.img ONLINE 0 0 0
/var/lib/zfs_img/zfs2.img ONLINE 0 0 0
errors: No known data errors
Zpool version update
With every update of sys-fs/zfs, you are likely to also get a more recent ZFS version. Also the status of your zpools will indicate a warning that a new version is available and the zpools could be upgraded. To display the current version on a zpool:
root #zpool upgrade -vThis system supports ZFS pool feature flags.
The following features are supported:
FEAT DESCRIPTION
-------------------------------------------------------------
async_destroy (read-only compatible)
Destroy filesystems asynchronously.
empty_bpobj (read-only compatible)
Snapshots use less space.
lz4_compress
LZ4 compression algorithm support.
The following legacy versions are also supported:
VER DESCRIPTION
--- --------------------------------------------------------
1 Initial ZFS version
2 Ditto blocks (replicated metadata)
3 Hot spares and double parity RAID-Z
4 zpool history
5 Compression using the gzip algorithm
6 bootfs pool property
7 Separate intent log devices
8 Delegated administration
9 refquota and refreservation properties
10 Cache devices
11 Improved scrub performance
12 Snapshot properties
13 snapused property
14 passthrough-x aclinherit
15 user/group space accounting
16 stmf property support
17 Triple-parity RAID-Z
18 Snapshot user holds
19 Log device removal
20 Compression using zle (zero-length encoding)
21 Deduplication
22 Received properties
23 Slim ZIL
24 System attributes
25 Improved scrub stats
26 Improved snapshot deletion performance
27 Improved snapshot creation performance
28 Multiple vdev replacements
For more information on a particular version, including supported releases,
see the ZFS Administration Guide.
Warning
systems with a lower version installed will not be able to import a zpool of a higher version
systems with a lower version installed will not be able to import a zpool of a higher version
To upgrade the version of zpool zfs_test:
root #zpool upgrade zfs_test
To upgrade the version of all zpools in the system:
root #zpool upgrade -aZpool tips/tricks
- You cannot shrink a zpool and remove vdevs after its initial creation.
- It is possible to add more vdevs to a MIRROR after its initial creation. Use the following command (/dev/loop0 is the first drive in the MIRROR):
root #zpool attach zfs_test /dev/loop0 /dev/loop2- More than 9 vdevs in one RAIDZ could cause performance regression. For example it is better to use 2xRAIDZ with each five vdevs rather than 1xRAIDZ with 10 vdevs in a zpool
- RAIDZ1 and RAIDZ2 cannot be resized after intial creation (you may only add additional hot spares). You can, however, replace the hard drives with bigger ones (one at a time), e.g. replace 1T drives with 2T drives to double the available space in the zpool.
- It is possible to mix MIRROR, RAIDZ1 and RAIDZ2 in a zpool. For example to add two more vdevs in a MIRROR in a zpool with RAIDZ1 named zfs_test, use:
root #zpool add -f zfs_test mirror /dev/loop4 /dev/loop5- Note
this needs the -f option
- It is possible to restore a destroyed zpool, by reimporting it straight after the accident happened:
root #zpool import -D pool: zfs_test
id: 12744221975042547640
state: ONLINE (DESTROYED)
action: The pool can be imported using its name or numeric identifier.
Note
the option -D searches on all hard drives for existing zpools
the option -D searches on all hard drives for existing zpools
Volumes
The program /usr/sbin/zfs is used for any operation regarding volumes. To control the size of a volume you can set quota and you can reserve a certain amount of storage within a zpool. By default zpool uses the full storage size.
Create Volumes
We use our zpool zfs_test to create a new volume called volume1:
root #zfs create zfs_test/volume1
The volume will be mounted automatically as /zfs_test/volumes1/
root #zfs listMount/umount volumes
Volumes can be mounted with the following command, the mountpoint is defined by the property mountpoint of the volume:
root #zfs mount zfs_test/volume1
To unmount the volume:
root #zfs unmount zfs_test/volume1
The folder /zfs_test/volume1 stays without the volume behind it. If you write data to it and then try to mount the volume again, you will see the following error message:
CODE
cannot mount '/zfs_test/volume1': directory is not empty
Remove volumes
To remove volumes volume1 from zpool zfs_test:
root #zfs destroy zfs_test/volume1root #zfs list
Note
you cannot destroy a volume if there exist any snapshots of it
you cannot destroy a volume if there exist any snapshots of it
Properties
Properties for volumes are inherited from the zpool. So you can either change the property on the zpool for all volumes or specifically per individual volume or a mix of both.
To set a property for a volume:
root #zfs set zfs_test/volume1
To show the setting for a particular property on a volume:
root #zfs get zfs_test/volume1
Note
The properties are used on a volume e.g. compression, the higher is the version of this volume
The properties are used on a volume e.g. compression, the higher is the version of this volume
You can get a list of all properties set on any zpool with the following command:
root #zfs get all
This is a partial list of properties that can be set on either zpools or volumes, for a full list see man zfs:
| Property | Value | Function |
| quota= | 20m,none | set a quota of 20MB for the volume |
| reservation= | 20m,none | reserves 20MB for the volume within it's zpool |
| compression= | zle,gzip,on,off | uses the given compression method or the default method for compression which should be gzip |
| sharenfs= | on,off,ro,nfsoptions | shares the volume via NFS |
| exec= | on,off | controls if programs can be executed on the volume |
| setuid= | on,off | controls if SUID or GUID can be set on the volume |
| readonly= | on,off | sets read only atribute to on/off |
| atime= | on,off | update access times for files in the volume |
| dedup= | on,off | sets deduplication on or off |
| mountpoint= | none,path | sets the mountpoint for the volume below the zpool or elsewhere in the file system, a mountpoint set to none prevents the volume from being mounted |
Set mountpoint
Set the mountpoint for a volume, use the following command:
root #zfs set mountpoint=/mnt/data zfs_test/volume1
The volume will be automatically moved to /mnt/data
Activate NFS share on volume:
root #zfs set sharenfs=on zfs_test/volume2root #exportfs
Per default the volume is shared using the exportfs command in the following manner. See exportfs(8) and exports(5) for more information.
CODE sharenfs default options
/usr/sbin/exportfs -i -o sec=sys,rw,no_subtree_check,no_root_squash,mountpoint *:
Otherwise, the command is invoked with options equivalent to the contents of this property:
root #zfs set sharenfs="no_root_squash,rw=@192.168.11.0/24" zfs_test/volume2root #exportfs
To stop sharing the volume:
root #zfs set sharenfs=off zfs_test/volume2root #exportfsSnapshots
Snapshots are volumes which have no initial size and save changes made to another volume. With increasing changes between the snapshot and the original volume it grows in size.
Creating
To create a snapshot of a volume, use the following command:
root #zfs snapshot zfs_test/volume1@22082011
Note
volume1@22082011 is the full name of the snapshot, everything after the @ symbol can be any alphanumeric combination
volume1@22082011 is the full name of the snapshot, everything after the @ symbol can be any alphanumeric combination
Every time a file in volume1 changes, the old data of the file will be linked into the snapshot.
Listing
List all available snapshots:
root #zfs list -t snapshot -o name,creationRollback
To rollback a full volume to a previous state:
root #zfs rollback zfs_test/volume1@21082011
Note
if there are other snapshots in between, then you have to use the -r option. This would remove all snapshots between the one you want to rollback and the original volume
if there are other snapshots in between, then you have to use the -r option. This would remove all snapshots between the one you want to rollback and the original volume
Cloning
ZFS can clone snapshots to new volumes, so you can access the files from previous states individually:
root #zfs clone zfs_test/volume1@21082011 zfs_test/volume1_restore
In the folder /zfs_test/volume1_restore can now be worked on in the version of a previous state
Removal
Remove snapshots of a volume with the following command:
root #zfs destroy zfs_test/volume1@21082011Maintenance
Scrubbing
Start a scrubbing for zpool zfs_test:
root #zpool scrub zfs_test
Note
this might take some time and is quite I/O intensive
this might take some time and is quite I/O intensive
Log Files
To check the history of commands that were executed:
root #zpool historyMonitor I/O
Monitor I/O activity on all zpools (refreshes every 6 seconds):
root #zpool iostat 6ZFS root
To boot from a ZFS volume as the root filesystem requires a ZFS capable kernel and an initial ramdisk (initramfs) which has the ZFS userspace utilities. The easiest way to set this up is as follows.
First, make sure to have compiled a kernel with ZFS support and used
make install to copy it to /boot/ and make modules_install to make the modules available at boot time.
Install and configure genkernel.
root #emerge --ask sys-kernel/genkernelroot #genkernel initramfs --zfs
Install a bootloader, for example GRUB2.
root #emerge --ask sys-boot/grub:2
Configure grub to use ZFS, and which volume to boot from.
FILE
/etc/default/grubSpecify real_root deviceGRUB_CMDLINE_LINUX="dozfs real_root=ZFS=mypool/myvolume"
Finally, install grub to your boot device and create the grub configuration.
root #grub-mkconfig -o /boot/grub/grub.cfg
Comments