So I ran some quick dd-based write numbers before choosing a filesystem for my new
Proxmox based virtual server. I really wanted ZFS, but I need OpenVZ and KVM more, and ZFS-Fuse just does cut it.
In short - ext4 is the winner, though ZFS sure looks pretty damn fast.
SolarisInternals has more info on ZFS write speed
Another thing to note, though the below numbers for ZFS-Fuse don't reflect it, is that ZFS performs better with Write-Back cache off on your Raid controller. Had I tested OpenSolaris with Write-Back cache on (boot up was ungodly slow with it on), I'm sure it would have been reflected in the numbers.
4 600GB, RAID 5, wb on, perc 7,
ext4proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 2.53637 s,
423 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 20.3379 s,
528 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (
107 GB) copied, 192.164 s,
559 MB/s---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7,
ext3proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 4.58121 s,
234 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 37.0681 s,
290 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (
107 GB) copied, 371.047 s,
289 MB/s---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7,
reiser3proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 2.91366 s,
369 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 28.923 s,
371 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
Hmm Not sure what happened here...
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7,
reiser4proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 2.87369 s,
374 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 28.9769 s,
371 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
Hmm Not sure what happened here...
---------------------------------------------------------------------------------------
4 600GB, RAID 5,
wb on, perc 7,
zfs-fuse (defaults)proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 15.0122 s,
71.5 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 159.544 s,
67.3 MB/s100GB - didn't bother
---------------------------------------------------------------------------------------
4 600GB, RAID 5,
wb off, perc 7,
zfs-fuse (defaults)
proxmox2:/# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 23.7539 s,
45.2 MB/s---------------------------------------------------------------------------------------
4 600GB, RAID 5,
wb off, perc 7,
zfs-fuse (64k recordsize)proxmox2:/# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 21.3764 s,
50.2 MB/s---------------------------------------------------------------------------------------
4 600GB,
raidz,
wb off, perc 7,
zfs-fuseproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 19.1704 s,
56.0 MB/sproxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 187.938 s,
57.1 MB/s---------------------------------------------------------------------------------------
4 600GB,
raidz,
wb off, perc 7,
ZFS opensolaris (Nexenta)root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (
1.1 GB) copied, 1.30164 seconds,
825 MB/sroot@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (
11 GB) copied, 5.18124 seconds,
2.1 GB/sroot@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (
107 GB) copied, 45.0769 seconds,
2.4 GB/s