I swear I added a CA cert to Firefox 1.5 via Netware back in the day, but the more I think about it, the more I recall Netware's application deployment was far better than Microsoft's. In any case I've found a lot of info and some bad scripts, which I'm compiling and fixing for your benefit.
I have to add that batch file writing skills are sorely lacking these days. Wow. A lot of copy/pasting without really understanding what basic variables are.
If you're starting from scratch, you'll probably just want the ESR release of Firefox, then the CCK Wizard to add your cert and any other changes and create an XPI Extension. Then goto FrontMotion and throw it all together in an MSI.
Firefox uses it's own certificate store. The reason being is that Mozilla has stricter requirements than Microsoft does when it comes to allowing Root certificates to be deployed. Ok. I'll buy that. Mozilla also argues that Windows Update can deploy certificates without user's warning. Hrm... I don't really buy that outside of home use (where a user would have no clue ANYWAYS).
Typically in a Corporate environment updates are controlled and pushed to the end user. We're already running Microsoft software, I don't quite see any additional harm in allowing Microsoft to push new certs as well. In fact, I've been on a Security Team that was blamed for certificate issues in a large company when the Windows team held back a certificate update, and everyone's root expired :/
In any case, in Firefox we're stuck with manually updating the certificate store on every instance. At least you can use FrontMotion to install and update Firefox (and the latest Flash). Supposedly FrontMotion will install a Base64 cert if you create your own package, but that only worked once for me and while the cert was imported it wasn't trusted for anything :/
In order to distribute new/internal root certs to your users, you need to gather some tools and write a batch file.
First - certutil.zip This is the exe's and dll's needed to modify the Mozilla certifiate store
Second - create your batch file:
if not exist k:\ exit
Set CERTDIR=k:\gpinstall\certutil
REM Set CERTDIR=c:\Temp\CertImport
Set FFProfdir=%Appdata%\mozilla\firefox\profiles
DIR /A:D /B %FFProfdir% > "%Temp%\FFProfile.txt"
FOR /F "tokens=*" %%i in (%Temp%\FFProfile.txt) do (
CD /d "%FFProfDir%\%%i"
COPY cert8.db cert8.db.orig /y
%CertDir%\certutil.exe -A -n "Cert1Name" -i "%CertDir%\Cert1.crt" -t "TCu,TCu,TCu" -d .
%CertDir%\certutil.exe -A -n "Cert2Name" -i "%CertDir%\Cert2.crt" -t "TCu,TCu,TCu" -d .
)
DEL /f /q "%Temp%\FFProfile.txt"
Make sure CERTDIR is set to a mapped share location. It should contain the contents of certutil.zip, your certificate(s) and your batch file. Temporarily map that share if you need to, or get more creative and copy all the files from a share location to a local temp folder. I leave that exercise to the reader.
Make sure "Cert1.crt" refers to your actual certificate's file name, and "Cert1" is the real certificate name. If you are unsure of the certificate name, manually import the certificate into Firefox and find it under Tools, Options, Encryption.
That's it. Run your batch file, and it should import. Add it to your login script to import your root cert for everyone.
Tuesday, December 11, 2012
Thursday, November 15, 2012
Enabling Exchange subaddressing or extension addresses
Another shortcoming with Exchange - it doesn't handle subaddressing. (you know, name-this@domain.com or name+this@domain.com) So I have give all these slimeballs my 'real' email address with no way to track or filter out that particular slimeball. My solution is to rewrite the address in Qmail, and put the subaddress in the Subject before forwarding to Exchange.
Since I already front-end my Exchange server with a qmail server for SpamAssassin, enabling support for subaddressing was pretty straightfoward. You may want to follow a toaster or Life With Qmail to get a basic working install with procmail included.
Assuming you have a working install -
First, you must create a 2nd qmail install if you haven't already. The reason for this is that to keep things from being too confusing, we'll need to deliver our domain mail locally and then re-forward to the same domain. If you try and forward within the same qmail structure, you'll end up in a loop.
Again, I already have a 2nd qmail install. The reason for this is to normal allow outbound mail to have a short queuelifetime (a couple hours), but provide longer lifetimes to other domains if necessary. Here, qmail-smtpd listens on localhost, and control/smtproutes just directs that domain to localhost to inject mail into the 2nd queue.
Second, create the file structures:
/usr/local/domains/bin
/usr/local/domains/domain.com
Third, create a user. If you do not have a users/assign file, you'll need to create one in the format of:
+domain.com-:domain.com:64012:64010:/usr/local/domains/domain.com:-::
.
Edit /var/qmail/control/virtualdomains and add:
domain.com:domain.com
This will instruct qmail-send (after we HUP it) to deliver all mail for domain.com to /usr/local/domains/domain.com . In addition, it will use the uid/gid of the qmail user/group. Make sure those are correct for your system.
Fourth, create a .qmail-default in your /usr/local/domains/domain.com directory:
|/usr/local/domains/bin/filter.sh
That's a script that will call our procmail script to rewrite the To: and Subject: lines.
So put the following into /usr/local/domains/bin/filter.sh - and make it executable.
#!/bin/sh
`/var/qmail/bin/preline /usr/bin/procmail -p -m //usr/local/domains/bin/procmail.rc`
EXITCODE=$?
cat > /dev/null
if [ $EXITCODE -ne 0 ]; then
exit 0
else
#Exited with 0, delivered, so set 99 to stop vdelivermail
exit 99
fi
As you can see, it calls for /usr/local/domains/bin/procmail.rc. Your procmail.rc file should look like:
SHELL="/bin/sh"
# If you need to debug your configuration of ackmail.rc, just "touch
# ackmail.rc.log" and the recipe below will log the ackmail.rc activity
# to ackmail.rc.log. When you are satisfied with your configuration,
# simply remove ackmail.rc.log and logging will stop.
# http://www.herring.org/tips/procmail/ackmail.rc
:0
* ? test -f /var/log/procmail.log
{
LOGFILE=/var/log/procmail.log
VERBOSE=yes
LOGABSTRACT=all
}
###Verify EXT User is not an alias
##Will be changed from user-ext@/user-ext-ext to user@
## Assuming no user-user usernames
:0
* EXT ?? -
{
EXT=`echo $EXT $HOST | awk -F- '{print $1}'|awk '{print $1}'`
#SUBJECT: our variable with the original subject, for writing the new subject
SUBJECT=`/usr/bin/formail -zxSubject:`
:0 f
| /usr/bin/formail -I "Subject: [$EXT2] $SUBJECT"
}
:0w
| /var/qmail2/bin/forward $EXT@$HOST
Bascially, the procmail.rc only touches emails with '-' in the To: ($EXT) address, and only runs formail to modify the subject when necessary. $EXT2 represents everything after the first EXT up to the @.
So username-ibm@domain.com turns into domain.com-username-ibm@domain.com (funky, but that's how the routing works), which creates the following qmail variables:
$USER-$EXT-$EXT2-$EXT3-$EXT4@$HOST
NOTE: The qmail2 at the end of the file. This is your 2nd qmail install with an smtproute to your Exchange server for your domain. If you use the same qmail install as the forward destination, the mail will just loop.
Just for completeness, here's the smtproute. I suggest you ensure this is working on your first email install prior to these changes:
smtproutes:
domain.com:192.168.1.100
That's it, restart qmail-send (via svc -t /service/send if you used daemontools) or use whatever method you want to send qmail-send a HUP signal to re-read the config files.
Since I already front-end my Exchange server with a qmail server for SpamAssassin, enabling support for subaddressing was pretty straightfoward. You may want to follow a toaster or Life With Qmail to get a basic working install with procmail included.
Assuming you have a working install -
First, you must create a 2nd qmail install if you haven't already. The reason for this is that to keep things from being too confusing, we'll need to deliver our domain mail locally and then re-forward to the same domain. If you try and forward within the same qmail structure, you'll end up in a loop.
Again, I already have a 2nd qmail install. The reason for this is to normal allow outbound mail to have a short queuelifetime (a couple hours), but provide longer lifetimes to other domains if necessary. Here, qmail-smtpd listens on localhost, and control/smtproutes just directs that domain to localhost to inject mail into the 2nd queue.
Second, create the file structures:
/usr/local/domains/bin
/usr/local/domains/domain.com
Third, create a user. If you do not have a users/assign file, you'll need to create one in the format of:
+domain.com-:domain.com:64012:64010:/usr/local/domains/domain.com:-::
.
Now run /var/qmail/bin/qmail-newu to 'activate' the user.
Edit /var/qmail/control/virtualdomains and add:
domain.com:domain.com
This will instruct qmail-send (after we HUP it) to deliver all mail for domain.com to /usr/local/domains/domain.com . In addition, it will use the uid/gid of the qmail user/group. Make sure those are correct for your system.
Fourth, create a .qmail-default in your /usr/local/domains/domain.com directory:
|/usr/local/domains/bin/filter.sh
That's a script that will call our procmail script to rewrite the To: and Subject: lines.
So put the following into /usr/local/domains/bin/filter.sh - and make it executable.
#!/bin/sh
`/var/qmail/bin/preline /usr/bin/procmail -p -m //usr/local/domains/bin/procmail.rc`
EXITCODE=$?
cat > /dev/null
if [ $EXITCODE -ne 0 ]; then
exit 0
else
#Exited with 0, delivered, so set 99 to stop vdelivermail
exit 99
fi
As you can see, it calls for /usr/local/domains/bin/procmail.rc. Your procmail.rc file should look like:
SHELL="/bin/sh"
# If you need to debug your configuration of ackmail.rc, just "touch
# ackmail.rc.log" and the recipe below will log the ackmail.rc activity
# to ackmail.rc.log. When you are satisfied with your configuration,
# simply remove ackmail.rc.log and logging will stop.
# http://www.herring.org/tips/procmail/ackmail.rc
:0
* ? test -f /var/log/procmail.log
{
LOGFILE=/var/log/procmail.log
VERBOSE=yes
LOGABSTRACT=all
}
###Verify EXT User is not an alias
##Will be changed from user-ext@/user-ext-ext to user@
## Assuming no user-user usernames
:0
* EXT ?? -
{
EXT=`echo $EXT $HOST | awk -F- '{print $1}'|awk '{print $1}'`
#SUBJECT: our variable with the original subject, for writing the new subject
SUBJECT=`/usr/bin/formail -zxSubject:`
:0 f
| /usr/bin/formail -I "Subject: [$EXT2] $SUBJECT"
}
:0w
| /var/qmail2/bin/forward $EXT@$HOST
Bascially, the procmail.rc only touches emails with '-' in the To: ($EXT) address, and only runs formail to modify the subject when necessary. $EXT2 represents everything after the first EXT up to the @.
So username-ibm@domain.com turns into domain.com-username-ibm@domain.com (funky, but that's how the routing works), which creates the following qmail variables:
$USER-$EXT-$EXT2-$EXT3-$EXT4@$HOST
NOTE: The qmail2 at the end of the file. This is your 2nd qmail install with an smtproute to your Exchange server for your domain. If you use the same qmail install as the forward destination, the mail will just loop.
Just for completeness, here's the smtproute. I suggest you ensure this is working on your first email install prior to these changes:
smtproutes:
domain.com:192.168.1.100
That's it, restart qmail-send (via svc -t /service/send if you used daemontools) or use whatever method you want to send qmail-send a HUP signal to re-read the config files.
Restarting Exchange
For whatever reason my new Exchange 2010 server does not properly update everyone's offline addressbook in Outlook. I've given up trying to fix it. Here's a batch file to restart all the services, so I don't have to reboot the entire damn thing.
@Echo Off
Echo 'Stopping Microsoft Exchange Services'
net stop MSExchangeAB
net stop MSExchangeADTopology
net stop MSExchangeAntispamUpdate
net stop MSExchangeEdgeSync
net stop MSExchangeFBA
net stop MSExchangeFDS
net stop MSExchangeIS
net stop MSExchangeMailboxAssistants
net stop MSExchangeMailboxReplication
net stop MSExchangeMailSubmission
net stop MSExchangeProtectedServiceHost
net stop MSExchangeRepl
net stop MSExchangeRPC
net stop MSExchangeSA
net stop MSExchangeSearch
net stop MSExchangeServiceHost
net stop MSExchangeThrottling
net stop MSExchangeTransport
net stop MSExchangeTransportLogSearch
Echo 'Starting Microsoft Exchange Services'
net start MSExchangeAB
net start MSExchangeADTopology
net start MSExchangeAntispamUpdate
net start MSExchangeEdgeSync
net start MSExchangeFBA
net start MSExchangeFDS
net start MSExchangeIS
net start MSExchangeMailboxAssistants
net start MSExchangeMailboxReplication
net start MSExchangeMailSubmission
net start MSExchangeProtectedServiceHost
net start MSExchangeRepl
net start MSExchangeRPC
net start MSExchangeSA
net start MSExchangeSearch
net start MSExchangeServiceHost
net start MSExchangeThrottling
net start MSExchangeTransport
net start MSExchangeTransportLogSearch
net start msexchangepop3
net start msexchangeimap4
End
@Echo Off
Echo 'Stopping Microsoft Exchange Services'
net stop MSExchangeAB
net stop MSExchangeADTopology
net stop MSExchangeAntispamUpdate
net stop MSExchangeEdgeSync
net stop MSExchangeFBA
net stop MSExchangeFDS
net stop MSExchangeIS
net stop MSExchangeMailboxAssistants
net stop MSExchangeMailboxReplication
net stop MSExchangeMailSubmission
net stop MSExchangeProtectedServiceHost
net stop MSExchangeRepl
net stop MSExchangeRPC
net stop MSExchangeSA
net stop MSExchangeSearch
net stop MSExchangeServiceHost
net stop MSExchangeThrottling
net stop MSExchangeTransport
net stop MSExchangeTransportLogSearch
Echo 'Starting Microsoft Exchange Services'
net start MSExchangeAB
net start MSExchangeADTopology
net start MSExchangeAntispamUpdate
net start MSExchangeEdgeSync
net start MSExchangeFBA
net start MSExchangeFDS
net start MSExchangeIS
net start MSExchangeMailboxAssistants
net start MSExchangeMailboxReplication
net start MSExchangeMailSubmission
net start MSExchangeProtectedServiceHost
net start MSExchangeRepl
net start MSExchangeRPC
net start MSExchangeSA
net start MSExchangeSearch
net start MSExchangeServiceHost
net start MSExchangeThrottling
net start MSExchangeTransport
net start MSExchangeTransportLogSearch
net start msexchangepop3
net start msexchangeimap4
End
Saturday, July 14, 2012
Proper NFS server service restart on FreeBSD
I always get this wrong, because I only deal with NFS maybe once a year - typically it just works, but I always have issues when first getting it up and running. I'd prefer not to reboot, so this is the proper order:
# killall -9 mountd
# /etc/rc.d/nfsd stop
# /etc/rc.d/rpcbind stop
# /etc/rc.d/rpcbind start
# /etc/rc.d/nfsd start
Starting mountd.
Starting nfsd.
Thanks to: https://www.secure-computing.net/wiki/index.php/NFS
# killall -9 mountd
# /etc/rc.d/nfsd stop
# /etc/rc.d/rpcbind stop
# /etc/rc.d/rpcbind start
# /etc/rc.d/nfsd start
Starting mountd.
Starting nfsd.
Thanks to: https://www.secure-computing.net/wiki/index.php/NFS
Thursday, December 29, 2011
Active Directory UserMod Assistant
The Active Directory UserMod Assistant is an awesome application for your end users to change their own AD profile data, such as telephone number or address. It's a free solution, and it's built on HTML and VBScript - so all the core requirements should be on your user's workstations. In addition, the program is not compiled, allowing easy editing. For example, we put extensions in the 'Business' telephone number field, and don't want to validate it.
I'm deploying it via shortcut to a shared folder (The shortcut is autocreated on the user's desktop if it doesn't exist on login), so users who need VPN access can add their mobile number(s) for use with Phone Factor.
Note - There may be Security Issues when running the application from a share that is not in your Trusted Sites.
I'm deploying it via shortcut to a shared folder (The shortcut is autocreated on the user's desktop if it doesn't exist on login), so users who need VPN access can add their mobile number(s) for use with Phone Factor.
Note - There may be Security Issues when running the application from a share that is not in your Trusted Sites.
Wednesday, December 21, 2011
'Enterprise' System Monitoring with OSS tools
I've been building an IT Department, and of course, one of the basic requirements is metrics and monitoring. I played with AlienVault for a bit, but it's more of a vulnerability management/log consolidation tool. I stumbled upon ZenOSS via Proxmox (it's an included VM), and seems to work quite well.
After some setup, I now get both WMI and SNMP notifications (like Nagios) from various devices. I also have traffic graphs (like Cacti) from SNMP for interfaces on those devices. Note - Exchange sucks, and when it crashes I get no notifications. Time to get the Qmail SMTP relay in place for reliable email delivery.
What I was missing was NetFlow data. Not a big deal - throw a OpenVZ Debian VM onto Proxmox and update it with nfdump and nfsen (modified from http://www.linuxscrew.com/2010/11/25/how-to-monitor-traffic-at-cisco-router-using-linux-netflow/ )
apt-get update
apt-get upgradehttp://www.blogger.com/img/blank.gif
dpkg-reconfigure tzdata
apt-get install nfdump
Whoa! Do you need to see ASA Netflow? - it's non standard, install an older 'nsel' version of nfdump from source instead!
aptitude install rrdtool librrd2-dev librrd-dev librrd4 librrds-perl librrdp-perl
apt-get install libmailtools-perl
apt-get install apache2 php5
apt-get install tcpdump
cd /usr/src/
wget http://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.5/nfsen-1.3.5.tar.gz/download
tar -xvzf nfsen-1.3.5.tar.gz
cd nfsen-1.3.5
cp etc/nfsen-dist.conf etc/nfsen.conf
mkdir -p /data/nfsen
In order to continue you should edit file etc/nfsen.conf to specify where to install nfsen, web server’s username, its document root directory etc. That file is commented so there shouldn’t be serious problems with it.
One of the major sections of nfsen.conf is ‘Netflow sources’, it should contain exactly the same port number(s) you’ve configured Cisco with — recall ‘ip flow-export …’ line where we’ve specified port 23456. E.g.
%sources = (
'Router1' => { 'port' => '23456', 'col' => '#0000ff', 'type' => 'netflow' },
);
Now it’s time to finish the installation:
./install.pl etc/nfsen.conf
In case of success you’ll see corresponding notification after which you will have to start nfsen daemon to get the ball rolling:
http://www.blogger.com/img/blank.gif
/path/to/nfsen/bin/nfsen start
From this point nfdump started collecting netflow data exported by Cisco router and nfsen is hardly working to visualize it — just open web browser and go to http://linux_web_server/nfsen/nfsen.php to make sure. If you see empty graphs just wait for a while to let nfsen to collect enough data to visualize it.
That’s it!
Parts taken from linuxscrew.com
Now I just need to figure out why my other ASA doesn't seem to support Netflow exports...
After some setup, I now get both WMI and SNMP notifications (like Nagios) from various devices. I also have traffic graphs (like Cacti) from SNMP for interfaces on those devices. Note - Exchange sucks, and when it crashes I get no notifications. Time to get the Qmail SMTP relay in place for reliable email delivery.
What I was missing was NetFlow data. Not a big deal - throw a OpenVZ Debian VM onto Proxmox and update it with nfdump and nfsen (modified from http://www.linuxscrew.com/2010/11/25/how-to-monitor-traffic-at-cisco-router-using-linux-netflow/ )
apt-get update
apt-get upgradehttp://www.blogger.com/img/blank.gif
dpkg-reconfigure tzdata
apt-get install nfdump
Whoa! Do you need to see ASA Netflow? - it's non standard, install an older 'nsel' version of nfdump from source instead!
aptitude install rrdtool librrd2-dev librrd-dev librrd4 librrds-perl librrdp-perl
apt-get install libmailtools-perl
apt-get install apache2 php5
apt-get install tcpdump
cd /usr/src/
wget http://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.5/nfsen-1.3.5.tar.gz/download
tar -xvzf nfsen-1.3.5.tar.gz
cd nfsen-1.3.5
cp etc/nfsen-dist.conf etc/nfsen.conf
mkdir -p /data/nfsen
In order to continue you should edit file etc/nfsen.conf to specify where to install nfsen, web server’s username, its document root directory etc. That file is commented so there shouldn’t be serious problems with it.
One of the major sections of nfsen.conf is ‘Netflow sources’, it should contain exactly the same port number(s) you’ve configured Cisco with — recall ‘ip flow-export …’ line where we’ve specified port 23456. E.g.
%sources = (
'Router1' => { 'port' => '23456', 'col' => '#0000ff', 'type' => 'netflow' },
);
Now it’s time to finish the installation:
./install.pl etc/nfsen.conf
In case of success you’ll see corresponding notification after which you will have to start nfsen daemon to get the ball rolling:
http://www.blogger.com/img/blank.gif
/path/to/nfsen/bin/nfsen start
From this point nfdump started collecting netflow data exported by Cisco router and nfsen is hardly working to visualize it — just open web browser and go to http://linux_web_server/nfsen/nfsen.php to make sure. If you see empty graphs just wait for a while to let nfsen to collect enough data to visualize it.
That’s it!
Parts taken from linuxscrew.com
Now I just need to figure out why my other ASA doesn't seem to support Netflow exports...
Friday, December 16, 2011
ZFS speed
So I ran some quick dd-based write numbers before choosing a filesystem for my new Proxmox based virtual server. I really wanted ZFS, but I need OpenVZ and KVM more, and ZFS-Fuse just does cut it.
In short - ext4 is the winner, though ZFS sure looks pretty damn fast. SolarisInternals has more info on ZFS write speed
Another thing to note, though the below numbers for ZFS-Fuse don't reflect it, is that ZFS performs better with Write-Back cache off on your Raid controller. Had I tested OpenSolaris with Write-Back cache on (boot up was ungodly slow with it on), I'm sure it would have been reflected in the numbers.
4 600GB, RAID 5, wb on, perc 7, ext4
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 2.53637 s, 423 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 20.3379 s, 528 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (107 GB) copied, 192.164 s, 559 MB/s
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, ext3
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 4.58121 s, 234 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 37.0681 s, 290 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (107 GB) copied, 371.047 s, 289 MB/s
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, reiser3
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 2.91366 s, 369 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 28.923 s, 371 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
Hmm Not sure what happened here...
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, reiser4
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 2.87369 s, 374 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 28.9769 s, 371 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
Hmm Not sure what happened here...
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, zfs-fuse (defaults)
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 15.0122 s, 71.5 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 159.544 s, 67.3 MB/s
100GB - didn't bother
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb off, perc 7, zfs-fuse (defaults)
proxmox2:/# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 23.7539 s, 45.2 MB/s
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb off, perc 7, zfs-fuse (64k recordsize)
proxmox2:/# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 21.3764 s, 50.2 MB/s
---------------------------------------------------------------------------------------
4 600GB, raidz, wb off, perc 7, zfs-fuse
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 19.1704 s, 56.0 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 187.938 s, 57.1 MB/s
---------------------------------------------------------------------------------------
4 600GB, raidz, wb off, perc 7, ZFS opensolaris (Nexenta)
root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 1.30164 seconds, 825 MB/s
root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 5.18124 seconds, 2.1 GB/s
root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (107 GB) copied, 45.0769 seconds, 2.4 GB/s
In short - ext4 is the winner, though ZFS sure looks pretty damn fast. SolarisInternals has more info on ZFS write speed
Another thing to note, though the below numbers for ZFS-Fuse don't reflect it, is that ZFS performs better with Write-Back cache off on your Raid controller. Had I tested OpenSolaris with Write-Back cache on (boot up was ungodly slow with it on), I'm sure it would have been reflected in the numbers.
4 600GB, RAID 5, wb on, perc 7, ext4
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 2.53637 s, 423 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 20.3379 s, 528 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (107 GB) copied, 192.164 s, 559 MB/s
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, ext3
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 4.58121 s, 234 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 37.0681 s, 290 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (107 GB) copied, 371.047 s, 289 MB/s
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, reiser3
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 2.91366 s, 369 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 28.923 s, 371 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
Hmm Not sure what happened here...
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, reiser4
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 2.87369 s, 374 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 28.9769 s, 371 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=102400 conv=fsync
Hmm Not sure what happened here...
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb on, perc 7, zfs-fuse (defaults)
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 15.0122 s, 71.5 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 159.544 s, 67.3 MB/s
100GB - didn't bother
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb off, perc 7, zfs-fuse (defaults)
proxmox2:/# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 23.7539 s, 45.2 MB/s
---------------------------------------------------------------------------------------
4 600GB, RAID 5, wb off, perc 7, zfs-fuse (64k recordsize)
proxmox2:/# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 21.3764 s, 50.2 MB/s
---------------------------------------------------------------------------------------
4 600GB, raidz, wb off, perc 7, zfs-fuse
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 19.1704 s, 56.0 MB/s
proxmox2:~# dd if=/dev/zero of=/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 187.938 s, 57.1 MB/s
---------------------------------------------------------------------------------------
4 600GB, raidz, wb off, perc 7, ZFS opensolaris (Nexenta)
root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=1024 conv=fsync
1073741824 bytes (1.1 GB) copied, 1.30164 seconds, 825 MB/s
root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=10240 conv=fsync
10737418240 bytes (11 GB) copied, 5.18124 seconds, 2.1 GB/s
root@proxmox2:/volumes# dd if=/dev/zero of=/volumes/images/file bs=1024k count=102400 conv=fsync
107374182400 bytes (107 GB) copied, 45.0769 seconds, 2.4 GB/s
Subscribe to:
Posts (Atom)