Being something of a professional nerd, my idea of storage doesn’t always line up with consumer thinking. I’ve been building my own systems for quite a while, so the idea of building my own NAS was going to happen sooner or later.

Hardware

The hardware chosen has changed over the years, in part, based on what was available. Sometimes good deals can be had on rackmount equipment if you happen to be in the right place at the right time, and know the right people.

Currently, I’m using the following hardware:

Previous configuration

  • Dell R610
    • 1x 4 core 2.4GHz Xeon
    • 24GB RAM
    • 1x USB Flash (OS)
    • 2x 32GB OCZ SSD (ZIL)
    • 1x 256G OCZ SSD (L2Arc)
  • Dell PowerVault MD1000
    • 8x 2TB Seagate SATA
    • 6x 256GB OCZ Vertex SSD

The R730xd is connected by 4x ethernet cables to a Juniper EX4200 gigabit switch using LACP (LACP provides additional bandwidth when multiple clients are connected to the same source).

If you are interested in more of an appliance type of hardware and don’t want to build your own, you can buy them on Amazon. These even come with FreeNAS pre-installed. You can choose an empty chassis and provide your own drives, or you can choose a pre-populated version.

Software

For software, I’m using FreeNAS. FreeNAS is a community appliance based on FreeBSD and sponsored by iX Systems. I’ve been using FreeNAS for over 9 years now and find it more than adequate for the job. I’m using ZFS to manage the disks. There are 2 pools of storage, each in a striped mirror configuration (similar to a RAID10). This results in 1x 8TB pool and 1x 768GB pool (usable storage, not raw capacity). Each pool has the ability to lose up to 1/2 of the disks with out any data loss. One of the advantages I find to using ZFS (I’ve been using ZFS since it became available in FreeBSD 7.0R) is that I can upgrade the individual disks in the volume to increase the amount of storage in place. Using traditional storage mechanisms, I would have to backup the data, replace the disks, and then restore the data. This way I save many hours of very boring work (yes, there are snapshots made before any hardware changes).

File Sharing Services

FreeNAS provides the ability to share data via CIFS/SMB, Appletalk, NFS, or iSCSI, so it has no trouble integrating into pretty much any environment. I’m currently sharing the SSD pool over iSCSI to VMware, and the SATA pool over AFP and NFS (different directories) to Apple and UNIX clients. I’ve managed to get about 850Mbit (my home network has both 802.11b/g/n, and Gig-e wire, the 850Mbit was on the wire) between my Mac Pro and my FreeNAS, but it seems that is where the desktop maxes out (it only has the speed of a single disk). This is pretty close to wire speed, and is “good enough”.

Note: If you share the same files over multiple protocols (such as CIFS and AFP), corruption can occur.

Statistics

I don’t push a lot of traffic on my home system. Its a mix of AFP/CIFS, and iSCSI (VMs).

Most of the ARC hit is VMs. The majority of the “nas” traffic is not regular in terms of the files that are requested.

)


Footnotes