davd.io

Building a FreeBSD NAS Part 1: Concept & Hardware selection

published on in category FreeBSD NAS , Tags: freebsd nas zfs selfhosted

Table of contents

After a long while I finally decided to build a new NAS / home server for my various needs. Though there are many solutions available, I chose to build one on my own as I want as much flexibility as possible. So I set out to buy all components needed for the system with upgradability and budget in mind.

1. Requirements & Conceptual ideas

Initially I took some time to come up with the main goals for this build. First of all, I never ever want to lose any of my data again like I did some months ago.

  • Solid storage solution including mirroring of my data as well as an offline backup
  • Ability to divide the system (software) into containers for the various tasks (Jails) that allows them to be upgraded and restarted separately
  • Quality hardware though as cheap as possible
  • Extensible hardware concept
  • Low power consumption
  • As silent as possible
  • Automatic time-based boot and shutdown for energy saving

These are the things I want to use the server for:

While this seems to be a lot of stuff, it’s actually not. I assume that the system will idle at least 3/4 of the time, with disk I/O and software RAID being the most performance intense tasks.

1.1. Why not FreeNAS / NAS4Free?

FreeNAS and NAS4Free provide powerful solutions and pretuned settings for running a file server. And although I really like FreeNAS, I decided to go with a vanilla FreeBSD installation and set up everything on my own. This has multiple reasons including the newer codebase (FreeNAS is still on the FreeBSD 9.3 branch) and full control over my jails (I don’t want to have my fileserver on the host directly). Additionally, I don’t really need the pretty web interface that FreeNAS offers.

2. IKEA Lack Rack

As you can probably tell knowing that I have http://reddit.com/r/HomeServer bookmarked, I really like DIY stuff. When I came across this post http://imgur.com/a/H7KZI of someone telling that he used a Ikea Lack table to build a small server rack, I recognized that I got one myself lasting it’s time unused in the basement. I decided to give it a try by mounting all the hardware needed to this table. This would also be a good option to place the printer on which I want to serve using the NAS.

IKEA Lack Rack

3. Hardware selection

As server hardware is really really expensive, I chose to go with desktop-class hardware for the most parts. I’ve chosen a 4th-gen Intel Celeron G1840 CPU, 8 gigs of DDR3 memory and a Asus H81M-K motherboard. I chose Non-ECC RAM for the beginning but as the CPU supports ECC, upgrading is only a matter of replacing the memory and the motherboard. 8 GB might sound little for many but as I’m not having that much storage and I don’t use the memory hungry ZFS features like deduplication, I’m sure that this is enough. I decided to go with Western Digital Red disks, each 3 TB of space which will run mirrored starting with two disks. Additionally I chose to buy a cheap Corsair 60GB SSD for L2ARC cache. The operating system will be installed on a WD Blue 500 GB that I already own. For power supply I chose a Cooler Master B500 with 500 Watts, which should be enough for powering all the disks that fit into the case. For the record: WD Red drives should not consumpt more than 15 Watts under load.

As I already mentioned, I want to use my self-made Lack rack to house the server, so I chose a rack-mount 4u server case for about 75 EUR.

I got all the stuff on Amazon, here’s my shopping list:

Total: 522,67 EUR

Additionally, to make anything as safe as possible, I could extend it using:

  • Separate SATA/PCIe card that I use to attach the mirror-disks for each drive to eliminate the risk of a single failing SATA controller
  • Additional hard drive to mirror the system disk

3.1. ECC vs. Non-ECC memory

What most people fear when using ZFS as a file system is called “scrub of death”. ZFS creates a checksum for every written block that allows the filesystem to detect whether a block has gone mad. When using a mirror or RAID 5-alike RAID structure, it even can recover the data from the mirror-disk or restore it from parity. But when the memory goes mad and starts to flip bits, the calculated checksum might not match the checksum written to disk and people fear that ZFS will kill the healthy data and override it with crap.

But I think this issue is somewhat overstated. First of all, the chance that all of the memory goes mad without recognizing it before starting a scrub is very unlikely as ZFS performs checksum matching per access. And even if there is a address that fails, not all checksums for the whole storage pool are written to that address so this kind of issue would more likely affect only some blocks of the file system. But even if that happened and the checksums are wrong, ZFS will either check if it can recover the block from the mirror disk and fail because that block doesn’t match the checksum either or if it would work, it would override a healthy block with the healthy opposite on the mirror disk. In the other case it would log an unrecoverable error and do nothing. Next time a scrub is requested, all the “unrecoverable errors” will likely be handled at another memory address, pass the test and ZFS will remove the remark.

I know that there’s way more to talk about here but then we’d also have to talk about the possibility that the SATA controller might fail or a socket or a cable… So long story short, there’s so many stuff that could probably fail that ECC memory would only eliminate one of many issues - but I want to be as safe as possible, so I will use an external USB 3 disk to back up the most relevant data from time to time and detach it when I’m done - easy as that.

3.2. Disk redundancy / Backup concept

I chose to set up a ZFS mirror which is equivalent to a RAID1. So if one disk fails, the system can keep running as long as the second one doesn’t (so it might be a good idea to pair disks not from the same production date so they won’t fail at the same time ;-)) and thanks to ZFS goodness, resilvering will not take that long when I need to replace a drive. Additionally, I have a separate USB 3 disk, that I will use to back up the most relevant stuff from time to time and keep it detached otherwise, so that my data is as safe as possible.

In Part 2 I describe how I assembled the hardware.