Cameron Stokes’s Blog

I ran out of free space on my file server a few months back and have been too cheap to upgrade until recently. I first started with four 400 GB drives a few years ago and setup a RAID 5 array using Linux and mdadm giving me roughly 1.2 TB of space. I upgraded a disk at a time as needed until I had seven drives and about 2.6 TB of space:

Filesystem Size Used Avail Use% Mounted on
/dev/md0 2.6T 2.6T 2.8G 100% /mnt/sata-array

Having filled this up I started looking for the best way to upgrade. The Seagate 1.5 TB drives offered the best per GB pricing however with the recent bad news specific to these drives I decided to stear clear of Seagate. The next best option I could find were the 1 TB Western Digital Green Power drives. The Green Power drives have a lower overall rotational speed and will spin down as allowed to save power with the added benefit of generating less heat and also noise.

After reading several threads (one here) regarding Bit Error Rates (BER) with drives this large and the potential for errors even with a RAID 5 array I decided to go for RAID 6 with my new array. RAID 6 can survive 2 disk drive failuers and will theoretically protect me from the high BER these drives are susceptible to.

In a previous post I mentioned trying ZFS. I scrapped that idea when I found you can’t expand a ZFS zpool by adding single drives to the system. You can add additional zpools to a zpool (not sure if my terminology is right there) but that means adding an equal amount of storage to your existing array when you want to upgrade. This means if I have 4 drives in a zpool, I’d have to add another 4 drives to upgrade. You can also replace drives in the zpool with larger drives to expand them, but you have to replace each drive. Neither option is very cost effective for me. I was a little disappointed not to be able to try something new but have been happy with Linux and mdadm so it was not a difficult decision to go back to them.

I ordered 6 drives from Amazon and once they arrived I assembled them into a RAID 6 array. mdadm has to sync the newly created array but the array is online and can be used normally. A quick ext3 format and I have 3.6 TB of space available.

Filesystem Size Used Avail Use% Mounted on
/dev/md0 3.6T 229G 3.2T 7% /opt/sata-array

After the format I started transferring my data which slows down the sync, but transferring all my data will take a few days anyway so the overall timeline doesn’t really matter for me. Doing a cat /proc/mdstat shows me the status of the array:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid6 sdf[5] sde[4] sdd[3] sdc[2] sdb[1] sda[0]
3907049984 blocks level 6, 64k chunk, algorithm 2 [6/6] [UUUUUU]
[======>..............] resync = 32.0% (312900864/976762496) finish=4727.0min speed=2340K/sec

unused devices: <none>

Short of having to setup a second computer to transfer the data, the entire process is very easy. I feel better knowing my data is reasonably safe with RAID 6 even though I don’t have a backup. Backing up that much data is just too expensive to be worthwhile.

Despite temperatures dipping into single digits only 2 weeks ago and a supposed 6 more weeks of Winter according to Punxsutawney Phil, it really looks and feels like Spring is on the way. The past week has really warmed up and there is new growth on many of the plants in my yard, even some daylilies peaking out of the ground.

Spring is on the way.

Any developer or system administrator worth their salt will know what I’m talking about.

Background

I’ve been needing to upgrade my home fileserver for a while and finally broke down and bought some parts. Being afraid of losing my data, I decided I’d move to OpenSolaris so I could take advantage of the extra features ZFS has to keep my bits safe. One of the first issues I encountered was that the 2 Promise SATA300 TX4 cards I had in my computer weren’t supported in OpenSolaris. After some googling I found that the Syba SD-SATA-4P SATA controller, based on the Sil3114 chipset from Silicon Image, available on Newegg was supported as long as the non-RAID BIOS was installed on the card.

The Issue

After receiving it from Newegg, I fired up the Sun Device Detection Tool to see if it was reported as supported. It was not, but having read the reviews on Newegg I knew I needed to install the non-RAID BIOS on the card. I downloaded the 5.4.0.3 version from the Silicon Imaging site (download here) and realized I needed either a DOS boot disk or a Windows machine from which I could update it. Having neither, I pulled out one of the older computers from my closet, installed Windows XP, and proceeded to install the card’s drivers. Once that was complete I tried to flash the non-RAID BIOS, but received a pretty nasty error message:

Flash erase operation failed.

Given how finicky BIOS updates can be, I was afraid I had bricked the card. I tried the RAID bios and it flashed without an issue. I decided to try the first rule in troubleshooting a computer issue, reboot. After rebooting I was still having the issue, I tried a few more times, read a few more posts online, checked the Syba website which gave me nothing.

The Solution

About to give up hope I re-read the readme that came with the download and noticed this line:

Ensure that a SATA device is plugged into a 3114 SATA port.

I remember reading the line earlier but brushed it off, figuring that couldn’t really be required. Having tried everything else, I shutdown, plugged in a SATA drive, started back up and was able to flash the BIOS without issue. I ran the Sun Device Detection Tool again and the card was recognized and shown as being supported. I’m still curious why there needs to be a drive connected in order to flash the card.

Our internet connection at home has been flaking out for the past few days and finally became intolerable today, with over 40% packet loss.  We called Comcast and have a technician scheduled to come out next week.  Usually we’d get by just fine but Rhonda needed some files from work so she could catch up on a few things over the weeken dso we ventured out to the local Taco Mac to take advantage of their free wifi.

Turns out their Wifi service is provided by Wandering Wifi.  The Wandering Wifi service requires you to sign up for an account but Rhonda and I were able to create one account and share it.  The wireless connection seems to drop every 5-10 minutes, I imagine its the microwave being turned on back on the connection.  Out of curiosity I figured out their ISP is Cbeyond.

Anyway, we got our dose of the internet.  I tried two beers I had never had before; Atlanta Brewing Company’s Peachtree Pale Ale and their Blonde Ale.  For whatever reason I’ve never had these before even though they’re brewed in Atlanta (obviously).  They’re quite tasty, though my local favorite is still all the brews from Sweetwater Brewing Company.