Its Alive!! 7.8 TB of RAIDz goodness

[Adul]

It's finally built. After deciding that virtualizing the storage server was just not going to work with XenCP. I finally just loaded FreeNAS 8.0 RC3 and built a beefy storage server for home use.

1 Quadcore Athalon II
8GB of DDR3
1 - 2GB thumb drive for FreeNAS
1 - 60 GB SSD for Cache
2 - 160GB HD for Logs
5 - 2TB HD for data
1 DVDROM to load it 0.o
1 GB 890GPA-UD3H MB (8 SATA ports FTW)

Overkill? yes. Worth it for ZFS? Hell yes

Stor01 /mnt/Stor01 34.3 KB (0%) 7.8 TB 7.8 TB ONLINE
Now to build my XEN CP server.

 

[Spikesoldier]

grats!

im running freenas 7.x built around a dually e6420/p35/2gb configuration made of spare parts.

definitely overkill, but i can guarantee the bottlneck is going to be the disks, so you got every other factor over-kilt.

if i could build the system again, i would go with a zacate mainboard and 1 or 2gb ddr3.

 

[Adul]

There are always bottle necks, but I can live with writing 60MB/sec over 1Gbps network connection any day.

So right now I am copying 128GB of various ISO from Technet, other software, random tools and utilities. its at 64.4MB/sec and holding around there. Not bad at all.

 

[Spikesoldier]

ive sustained like 84-86MBps over gigabit. kind of unfair though because it was a 246gb single file.

this was from my local computer with a 320gb 7200rpm wd blue to nasbox with a target of a pair of samsung 1tb 7200rpm drives in raid 1

 

[Adul]

ive sustained like 84-86MBps over gigabit. kind of unfair though because it was a 246gb single file.

this was from my local computer with a 320gb 7200rpm wd blue to nasbox with a target of a pair of samsung 1tb 7200rpm drives in raid 1

I think the RAIDz and 5400RPM HD are hurting the write speed a tad. But it is more than 4 to 5 times faster than previous setup. So I am good with it as it is. The read speeds so be good though. 60GB SSD cache is there for a reason.

Damn 500GB of photos take a long time to copy At least they are all RAWs

 

[Emulex]

wow that's painful. i can sustain two gigabit with raid-5 on windows storage server but you might have more parity than raid-5

 

[Adul]

wow that's painful. i can sustain two gigabit with raid-5 on windows storage server but you might have more parity than raid-5

Well i am not just doing RAID 5, its RAIDz. So I dont know what to expect for writes. The read speeds on the other hand are great. Just maxed out the gb link
transferring a 1GB file

 

[Emulex]

yah bring up a second gigabit and see if you can saturate both on read. i bet you can.

 

[Adul]

yah bring up a second gigabit and see if you can saturate both on read. i bet you can.

prob I could. but I need to run more lines into that room before I can. I haven't touched the network yet.

 

[dorion]

Wait you didn't give the zil a SSD? Is the hdd for the zil faster than the raidz drives?

 

[Adul]

Wait you didn't give the zil a SSD? Is the hdd for the zil faster than the raidz drives?

I didn't have another SSD drive for that . Maybe I can snag my GF dual 30GB drives at a later date for it when she upgrades . But yeah those drives for ZIL are well, not exactly fast

 

[Rubycon]

No infiniband link?

 

[ChrisBenn]

I didn't have another SSD drive for that . Maybe I can snag my GF dual 30GB drives at a later date for it when she upgrades . But yeah those drives for ZIL are well, not exactly fast

I'd just remove the ZIL all together then - you know if the ZIL craps out and you are using an older ZPOOL version you can loose your entire pool (so basically you aren't disk failure redundant any more). You don't need a separate ZIL drive, you can just use one to speed up performance. But a slow zil drive will probably have worse specs than your array, so I wouldn't run it. (Now if you can safely remove it or not is another issue - I think that feature came in around zpool 22?)

 

[Spikesoldier]

I think the RAIDz and 5400RPM HD are hurting the write speed a tad. But it is more than 4 to 5 times faster than previous setup. So I am good with it as it is. The read speeds so be good though. 60GB SSD cache is there for a reason.

Damn 500GB of photos take a long time to copy At least they are all RAWs

i hear ya, bud. i was using a d-link nas before and while its simple and useful for any soho setup, i wanted more performance and went the freenas route. glad i did!

 

[Adul]

i hear ya, bud. i was using a d-link nas before and while its simple and useful for any soho setup, i wanted more performance and went the freenas route. glad i did!

well it has been 6 days with current setup and I am extremely happy with the way it has been performing. I almost wish i had made the array larger, for the full 10TB. I do plan to tweak the setup a bit though. I want to add an 40GB F40, maybe 2 for ZIL logs and bond 2 of the three gb nics in the system together.

Up next for me is I need to assemble my XenServer so I can add some nagios monitoring to this along with a proper lab environment for me to learn more from .

 

[Emulex]

you figure out how to do LACP to the switch?

 

[ChrisBenn]

you figure out how to do LACP to the switch?

You having a problem with the solaris/linux side, or the switch side?

 

[Adul]

you figure out how to do LACP to the switch?

I have 2 PCIe intel gigabit nics in the server.

I went into the console at the server, created a new lagg0 interface. I selected the network adapters em0 and em1, set it to LACP. I then configured the interface to be set up as DHCP. My DHCP server is setup to assign a static ip for this server. I only had 1 nic configured prior to doing this.

It does help to have a descent switch. I have a Dell PowerConnect 2624 and PowerConnect 2708 (web managed) gigabit switches. Some switches will not work with LACP.

BTW I was able to max out both ports transferring data to the gf and my pc.

 

[Emulex]

nice. i was just curious how easy it was. seems easy enough. I assume with those switches you did one nic to one switch and one to another? are they on a vlan to seaprate storage since they are jumbo packet? virtual or physical vlan?

 

[randomlinh]

how's FreeNAS these days? Reliable enough for production work? Or at the very least, good enough for backups?

 

[taltamir]

Congrads on your ZFS adul.

 

[Adul]

nice. i was just curious how easy it was. seems easy enough. I assume with those switches you did one nic to one switch and one to another? are they on a vlan to seaprate storage since they are jumbo packet? virtual or physical vlan?

I put them both into the same non managed switch to see if it works. The switch recognizes the LAGG connection and just supports it.

how's FreeNAS these days? Reliable enough for production work? Or at the very least, good enough for backups?

Well I am not sure about production work yet, but so far so good with RC4 now running. I'd say it is ready for testing, but not quite ready for production. It just hasn't been out long enough nor stressed enough to really gauge that yet. It is definitely worth playing with in a VM environment for testing however.

 

[Emulex]

i'd stick with ISCSI - certain older vma's like celerra (Emc) and lefthand (HP) had products that would run without extra special features or just without support. these were enterprise grade software with no support. Things may have changed but googling you will find them. you could run esx - and export the storage to a VM that would do you protocol converting or two. bump the priority up on the VSA. all free.

Production - runs for 700+ days without a hiccup.
anything else - does not.

i'd check out the emc offering if you can its pretty badazz.