Best protocol for simple storage network

[Essence_of_War]

I want to provide fast storage to two clients, A a dual boot Windows/Ubuntu desktop, and B a Mac Pro (currently 2008) running El Capitan that will likely be upgraded to a trash can Mac Pro or iMac in the next year or so, from C, a Linux or FreeBSD box.

What is the best hardware infrastructure to do this?

I'm considering fiber channel, 10gbe, or Infiniband.

FC:
4 and 8 gb HBAs are pretty cheap, Linux and Windows work pretty well with qlogic cards, but I have no idea what the state of FC on OS X is.

10gbe:
Linux and Windows have pretty good support for 10gbe Intel nics, and Xeon-d boards come with integrated 10gbe. Mac Pro currently has only native 1gbe ports and I hav no idea what driver support is like for 10gbe cards, the trash can version requires pricey TB to 10gbe adapters.

IB:
I know nothing about IB support on any platform, but 10 and 20 gb HBAs seem pretty cheap.

 

[eton975]

ATA over ethernet SAN, with 10 GBE.

 

[gea]

Go with SMB 2 and 10 Gbase-T as it is fast, everywhere supported (on Mac Pro you
need a Sanlink2) and switches are become cheaper and cheaper.
With two clients you can connect them drectly to a storage with a dual 10GbE Adapter

See my tests with MacPro and 10 GbE with ZFS and Solaris
https://www.napp-it.org/doc/downloads/performance_smb2.pdf

 

[Essence_of_War]

Go with SMB 2 and 10 Gbase-T as it is fast, everywhere supported (on Mac Pro you
need a Sanlink2) and switches are become cheaper and cheaper.
With two clients you can connect them drectly to a storage with a dual 10GbE Adapter

See my tests with MacPro and 10 GbE with ZFS and Solaris
https://www.napp-it.org/doc/downloads/performance_smb2.pdf

Nice! So you would suggest 10gbe and sharing with both clients over samba? Is there any advantage to using AFP with the Mac client?

 

[dave_the_nerd]

Nice! So you would suggest 10gbe and sharing with both clients over samba? Is there any advantage to using AFP with the Mac client?

I don't think so. You might consider NFS though.

I've done some limited testing with my Macbook Pro, and what I saw was that they all had about the same maximum sequential transfer, but when it came to latency, CIFS was way better than AFP, and NFS was better than CIFS.

None of them are as good as iSCSI, but an OS X iSCSI initiator is money and sharing block level storage is more idiosyncratic than managing a file server running NFS/CIFS.

 

[Essence_of_War]

None of them are as good as iSCSI, but an OS X iSCSI initiator is money and sharing block level storage is more idiosyncratic than managing a file server running NFS/CIFS.

Wow, I didn't realize that OSX didn't have an iSCSI initiator built-in. That seems like a somewhat embarrassing oversight.

The osx client should be able to use Time Machine over CIFS, right?

 

[dave_the_nerd]

Wow, I didn't realize that OSX didn't have an iSCSI initiator built-in. That seems like a somewhat embarrassing oversight.

Consumer OS.

The osx client should be able to use Time Machine over CIFS, right?

Not out of the box. There are hacks/tutorials out there that involve a lot of CLI stuff.

http://rajiv.sg/blog/2012/11/19/con...ion-time-machine-to-work-with-cifs-smb-share/

^^ For instance

 

[Essence_of_War]

Consumer OS.

Yeah, but so is windows. And OS X is still a preferred platform for people like video editors who might like to connect to a high performance block storage device.

 

[dave_the_nerd]

Yeah, but so is windows.

Windows at least pretends to have presence in the enterprise space. I'll admit that I'm a little surprised WinHome leaves the initiator enabled, but I'm not surprised that Windows has it and OS X doesn't.

And OS X is still a preferred platform for people like video editors who might like to connect to a high performance block storage device.

When Apple actually tried selling servers, they used fiber channel. I think that support is still baked in.

If you're a video pro spending $20k+ on a box of hard drives, you'll go to the trouble of installing a third party initiator. If you just want to tinker with iSCSI in your home lab, use Linux.

Also worth noting that, using OS X in a VMware environment is totally doable if you're using Apple branded hardware. In that case, ESX handles the iSCSI stuff and passes volumes through to the vm just like any other virtualized drive. I kinda wonder if that was part of some agreement Apple struck with VMware, since ESX is "smart" enough to enforce Apple's licensing restrictions.

 

[gea]

Nice! So you would suggest 10gbe and sharing with both clients over samba? Is there any advantage to using AFP with the Mac client?

Solaris does not use SAMBA. It comes with its own kernelbased and multithreaded SMB server.

Regarding AFP
The only advantage of AFP is Timemachine support out of the box. But using a ZFS shared storage with snaps for data is far superiour to local HFS+ storage with a Timemachine based copy of files.

Another problem of AFP on Linux/Unix is that netatalk is a constant source of trouble. It is also not compatible to SMB and Windows permissions so the easiest and fastest is SMB2. Even the makers of netatalk switched to SMB just like Apple did.