Question Are we ready for 10 Gbe networks? I just want to chitchat

[iamgenius]

Hi networking gurus.

I'm a guy who just hates slowness

Whenever there is a way to speed things up, I'll just do it. I have been toying with the idea of converting my 1Gb/s LAN into a 10 Gb/s LAN. Of course I will need to change my main router and NIC's in my machines. I save all my data into a NAS device, and this NAS backs up itself to another similar NAS in the same network. Not the very best solution but it works. I also move do a manual backup to a local drive and move files between my machines. I will be limited of course by the maximum transfer rate of my mechanical HDD's and the SATA interface transfer rate. But my nvme ssd's configured in RAID 0 can still utilize the 10 Gbps connection. It is very nice seeing huge files get transferred immediately. Are 10gbe routers/switches readily available? Are there NAS devices with 10 Gbps network connection? When I search, I still don't see them as common. Maybe we are still not ready? But 10 Gbe has been around for some time now.

I won't even mind building a NAS devices based on ssd's, heheh !

Let's chitchat

 

[iamgenius]

In this video:

at about 6:00 he talks about the normal 1Gbps network that provides internet. So, the direct connection between the two machines should just be plug and play and will work right away without any configuration, and will not disrupt the original network, right? It is just when you move files between the directly connected machines, you will see faster speeds, right?

 

[mxnerd]

Test NAS Speed

https://www.youtube.com/results?search_query=test+nas+speed+

In this video:

at about 6:00 he talks about the normal 1Gbps network that provides internet. So, the direct connection between the two machines should just be plug and play and will work right away without any configuration, and will not disrupt the original network, right? It is just when you move files between the directly connected machines, you will see faster speeds, right?

right.

==

Don't think you want to use RAID 0 on your NAS, so single 10Gbe should suffice your need for a while.

 

[iamgenius]

Test NAS Speed

https://www.youtube.com/results?search_query=test+nas+speed+

right.

==

Don't think you want to use RAID 0 on your NAS, so single 10Gbe should suffice your need for a while.

I saw in the videos that it was measured manually by mere file copies. I also saw AJA system test used to measure transfer rate. I download this software. It recognizes network drives, but it is still bottle nicked by my 1gbps network speed. I have to test from within my NAS itself.

And no, I'm not scarifying redundancy for speed. Raid 5 is better for me. No Raid 0.

And thanks again mxnerd.

 

[mxnerd]

There is no way to test 10Gbe performance until you own 10Gbe adapter.

==
But you can calculate disk performance

RAID Performance calculator

RAID Performance Calculator - WintelGuy.com

RAID Performance Calculator - evaluates performance of different RAID types and configurations

wintelguy.com

 

[iamgenius]

There is no way to test 10Gbe performance until you own 10Gbe adapter.

==
But you can calculate disk performance

RAID Performance calculator

RAID Performance Calculator - WintelGuy.com

RAID Performance Calculator - evaluates performance of different RAID types and configurations

wintelguy.com

I don't want to test 10GbE ! How can I do that? I just want to test how fast my NAS (without and with ssd caching) is to see how close I'm to 10GbE speeds. I will check your link.

 

[aigomorla]

Hmm... A Video from more than TwoYears ago is a Good solution today???

jack i have that switch and there is nothing wrong with it besides it heats up like no tommorrow, so you want a fan ontop of it.
It even works great with 10GbaseT transceivers.

The only issue i have is that again it heats up like no tomorrow, so i really recommend the CRS309 instead which has a massive passive heat pipe heatsink, and also offers 8xSFP+ which again works great with transceivers.

In honesty if i ever decide to go up to 10GbaseT i will probably still use a CRS309 and have transcivers on them as compared to other 10gbaseT switches they are a godsend to the ears and noise.

 

[VirtualLarry]

Yeah, I also have that Microtek 4-port 10GbE SFP+ plus 1GbE-T passively-cooled switch. I have a USB Rosewill external 120mm fan pointed at it / on top of it to keep it cool. I also have 4x IPolex 10GbE-T tranceivers, the ones that will step down to 2.5GbE/5GbE-T as well, connected to three 2.5GbE-T workstation NICs, and one a 10GbE-T uplink to the rest of my network cluster, as well as I have a 1GbE-T 8-port switch hanging off of the 1GbE-T port, for other devices to plug into. Works fine for me.

I recently picked up an Asustor Lockerstor 10 (with 2x 10GbE-T and 2x 2.5GbE-T ports, and dual M.2 NVMe caching, 10x 3.5" HDD bays), for only $409 @ Newegg the other day.

 

[mxnerd]

I also have 4x IPolex 10GbE-T tranceivers, the ones that will step down to 2.5GbE/5GbE-T as well, connected to three 2.5GbE-T workstation NICs

This one? https://www.servethehome.com/ipolex-asf-10g-t-and-sfp-10g-t-s-review-10gbase-t-and-nbase-t-adapter/ Nice.

Asustor Lockerstor 10 (with 2x 10GbE-T and 2x 2.5GbE-T ports, and dual M.2 NVMe caching, 10x 3.5" HDD bays), for only $409

That's a steal!

 

[iamgenius]

I recently picked up an Asustor Lockerstor 10 (with 2x 10GbE-T and 2x 2.5GbE-T ports, and dual M.2 NVMe caching, 10x 3.5" HDD bays), for only $409 @ Newegg the other day.

WOW, that' one heck of a NAS, I wish I didn't have mine. I like the specs. Great price also.

 

[iamgenius]

Forgot to ask something. If I copy a huge file (say 100 GB file) and paste it somewhere else in the same NAS? How much indicative will that be of its maximum transfer rate? I get 200 MB/s .

I guess that's a way to measure my NAS current speed.

 

[mxnerd]

The RAID calculator already gives you maximum possible speed (single huge file sequential read. (not 100 files, which slows down everything because random files are located in different locations on the disk/array ), you just translate it into network speed with 10bit encoding. So 300MB/s = 3000mbps (1 byte on disk needs to be encoded into 10 bits for ethernet ).

Look up your disk's spec sheet. My chitchat ends here.

 

[aigomorla]

1 byte on disk needs to be encoded into 10 bits for ethernet

i thought it was always 1 byte per 8bits no?

 

[mxnerd]

i thought it was always 1 byte per 8bits no?

Yep. 1 byte is 8 bits.
8b/10b encoding for fiber.

Don't know if it's also true for copper. I could be very wrong. Not that familiar with encoding.

==

OK, 1GBase-T use 8B1Q4 and 4D-PAM5 encoding. No idea what they are.

 

[iamgenius]

The RAID calculator already gives you maximum possible speed (single huge file sequential read. (not 100 files, which slows down everything because random files are located in different locations on the disk/array ), you just translate it into network speed with 10bit encoding. So 300MB/s = 3000mbps (1 byte on disk needs to be encoded into 10 bits for ethernet ).

Look up your disk's spec sheet. My chitchat ends here.

I actually wanted to measure it. Not calculate it. I got 340 MB/s when I used your link. But what exactly is the percentage of read operations? It greatly affects the result but I don't know what it is.

 

[mxnerd]

You don't have 10Gbe link, how can you test? Calculated result of 340MB/s of course is 100% for reading. HDD spec sheet always give you reading speed. Reading data does not change disk's electric field, writing does, and always will be slower. 340MB/s is way over 1Gbps.

If you are not ready for 10Gbe, buy a used quad port Gbe ethernet card on eBay for your PC. Also buy a managed switch (with LACP) so you can do network teaming with 4 ethernet cables and create 4Gbe channel, or use Microsoft SMB 3.0 multichannel with unmanaged switch. Don't know if your Synology supports SMB 3.0 yet, however.

==

for reference

https://forums.anandtech.com/threads/dual-gbe-nics-that-support-channel-bonding-in-win10.2589053/https://forums.anandtech.com/threads/dual-gbe-nics-that-support-channel-bonding-in-win10.2589053/

 

[iamgenius]

You don't have 10Gbe link, how can you test? Calculated result of 340MB/s of course is for reading. Reading data does not change disk's electric field, writing does, and always will be slower. 340MB/s is way over 1Gbps.

If you are not ready for 10Gbe, buy a used quad port Gbe ethernet card on eBay for your PC. Also buy a managed switch so you can do network teaming with 4 ethernet cables and create 4Gbe channel, or use Microsoft SMB 3.0 multichannel with unmanaged switch. Don't know if your Synology support SMB 3.0 yet, however.

==

for reference

https://forums.anandtech.com/threads/dual-gbe-nics-that-support-channel-bonding-in-win10.2589053/https://forums.anandtech.com/threads/dual-gbe-nics-that-support-channel-bonding-in-win10.2589053/

Sometimes I think I don't know how to explain myself at all. Forget the 10 GbE link. I want to measure the speed of my NAS. If I log into my NAS OS, load a program that measures transfer rate(Something similar to HD tune for windows), that should give me my NAS current speed. If I measure it from within windows (Accessing it via the network), I will be bottle nicked by the 1Gbps link. In fact, when I use AJA system test I get maximum speed of 115 Mb/s which is simply 1Gbps (the network link limit). That's because the program I think copies a file from the NAS to a local drive nad/or vice versa. Anyways, like I said before: when I copied one huge file and pasted it in the same location I got close to 220 MB/s which is close to 2Gbps. I don't know how much improvement will ssd caching do to it. If my NAS can do more than 1Gbps then I definitely can benefit from going 10Gbps using sfp+ which is not going to be expensive since there is no switch required.

If this is not clear, then I'm too stupid to explain anything!

 

[mxnerd]

I don't have NAS and don't have a real network. I do everything inside my VMware workstation VM's.

SSD caching does help if you use the same network file from NAS, other than that, I don't think it helps much.

I also couldn't find Synology APP that can test RAID speed locally.

There is no doubt that your NAS RAID speed already surpass 1Gbps, however,

don't know why you kept questioning it? And why don't you just give your HDD spec and your RAID setup?

 

[iamgenius]

I have got another question ! To avoid using the switch, and to be able to use that E10M20-T1 above for ssd caching, can I use a media converter to convert it from RJ-45 to sfp+? More devices but it should work, right?

Edit: Oh wait what am I thinking ! I should just go Cat6 or Cat7 direct connection (crossover cable) between my main Machine and NAS If I want to use that card. Why convert. I just got back from doing my PT so I guess my head is till turning. Never mind.

 

[iamgenius]

I don't have NAS and don't have a real network. I do everything inside my VMware workstation VM's.

SSD caching does help if you use the same network file from NAS, other than that, I don't think it helps much.

I also couldn't find Synology APP that can test RAID speed locally.

There is no doubt that your NAS RAID speed already surpass 1Gbps, however,

don't know why you kept questioning it? And why don't you just give your HDD spec and your RAID setup?

Sorry I didn't notice your question. I have 5 WD RED 8TB SATA HARD DRIVE (WD80EFAX) in RAID 5 . I get 340 MB /s when I use 50% read operations using the link your provided. If I use 20% I get 250 MB/s. If I use 100% I get 850 MB/s !@#!@? I don't understand this read operations thing and how it greatly affects the resulting speed.

 

[mxnerd]

Well, you better ask the NAS performance question at the Memory & Storage subforum. I'm not good at the subject.

 

[iamgenius]

You have been great help. I thank you very much. I will do ask there.

 

[aigomorla]

I have 5 WD RED 8TB SATA HARD DRIVE (WD80EFAX) in RAID 5

NO NO NO NO NEVER RAID 5 unless its SSDs because of URE.

Michael Holtstrom

holtstrom.com

 

[iamgenius]

NO NO NO NO NEVER RAID 5 unless its SSDs because of URE.

Michael Holtstrom

holtstrom.com

Never heard of it. I have been using RAID 5 since 2013 with no problems. I had one disk fail one time and was able to rebuild the volume. In synology, they almost always recommend using SHR which is essentially RAID 5. But, that's scary news, I will have a look.

 

[KentState]

Never heard of it. I have been using RAID 5 since 2013 with no problems. I had one disk fail one time and was able to rebuild the volume. In synology, they almost always recommend using SHR which is essentially RAID 5. But, that's scary news, I will have a look.

The risk you take is that during a rebuild of RAID5, all of the disks are pounded, trying to recalculate the parity. Each block has to be read from all disks to do that calculation. With RAID 6, the parity is stored twice so the reads are just for copying of the parity itself. Much less stress being put on the volume and a less chance of a second disk failure which results in a failure of the array. I always regretted doing just RAID 5 when I had my Synology and never had a good way to redo it.

I'm now using freenas on two servers and use RAIDZ2 vdevs. Of my 24 disk enclosure, I do give up 8 drives but still worth it in my opinion.

 

[iamgenius]

Sorry for the many questions, but I think I changed my mind a bit. Yes my NAS speed is faster than 1Gbps but it is not much faster, and since I mainly want the transfer rate to be fast between my main machine and my main NAS device, I think the wisest option is to just to add an additional NIC to my main machine and setup link aggregation between the PC and the NAS. I will need a switch that supports link aggregation. I can buy a used one for that. Now that I think about it, SSD caching may not benefit me much. I will only be able to increase my NAS speed very much if I invest heavily on SSD's.

Right? What do you think? Is it easy to setup? Is it possible to run a cat6 cable directly between the pc and the NAS, or should both ports on the pc and the NAS go to the switch (4 ports total) and then I should bond the two links?