Question re-doing my network as of Dec. 2020. (2.5GbE is out, 10GbE is in, 10GbE WAN/LAN router, NAS, and more)

[VirtualLarry]

Discovered some new "Toys" have hit the market, time to upgrade! More info soon.

 

[aigomorla]

lol...

larry lemme throw something else in your bucket if you already do not have one....

Pi-hole...

Pi-hole – Network-wide Ad Blocking

pi-hole.net

Absolutely love it... only thing i regret is using a 8GB RSBPi4+
It would of done fine on a 2GB version even.

But DNS lvl adblocking is awesome especially on phones and tablets which some do not support native adblocking without jail breaking the device.

 

[mxnerd]

lol...

larry lemme throw something else in your bucket if you already do not have one....

Pi-hole...

Pi-hole – Network-wide Ad Blocking

pi-hole.net

Absolutely love it... only thing i regret is using a 8GB RSBPi4+
It would of done fine on a 2GB version even.

But DNS lvl adblocking is awesome especially on phones and tablets which some do not support native adblocking without jail breaking the device.

You can install Diet-Pi OS distro (text based UI) on your Raspberry Pi and run many things on it, including Pi-Hole, and not wasting your 8GB RAM.

Lightweight justice for your SBC!

Optimised | Simplified | For everyone - Backed by community, DietPi is a minimal OS image for SBCs - Raspberry Pi, Odroid, PINE64 etc. Install software optimised for you!

dietpi.com

 

[VirtualLarry]

updated OP. Should I check out and get those items?

I've got:
1) QNAP dual 10GbE-T / quad 1GbE-T SD-WAN Wifi 6 router
2) Microtik first primarily 10GbE-T (not SFP+) switch, which has 8 10GbE-T ports, and 4 10GbE-T/SFP+ combo ports, for what I consider to be a reasonable price for a 12-port 10GbE-T switch. I believe that it's managed as well.
3) Recent QNAP NAS unit, that is based on their ARM (Annapourna Labs) quad-core 1.7Ghz SoC, with 5x 3.5" HDD bays, and 4x 2.5" (SATA SSD) bays, with dual 10GbE (SFP+, sadly), and dual 2.5GbE-T. (Or should I get an Asustor LockerStor 8, and utilize it's dual M.2 NVMe caching slots instead, and 4 or 8x 3.5" HDD bays? At least it would have an Intel-based SoC/CPU, so I could run Windows VMs.)

Edit: Hold up, this one looks more interesting.

QNAP TS-h973AX | QNAPWorks.com

www.qnapworks.com

TS-h973AX | Enhance business productivity using the quad-core 9-bay QuTS hero NAS that supports U.2 NVMe SSD and 10GbE/2.5GbE connectivity

Featuring an AMD Ryzen™ V1000 series V1500B quad-core processor and 10GbE/2.5GbE connectivity, the 9-bay TS-h973AX packs a high-capacity hybrid storage infrastructure into a compact physical footprint. With five 3.5-inch SATA 6Gbps drive bays, two 2.5-inch U.2 NVMe SSD slots (Slot 1 & 2 support...

www.qnap.com

Edit: I did go with the ZFS/QuHero/Ryzen embedded NAS unit with the 32GB of DDR4 RAM, for nearly $1000. It takes U.2 NVMe 2.5" SSDs in slots 1 and 2 for caching! (You'd like that one, @aigomorla !)

I also ordered the QNAP dual 10GbE-T Wifi 6 router, finally, I can have a router with real 10GbE-T/multi-gig both LAN and WAN, ready for FIOS to upgrade to 2+Gbit/sec!

I held off on the 12-port 10GbE-T Microtik switch for now, I'm going to use my existing LAN switch infrastructure, maybe next month, maybe prices will go down.

 

[aigomorla]

https://www.amazon.com/Cloud-Router-Switch-317-1G-16S-RM/dp/B0747TC9DB/ref=sr_1_5?crid=VRZJQC9AF7I1&dchild=1&keywords=mikrotik+10gbe+switch&qid=1606995367&sprefix=mirkriotic%2Caps%2C197&sr=8-5

set it on SwitchOS. I think that is a much better switch and its also passive on top.

but why do you need so many 10gbe SFP+ ports?

This one i use personally:

https://www.amazon.com/MikroTik-Desktop-Gigabit-Ethernet-CRS309-1G-8S/dp/B07NFXN4SS/ref=sr_1_2?crid=I9CXT7T451L3&dchild=1&keywords=crs309&qid=1606995602&s=electronics&sprefix=CRS3%2Celectronics%2C197&sr=1-2

It gives you 8 ports.
Each port is 10gbe, so its about 8 devices connected to each other under 10gbe sfp+.

If you need SFP+ to be a 10gbe RJ45:

https://www.amazon.com/gp/product/B06XQBFHNL/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

I have gotten mine to work with those. But i am not sure about durability. Also i ran it only at a very short distance of 6ft.

 

[VirtualLarry]

I much prefer -T to SFP+.

 

[HardWarrior]

Looks like you're in for a load of networking fun, Larry. My offer still stands, if you need some added energy figuring something out, let me know.

 

[aigomorla]

I much prefer -T to SFP+.

Honestly IMO id still probably run the CRS309 and load all 8 bays with 40 dollar 10GB-T transceivers just on the raw fact its passive with a massive heat sink, and has no roar like other 10gb-T switches.

I am sort of sick and done with loud switches.

 

[VirtualLarry]

Well, fitting out the tranceivers for 12 ports, is 12x $40, or $480 alone for trancievers, might as well just get the $625 switch. I don't use fiber, other than my FIOS WAN. Plus, with those trancievers (they get HOT!), just because the router/switch is passive, isn't such a great advantage IMHO. I "have to" (for peace of mind) run a rosewill USB external fan on my 4-port 10GbE Microtik. So if you're going to run a fan anyways... plus the 8-port 2.5GbE-T D-Link monster switch that that 12-port 10GbE-T Microtik would be replacing, already has semi-loud fans (when it's running, like in the summer when it gets hot), so nothing new in that dept., either.

Edit: I guess what I'm saying is, we have different needs. I need the -T ports, and don't mind the fans (I've got mining rigs and exhaust fans going all the time anyways, I prefer the drone of fans, although the smaller ones on a switch like this one could get noisy, the D-Link ones do on a hot day.)

You prefer silence from your switches (probably after going deaf using "enterprise gear" for a few years at home), and don't mind paying for the tranceivers.

Edit: Sigh, they had trouble charging my card, so this plan is ON HOLD for the time being...

 

[Fallen Kell]

Welcome to the 10Gbe club... But once you have a taste of it, you will say to yourself, why didn't I join the 40Gbe club

That said, the 40Gbe club needs to be able to put their switch(es) in a basement or somewhere else since most are pretty loud (mine is about 2x as loud as my managed 24port Gbe switch was).

I did the math and realized I could pickup a Brocade ICX-6610-24 refurb'ed for $200 (which has 4xQSFP+ ports, 8xSFP+, and 24Gbe ports) and is a full layer 3 router/switch, and get any transceivers I need for 10Gbase-t (which I only need 3) for much cheaper than the $600+ 10Gbase-T switches (and bonus is that this is a much better network core as it is full wire speed routing on all ports). My storage/VM server is connected via 40Gb QSFP+ DAC and my wifi access point is connected via 10Gb SFP+ DAC. My edge router is currently setup as a router-on-a-stick using a QSFP+ DAC (i.e. it only has a single connection to the physical network, but uses VLANs to route between the internet and the rest of my internal network). The edge router does have an additional QSFP+ port available (so if in the future an ISP provides QSFP+ 40Gbe to the home, I am ready, or I can continue to use it in the current configuration if/when the ISP offers 2xGbe, or 10Gbe connections), but it will only be capable of ~60Gbe bi-directional total throughput due to PCI-E bus limitations (I'm not sure it's CPUs would be able to handle that much traffic anyway as it is a pfsense system).

Trade off is the network is a little loud, and definitely not a consumer friendly interface (the 6610 runs fastiron, which is very similar to CISCO IOS). I have all my VLANs defined, but I am still working on setting up proper ACL rules for segregating certain VLANs from others (i.e. for guest VLAN and IoT VLAN).

 

[HardWarrior]

40Gbe! My mouth is watering...

 

[Fallen Kell]

Yeah, as stated, I got in cheap because the enterprise grade stuff for 40Gbe is all getting dumped as they are all upgrading to SFP28/QSFP28/QSFP-DD for 25/100/400Gbe networks. The network cards I got for 40Gbe were only $29 (again, refurbed, but still, that is a I don't care if it runs into issues price, I will just get another).

 

[aigomorla]

gah i don't even think my ramdrive could max out a 40gbe.
I really just am tired of having loud switches... my Supermicro Racks make enough noise with the ultra loud 80mm nidac or sanace fans.

The last switch i had was a Quanta Lb6m and well the CRS309 only having 1/3rd of the ports which is all i need, is a god send because its passive.
You can just see how massive the passive heatsink is on the CRS309

MikroTik CRS309-1G-8S+IN Review Inexpensive 8x 10GbE Switch

In our MikroTik CRS309-1G-8S+IN review, we show what this low-cost, low-power, and silent 8x 10GbE SFP+ switch offers and when it can be used

www.servethehome.com

So Larry i have no need / reason / issue to put a fan on top of it like you would on a CRS305, which i have one of those as well, and like you also have a cheap fan on.

 

[sdifox]

Larry is there a reason you are not running one server instead of a whole bunch of single function boxes?

 

[VirtualLarry]

Larry is there a reason you are not running one server instead of a whole bunch of single function boxes?

I don't get it. Router, Switch, NAS. How would you combine them? (I think that I may have posted an idea for combining a PFsense router, with a FreeNAS NAS, somehow, but I'm not aware of any actual "combined" distros.)

 

[sdifox]

I don't get it. Router, Switch, NAS. How would you combine them? (I think that I may have posted an idea for combining a PFsense router, with a FreeNAS NAS, somehow, but I'm not aware of any actual "combined" distros.)

switch is the only thing you cannot vm. a server can do file serving and firewall fine.

 

[Fallen Kell]

I don't get it. Router, Switch, NAS. How would you combine them? (I think that I may have posted an idea for combining a PFsense router, with a FreeNAS NAS, somehow, but I'm not aware of any actual "combined" distros.)

You combine them by having a beefy enough server that you can setup multiple virtual machines for the specific duty. That said, I am not a big fan of having your edge router (like pfsense) run in a VM. If it is compromised, it is possible to find a method to break out of the VM to the hypervisor and access any other VMs on the server. If it is a separate device, you have more protection because your server's internal firewall can help block the intrusion.

 

[ElFenix]

lol...

larry lemme throw something else in your bucket if you already do not have one....

Pi-hole...

Pi-hole – Network-wide Ad Blocking

pi-hole.net

Absolutely love it... only thing i regret is using a 8GB RSBPi4+
It would of done fine on a 2GB version even.

But DNS lvl adblocking is awesome especially on phones and tablets which some do not support native adblocking without jail breaking the device.

remember to fix your smart tv as well:

Your Smart TV is probably ignoring your PiHole - LabZilla

Welcome Hacker News readers!• Thank you to M. Hammad Mazhar for his research that inspired this guide.• @healyio made some great additional suggestions in th...

labzilla.io

 

[mxnerd]

You just choose which part you want to virtualize. Switch is the part you can not totally virtualize like sdifox has said.

Home | Unraid Docs

Unraid Documentation

wiki.unraid.net

Unraid

forums.unraid.net

r/unRAID

r/unRAID:

www.reddit.com

https://www.youtube.com/results?search_query=unraid+NAS+AIO

==

ESXi is a popular alternative.

 

[sdifox]

You combine them by having a beefy enough server that you can setup multiple virtual machines for the specific duty. That said, I am not a big fan of having your edge router (like pfsense) run in a VM. If it is compromised, it is possible to find a method to break out of the VM to the hypervisor and access any other VMs on the server. If it is a separate device, you have more protection because your server's internal firewall can help block the intrusion.

That would be quite impressive since each vm runs in isolation mode, meaning they all have their own kernel and I use two dedicated nic ports for pfSense. Of course it is not theoretically as secure as having a dedicated hardware firewall, but it is not that distinct either. It's a homelab.

I don't need more than gigabit network since I just have one host and vm comm is in memory. And I am not threading a fiber up to second floor from the basement lol.

 

[mxnerd]

Don't know why VL needs so much bandwidth? Cloud / offsite backups?

 

[sdifox]

Don't know why VL needs so much bandwidth? Cloud / offsite backups?

it's more of a just because.

 

[Fallen Kell]

That would be quite impressive since each vm runs in isolation mode, meaning they all have their own kernel and I use two dedicated nic ports for pfSense. Of course it is not theoretically as secure as having a dedicated hardware firewall, but it is not that distinct either. It's a homelab.

I don't need more than gigabit network since I just have one host and vm comm is in memory. And I am not threading a fiber up to second floor from the basement lol.

One of the first known exploit of VM to hypervisor was in 2008. There have been many more vulnerabilities found since then. See the following wiki page about such an attack and many of the vulnerabilities used to perform such an attack listed below:

Virtual machine escape - Wikipedia

en.wikipedia.org

If you look, you will notice that many of these rely on using any kind of shared bus/hardware between the VM and hypervisor in order to exploit typically a memory vulnerability to inject code to run onto the hypervisor.

 

[sdifox]

One of the first known exploit of VM to hypervisor was in 2008. There have been many more vulnerabilities found since then. See the following wiki page about such an attack and many of the vulnerabilities used to perform such an attack listed below:

Virtual machine escape - Wikipedia

en.wikipedia.org

If you look, you will notice that many of these rely on using any kind of shared bus/hardware between the VM and hypervisor in order to exploit typically a memory vulnerability to inject code to run onto the hypervisor.

yeah I have seen that list. like I said I don't Share anything from the fpSense vm, dedicated memory and nics.

 

[Fallen Kell]

yeah I have seen that list. like I said I don't Share anything from the fpSense vm, dedicated memory and nics.

Did you include removing all VGA/graphics/console devices from the VM as well? The virtualized graphics layer has been a huge hole as of late, with any level of being able to obtain graphics from within the VM seemingly a vulnerability. Dedicated hardware passthrough seems to be the only thing that is not as exploitable (where-in the hypervisor does not use the hardware in any way).