• Guest, The rules for the P & N subforum have been updated to prohibit "ad hominem" or personal attacks against other posters. See the full details in the post "Politics and News Rules & Guidelines."

Question 2 Separate LANs on the same hardware - how?

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

mxnerd

Diamond Member
Jul 6, 2007
6,152
887
126
If you learned VLAN (I have a hard time), you don't need a lot of LAN ports. pfsense even works with just one port with VLAN (router on a stick, mentioned by @Fallen Kell ).

No need to rush out to buy QOTOM or Protectili, test on your leftover PCs with a quad port gigabit NIC first then make your decision later.

 

bob4432

Lifer
Sep 6, 2003
11,647
19
81
Thanks again for the assistance. I read about the "router on a stick" and I have not learned VLANS yet as I just learned I needed this setup less than a day ago, but appreciate your faith in me 👍😀.

I was planning on using the i3 to run 24/7 for some serving duties for some gaming I am planning on getting back into, so I am going to order one of those quad nic cards, I just wanted to know if for any reason I would ever need 2 of them?

Also, I was wrong on the chip, it is a i3-10100F and does support AES-NI.

Off to watching that YouTube video you linked to.

Appreciate the support, I have gained much knowledge today and want to thank all of you for the assistance 👍
 

ch33zw1z

Lifer
Nov 4, 2004
33,969
12,808
146
Thanks again for the assistance. I read about the "router on a stick" and I have not learned VLANS yet as I just learned I needed this setup less than a day ago, but appreciate your faith in me 👍😀.

I was planning on using the i3 to run 24/7 for some serving duties for some gaming I am planning on getting back into, so I am going to order one of those quad nic cards, I just wanted to know if for any reason I would ever need 2 of them?

Also, I was wrong on the chip, it is a i3-10100F and does support AES-NI.

Off to watching that YouTube video you linked to.

Appreciate the support, I have gained much knowledge today and want to thank all of you for the assistance 👍
VLAN's aren't terribly hard, but it will take a bit to wrap your head around it. I found drawing things on paper is a great way to design your network. It's really just learning where you want the traffic to flow and how to make it happen (and how to prevent traffic between networks)

VLAN'S (aka Virtual LAN's) is a way to control traffic logically and reducing the need to have tons of hardware otherwise. It's much more flexible than the classic straight subnetting as well.

VLAN traffic is tagged traffic. It's important to remember this, especially if you plan to extend your network with unmanaged switches. Unmanaged switches cannot pass tagged traffic, but can still be used to extend a VLAN from a managed switch. I do this to extend LAN ports on my main network. If you want to extend your network while passing ALL VLAN tagged traffic, a managed switch will be required and a trunk port configured.

*Note: all managed gear I work with has a default VLAN ID of 1. This tends to be referred to as the "native" VLAN or "management" VLAN. If I look at my UDM PRO, my "main" network (called LAN, it's the default network), doesn't actually have a VLAN ID configured, but under the covers it has ID 1. When I configure the isolation firewall rules, I use ID 1.

You will find different vendors don't necessarily call these things the same name, but it's really the same thing. Try to just understand the basics and don't get stuck on too much lingo, and use google to narrow it down if needed.

Here's a very basic description from a quick DDG search: https://geek-university.com/ccna/vlans-explained/

Anyways, I drew a quick network diagram that may help explain. Hardware wise, it's only 3 devices (plus 1 unmanaged switch), but it is quite flexible.

*side note, every so often I have to recert a Cisco test so I'll swap a switch port on the UDM Pro to a trunk port and extend the network to a Cisco managed switch I have in order to brush up on the cli. It's kinda fun cause I have 48 more managed ports to play with. IIRC, I left the Cisco switch with 8 ports grouped together for each VLAN.

1633778026778.png

Keep in mind, this is very broad, but the basics here will apply to ALL brands that adhere to IEEE standards, anything recommended in this thread will do that.

*Note, I'm not trying to push Ubiquiti here, it's just what I've gone with. The other options presented here are good as well. @Fallen Kell and @mxnerd know their stuff :)
 
Last edited:
  • Like
Reactions: mxnerd

bob4432

Lifer
Sep 6, 2003
11,647
19
81
VLAN's aren't terribly hard, but it will take a bit to wrap your head around it. I found drawing things on paper is a great way to design your network. It's really just learning where you want the traffic to flow and how to make it happen (and how to prevent traffic between networks)

VLAN'S (aka Virtual LAN's) is a way to control traffic logically and reducing the need to have tons of hardware otherwise. It's much more flexible than the classic straight subnetting as well.

VLAN traffic is tagged traffic. It's important to remember this, especially if you plan to extend your network with unmanaged switches. Unmanaged switches cannot pass tagged traffic, but can still be used to extend a VLAN from a managed switch. I do this to extend LAN ports on my main network. If you want to extend your network while passing ALL VLAN tagged traffic, a managed switch will be required and a trunk port configured.

*Note: all managed gear I work with has a default VLAN ID of 1. This tends to be referred to as the "native" VLAN or "management" VLAN. If I look at my UDM PRO, my "main" network (called LAN, it's the default network), doesn't actually have a VLAN ID configured, but under the covers it has ID 1. When I configure the isolation firewall rules, I use ID 1.

You will find different vendors don't necessarily call these things the same name, but it's really the same thing. Try to just understand the basics and don't get stuck on too much lingo, and use google to narrow it down if needed.

Here's a very basic description from a quick DDG search: https://geek-university.com/ccna/vlans-explained/

Anyways, I drew a quick network diagram that may help explain. Hardware wise, it's only 3 devices (plus 1 unmanaged switch), but it is quite flexible.

*side note, every so often I have to recert a Cisco test so I'll swap a switch port on the UDM Pro to a trunk port and extend the network to a Cisco managed switch I have in order to brush up on the cli. It's kinda fun cause I have 48 more managed ports to play with. IIRC, I left the Cisco switch with 8 ports grouped together for each VLAN.

View attachment 51147

Keep in mind, this is very broad, but the basics here will apply to ALL brands that adhere to IEEE standards, anything recommended in this thread will do that.

*Note, I'm not trying to push Ubiquiti here, it's just what I've gone with. The other options presented here are good as well. @Fallen Kell and @mxnerd know their stuff :)
Thanks so much for the drawings and further education. I will keep an eye out for a small managed switch I can pick up for a good $$. I have seen the CISCO SB200 for decent prices and it was in one of the YT videos in this thread, so that will help me too.

Sent from my moto g power using Tapatalk
 

Fallen Kell

Diamond Member
Oct 9, 1999
5,663
250
126
I would say do some quick reading up on Brocade switches (ICX 6450, 6610, 6650, 7150, 7250) which you can get quite a few for cheap on ebay. Just recognize that they will be louder than your home switches. Most companies are ditching many of these as some were 10gbe/40gbe and the new tech is now 25gbe/100gbe/400gbe and not compatible.
 

bob4432

Lifer
Sep 6, 2003
11,647
19
81
I would say do some quick reading up on Brocade switches which you can get quite a few for cheap on ebay. Just recognize that they will be louder than your home switches. Most companies are ditching many of these as they were 10gbe/40gbe and the new tech is now 25gbe/100gbe/400gbe and not compatible.
Thanks for the info. A LOOONG time ago I a had a very large Cisco router that a friend got from a company that was no longer needing it and it sounded like a few hair dryers on @ the same. I think it 5-10k rpm 80mm fans, a lot of them so I know the noise. Hell moving to managed switches ma make my move to GbE+ faster and sooner than I thought.

Also, after I do get the 2 new AX routers up & running, between the old "b/g", "n" & "ac" routers I will have 4 to put dd-wrt on. I am thinking hopefully 1 will have a firmware with VLANs and it can't hurt to learn this stuff since IOT is not going anywhere.

Can you tell me the minimum of the Brocade family of switches I need for what I am wanting to accomplish? Thanks so much in advance.
Bob

Sent from my moto g power using Tapatalk
 

bob4432

Lifer
Sep 6, 2003
11,647
19
81
In an effort of saving $ (since I was not planning the computer builds nor the AX routers) could I get away with a dual port Intel card and a single port Intel card (all pci-e)? I don't know the model #s off the top of my head but will post when I know them.
The machine that will the main rig that will be running PFSense as a VM is based on this motherboard.
 

bob4432

Lifer
Sep 6, 2003
11,647
19
81
In an effort of saving $ (since I was not planning the computer builds nor the AX routers) could I get away with a dual port Intel card and a single port Intel card (all pci-e)? I don't know the model #s off the top of my head but will post when I know them.
The machine that will the main rig that will be running PFSense as a VM is based on this motherboard.
I guess I am asking if I am making things harder to run PFSense as a VM when I have the extra hardware to run it as a dedicated machine? I will understand the $ savings, but my sanity is worth $ too.

Sent from my moto g power using Tapatalk
 

mxnerd

Diamond Member
Jul 6, 2007
6,152
887
126
Like I said earlier, use the extra PC you have , add a quad port PCIE NIC and run pfsense on the physical machine first. Until you are comfortable with the whole setup, later then start thinking whether move to VM.

Just don't understand why you are so against quad port NIC? I don't know if a dual port NIC + 1 port NIC will be cheaper than 30 bucks.
If you don't need the quad NIC anymore, I believe you can still sell it for $30 or even higher in the future.
And that NC365T uses just 5 watts https://www.hpe.com/psnow/doc/c04111679?jumpid=in_lit-psnow-red (page 13)
 
Last edited:

bob4432

Lifer
Sep 6, 2003
11,647
19
81
Like I said earlier, use the extra PC you have , add a quad port PCIE NIC and run pfsense on the physical machine first. Until you are comfortable with the whole setup, later then start thinking whether move to VM.

Just don't understand why you are so against quad port NIC? I don't know if 2 dual port NIC + 1 port NIC will be cheaper than 30 bucks.
And that NC365T uses just 5 watts https://www.hpe.com/psnow/doc/c04111679?jumpid=in_lit-psnow-red (page 13)
I am not against the quad NIC, it is just I have put out a lot of $$ I was not expecting to put out - usually with stuff like this I am in kind of an always updating cycle, but I got out of computers for 5-6 years which led to stuff not getting upgraded and now it has all happened at once, so I am a bit sticker shocked best I can describe it.

I do appreciate all the info you all have shared with me - I have a belief that something I was not sure could be done is now going to happen completely.

Again, I appreciate all of info you have bestowed on me .

Will get a quad NIC inbound .

Sent from my moto g power using Tapatalk
 

Fallen Kell

Diamond Member
Oct 9, 1999
5,663
250
126
You can easily get away with using a Brocade ICX 6450 (which can usually be found for about $100 on ebay). It just won't have much 10Gbe with only 4 SFP+ ports. That said, if you don't have much (if any) 10Gbe devices now, then this would probably be all you need. The benefit of this model is that it is relatively quiet, and can be made very quiet by modding the fans. I went with the ICX 6610, but I have a small ~20u rack in my basement where it is away from most people being able to hear it (the models with 40Gbe are definitely loud).

The 6450 won't support some protocols that the 6610 will (BPG, VFRs, tunnels, hardware encryption on the SFP+ ports, and a few other features that are overkill for a home network but are used by ISPs trying to isolate multiple customers from each other while using the shared hardware). The nice thing about the Brocades is that they pretty much work with just about any kind of transceivers and DACs. I havn't heard of any SFP+ transceiver that didn't work (including using the 1Gbe/2.5Gbe/5Gbe/10Gbe NBaseT transceivers even though these switches came out years before those transceivers existed).
 
Last edited:

bob4432

Lifer
Sep 6, 2003
11,647
19
81
You can easily get away with using a Brocade ICX 6450 (which can usually be found for about $100 on ebay). It just won't have much 10Gbe with only 4 SFP+ ports. That said, if you don't have much (if any) 10Gbe devices now, then this would probably be all you need. The benefit of this model is that it is relatively quiet, and can be made very quiet by modding the fans. I went with the ICX 6610, but I have a small ~20u rack in my basement where it is away from most people being able to hear it (the models with 40Gbe are definitely loud).

The 6450 won't support some protocols that the 6610 will (BPG, VFRs, tunnels, hardware encryption on the SFP+ ports, and a few other features that are overkill for a home network but are used by ISPs trying to isolate multiple customers from each other while using the shared hardware). The nice thing about the Brocades is that they pretty much work with just about any kind of transceivers and DACs. I havn't heard of any SFP+ transceiver that didn't work (including using the 1Gbe/2.5Gbe/5Gbe/10Gbe NBaseT transceivers even though these switches came out years before those transceivers existed).
Appreciate the info, will be keeping an eye out


Sent from my moto g power using Tapatalk
 

bob4432

Lifer
Sep 6, 2003
11,647
19
81
Can I get an approximate size of pfsense & pfblockerng all setup and configured, along with any other software I may want/need (this will be a dedicated appliance just for pfsense, pfblockerng & whatever else you guys would suggest to enhance this setup) so I know which SSD size I need? I have multiple sizes relaxing on a bench, just figured I would start pulling parts.

This will be on the z77 m/b + i5-2500k (AES-NI = yes) w/ 8 or 16GB - does moving too 16GB help in any areas or is it just wasted? FWIW, it is just my wife & I here and between the 2 of us there will be ~20 devices between laptops, desktops, phones & IoT devices, plus I want to make sure I have plenty of headroom for more IoT devices.

I have read this thread a few times - lots of good info, thanks in advance for all the help.
Bob

Getting excited for the NIC to get here, going into new territory is always fun :D.
 

ch33zw1z

Lifer
Nov 4, 2004
33,969
12,808
146
Don't over think it dude, 8GB is probably enough, but you can upgrade later if it needs it. The best part about rolling your own is fiddling with it. I'd say 128GB SSD is plenty
 

Fallen Kell

Diamond Member
Oct 9, 1999
5,663
250
126
Yeah, a 128GB SSD (or even smaller) is perfectly fine for pfsense (unless you want to do full packet capture/logging).

My pFsense system is a Dell Optiplex 9020 SFF with a i7-4790 and 16GB RAM and it is complete overkill for at least the things I am doing, which seems to be very similar to what you are planning (basic edge routing, DNS+pfblockerng for basic IDS, filtering ads/malware/adware/scams). It was just too good of a deal to pass on at the ~$200 I spent for it.

That said, I am still running an older version of pfsense, and I have not upgraded to the current version (I have been debating if I want to do so or bite the bullet and switch to another router OS due to the changes pfsense has been making away from the opensource community into a more proprietary OS).
 
Last edited:

bob4432

Lifer
Sep 6, 2003
11,647
19
81
Yeah, a 128GB SSD (or even smaller) is perfectly fine for pfsense (unless you want to do full packet capture/logging).

My pFsense system is a Dell Optiplex 9020 SFF with a i7-4790 and 16GB RAM and it is complete overkill for at least the things I am doing, which seems to be very similar to what you are planning (basic edge routing, DNS+pfblockerng for basic IDS, filtering ads/malware/adware/scams). It was just too good of a deal to pass on at the ~$200 I spent for it.

That said, I am still running an older version of pfsense, and I have not upgraded to the current version (I have been debating if I want to do so or bite the bullet and switch to another router OS due to the changes pfsense has been making away from the opensource community into a more proprietary OS).
Yes, we do seem to have very familiar desires in what we are wanting to accomplish with this setup.

I found an old Intel 520 120GB SSD that will be my go-to drive. I think I will just build it with the parts I have and let her go. I did find a i7-3770 last night for cheap but if I am pretty close to overkill, I passed on it because this needs to a net 0 build, which it will be (unless the guy guy eith the i7 will deliver.....I kid, I kid...).

Plus I could pick up a Cisco SG200 as I know they will be supported only through some time next year, or for a bit more a SG250 and that is where I think for the foreseeable future is where I need to put my $$ - 10Gb/s managed switches that are more desktop size than rack size. I still need to look up those Brocade numbers listed and see if I can find a few 10Gb 8-16 port switches.

Sent from my moto g power using Tapatalk
 

bob4432

Lifer
Sep 6, 2003
11,647
19
81
Yes, we do seem to have very familiar desires in what we are wanting to accomplish with this setup.

I found an old Intel 520 120GB SSD that will be my go-to drive. I think I will just build it with the parts I have and let her go. I did find a i7-3770 last night for cheap but if I am pretty close to overkill, I passed on it because this needs to a net 0 build, which it will be (unless the guy guy eith the i7 will deliver.....I kid, I kid...).

Plus I could pick up a Cisco SG200 as I know they will be supported only through some time next year, or for a bit more a SG250 and that is where I think for the foreseeable future is where I need to put my $$ - 10Gb/s managed switches that are more desktop size than rack size. I still need to look up those Brocade numbers listed and see if I can find a few 10Gb 8-16 port switches.

Sent from my moto g power using Tapatalk
While looking for 10GbE switches, are the lower priced ones SFP+ or rj45? Reason I am asking is the only ones I am finding that are rj45 that are decently priced and in my size preference are the QNAP ones mentioned on these networking forum.

Maybe I have just been doing things too simple for too long. When using a managed switch, is port aggregation just part of the package? I am going to have to run cat6/cat6a in the the not too distant future, should I just run 4 or 5 cables and have a switch on each side with the amount of ports needed to accommodate "connecting" the extra 4-5 ports, or does it not work this way?

Last, since I have read dumb switches can't transmit tagged data, is there any place for them in a managed network? I know this is a bit off topic, but I would just like to know what to look for so in my internet searches, if I stumble on to something I need, I would know it.

Sent from my moto g power using Tapatalk
 
Last edited:

Fallen Kell

Diamond Member
Oct 9, 1999
5,663
250
126
Most of the cheaper 10GbE switches are SFP+. I mean seriously, the Brocade that I have was $200, which has standard 8 SFP+ ports, and 2xQSFP+ -> 8xSFP+ breakout ports (as well as 2 standard QSFP+ ports that are 40GbE), and in my case 24x1GbE RJ45 ports. That is less than $12 a port for 10GbE, and then factor in transceivers, such as the Wiitek SFP-10G-T (Nbase-T RJ45 port capable of 2.5, 5, and 10GbE) for $45 each and you are still only looking at under $57 per 10GbE port (which is still less than the $77 per port of the QNAP QSW-1208-8C-US) and at the same time the Brocade is a fully managed L3/L4 switch and the QNAP is an unmanaged L2 switch.

It is even cheaper on the SFP+ ports if you are connecting to equipment in the same room and just use a DAC and SFP+ ports on the computer as well. I use DAC with both my QSFP+ and one of my SFP+ connections (my pfsense is connected via 40GbE QSFP+ DAC (~$25 for the cable, thus no transceivers needed at either the switch or computer), and so is my VM/storage server, and my WiFi access point is connected via 10GbE SFP+ DAC (~$20 for the cable and thus no transceivers needed)). My other computers that are 10GbE are not in the room with the switch, and are connected via CAT6a to 10Gbase-T cards in the computers and use a Wiitek SFP-10G-T transceiver on the switch side.

I have to believe that you will want to connect your pfsense system either with gigabit or go the 10GbE or 40GbE like I did and co-locate it in the same location as the switch, meaning you can then just get cheap SFP+ or QSFP+ network card for the pfsense system and use a cheap ($20-25) DAC connection. My 40GbE cards were Mellanox ConnectX-3 VPI card (specifically MCX354A-FCBT). These have CPU offload engines, RoCE/RDMA support, and virtualization support. They were very cheap when I got them, but prices have gone up 2-3x (from $30-35 when I bought to $60-90 now).
 
Last edited:

mxnerd

Diamond Member
Jul 6, 2007
6,152
887
126
I would say very old SSD like SATA II 32GB/64GB would work even if you do a lot of system logs.

==

If all of your PCs are Windows based and are 8.1 and up, SMB 3.0 multichannel is turned on by default, you don't even need to have a managed switch and configure link aggregation. If you have a fast enough link, why even bother, however? Managed switch does give you VLAN and other features though.


==

40Gbps between 2 PCs or even in the same PC if the PC can handle it, link the ports together without a switch.


VMware or other VM platform should be able to let you configure each virtual NIC's bandwidth. Virtual NIC always shows up as 1G Intel NIC in the VM, but the real bandwidth depends on physical link and how much bandwidth you give to the VM. Default is unlimited.

Untitled.png

==

*** I don't have any enterprise grade switches, better consult who really owned them. 😁 ***
 
Last edited:

bob4432

Lifer
Sep 6, 2003
11,647
19
81
I would say very old SSD like SATA II 32GB/64GB would work even if you do a lot of system logs.

==

If all of your PCs are Windows based and are 8.1 and up, SMB 3.0 multichannel is turned on by default, you don't even need to have a managed switch and configure link aggregation. If you have a fast enough link, why even bother, however? Managed switch does give you VLAN and other features though.


==

40Gbps between 2 PCs or even in the same PC if the PC can handle it, link the ports together without a switch.


VMware or other VM platform should be able to let you configure each virtual NIC's bandwidth. Virtual NIC always shows up as 1G Intel NIC in the VM, but the real bandwidth depends on physical link and how much bandwidth you give to the VM. Default is unlimited.

View attachment 51428

==

*** I don't have any enterprise grade switches, better consult who really owned them. ***
Appreciate the info :)

Sent from my moto g power using Tapatalk
 

ASK THE COMMUNITY