Question Network connection to server dropping occasionally

[Shmee]

So I have a home server, running Ubuntu 24.04 now, with an Intel X540 10GBaseT NIC, connected to my Xyxel switch via Cat6A (25ft cable I think) and an FSP+ 10GBaseT transceiver. The Xyxel switch has two 10Gb SFP+ ports, the other one also has a 10GBaseT transceiver going to my desktop, connected to a NIC with a Marvell AQtion AQC107 controller (10Gb Ethernet). The Xyxel switch also has two 2.5Gb ethernet ports, and eight GbE ports, but that isn't really important right now.

What I have noticed, is occasionally the server will lose connection, though can often find the connection again if I unplug and replug in the ethernet cable. The desktop seems to be fine. Now I am thinking the problem is probably lieing somewhere switchside, either with the one transceiver in question, or the SFP+ port itself on the switch. Perhaps the one transceiver or port is overheating? The switch does get pretty warm, and is especially hot around the 10Gb ports, and on the transceivers themselves. I am thining of putting a fan directly over them, hopefully that will help. I am hoping that it is not simply a defect in the switch or the transceiver.

I also have a fifteen foot CAT7 cable on the way, is it possible that if the cable is rated higher and the length is shorter it will put less heat stress on the transceiver? I think that should be long enough, and something to try, though 25ft is still far less than the 30m rated for the transceiver.

I also have a new 10Gb NIC coming, something I can use to swap out and troubleshoot with. Based on Marvell AQtion AQC113C.

 

[thecoolnessrune]

Does the Xyxel switch have any reporting available for the transceivers to see how hot they're getting? To your point, there are usually very strong limits put in place by vendors on using 10GBase-T transceivers because of their dense, high heat load. From the top with Cisco, all the way down to ones like Mikrotik, they all usually have guidelines published that indicate how many transceivers can be installed. Most limit you to every-other-port (and in the case of a switch with only 2 SFP+ ports, limits you to only 1 10GBase-T Transceiver).

Broadcom has released a newer 10GBase-T PHY as the BCM84891L that is much more efficient than previous transceivers. It only needs ~1.6W for 30M compared to the 2.5W - 3W requirements of the previous generation Broadcom and current generation Marvell PHYs. GTek offers it as the ASF-10G-T80 (and ASF-10G-T+) and FS offers it as the SFP-10G-T100. They are going into designs where they weren't traditionally able to be used. For instance the Intel X552 / X553 won't normally turn up 10GBase-T transceivers because of power consumption, but it will turn up the new Broadcom based transceivers. Might be worth a consideration to bring the heat / power load down.

 

[Shmee]

I'll have to check out the switch admin panel some more to see if I can find any clues. In the mean time, I also found a cheap Sodolo switch on Amazon, with 2 10 Gb SFP+ ports and 4 2.5Gb RJ45. That just came today, along with a spare 10GbE NIC and a Cat7 cable earlier.

 

[Micrornd]

I also have a new 10Gb NIC coming, something I can use to swap out and troubleshoot with. Based on Marvell AQtion AQC113C.

I have Nicgigas based on the AQC113C for my network cards and they have been flawless and cool running since I switched to them 16 months ago

 

[Shmee]

Just a small update here, server and switch have been working fine, no network drops for some time. I am not entirely sure of the original culprit, but I suspect it may have been the Intel X540 NIC. I since have put TP Link NIC in my server, and the new NIC I just ordered is installed in my desktop.

I also did the following: I put the Xyxel switch on two small metal Logitech mouse wait containers, which I am using as stands for the switch, hopefully improving airflow underneath so it is not just heat soaking on the wooden desk. I am also connecting one of the 2.5GbE ports to the new Sodolo switch, to gain additional 2.5GbE ports. Lastly I used a shorter cable, CAT7, to connect the server to the transceiver in the Xyxel switch.

 

[aigomorla]

I suspect it may have been the Intel X540 NIC

did you have good active cooling on it?

These guys gets REALLY REALLY HOT.
They were intended for pizza boxes with screamers pushing air.

 

[Shmee]

did you have good active cooling on it?

These guys gets REALLY REALLY HOT.
They were intended for pizza boxes with screamers pushing air.

Not sure how much airflow was on it to be fair, possibly not enough. Anyway so far the setup seems stable, using the Marvell based 10GbE NICs.

Also, I have noticed that my Zyxel switch seems to be running cooler, at least seeming that way to the touch. So I think giving it additional airflow/room with the stands helped in that regard.