Ok, first off, "modems" don't have wifi. "Gateways", have modem/router/wifi. Do you have a Gateway?
Second, do you already / also have a secondary Router, like a Netgear, Asus, or Linksys? Is it plugged into the Gateway? If so, and you haven't logged into the Gateway, and disabled the wifi and the routing function, and enabled Bridge Mode to effectively turn it into a dumb modem, then your connected devices are going to be suffering from Double NAT, which is generally considered a Bad Thing, although it barely affects basic Web Browsing.
You said that "before, everything on the smart TV worked fine". What Router was the smart TV connecting to before? Was your previous device a Gateway, or is it connecting to a local wifi Router, that is now plugged into the new Gateway, rather than just a modem before?
Is it the Speed Test on the Smart TV that's giving the "latency error"? What happens if you run a Speed Test on your wired PC? Wireless laptop? Do they all give "latency error", or just the Smart TV?
Edit: Did you first try powering off all of the connected devices, and then powering them up in the proper sequence?
If you swapped modems or Gateways or whatever, and left the TV on and running, there may have been IP address issues with the TV and router/gateway.