Nvidia Announces Tesla T4 GPUs With Turing Architecture

Mar 10, 2004
28,493
224
126
#1
Looks interesting at just 75W.

https://www.tomshardware.com/news/nvidia-tesla-t4-turing-gpu,37788.html

The Tesla T4 GPU comes equipped with 16GB of GDDR6 that provides up to 320GB/s of bandwidth, 320 Turing Tensor cores, and 2,560 CUDA cores. The T4 features 40 SMs enabled on the TU104 die to optimize for the 75W power profile.

The GPU supports mixed-precision, such as FP32, FP16, and INT8 (performance above). The Tesla T4 also features an INT4 and (experimental) INT1 precision mode, which is a notable advancement over its predecessor.
From WCCF:
  • NVIDIA Tesla T4 GPU – Featuring 320 Turing Tensor Cores and 2,560 CUDA cores, this new GPU provides breakthrough performance with flexible, multi-precision capabilities, from FP32 to FP16 to INT8, as well as INT4. Packaged in an energy-efficient, 75-watt, small PCIe form factor that easily fits into most servers, it offers 65 teraflops of peak performance for FP16, 130 teraflops for INT8 and 260 teraflops for INT4.
  • NVIDIA TensorRT 5 – An inference optimizer and runtime engine, NVIDIA TensorRT 5 supports Turing Tensor Cores and expands the set of neural network optimizations for multi-precision workloads.
  • NVIDIA TensorRT inference server – This containerized microservice software enables applications to use AI models in data center production. Freely available from the NVIDIA GPU Cloud container registry, it maximizes data center throughput and GPU utilization, supports all popular AI models and frameworks, and integrates with Kubernetes and Docker.
 

crisium

Platinum Member
Aug 19, 2001
2,578
0
136
#2
Can anyone do the math from those FP16 TFLOPs to determine what frequency it's running at? Seeing as it's 2560 shaders and 320 GB/s bandwidth, same as GTX 1080, I'm curious how low they had to clock Turing to get it into 75w.
 

Hitman928

Golden Member
Apr 15, 2012
1,755
187
136
#3
Can anyone do the math from those FP16 TFLOPs to determine what frequency it's running at? Seeing as it's 2560 shaders and 320 GB/s bandwidth, same as GTX 1080, I'm curious how low they had to clock Turing to get it into 75w.
I get 1.27 GHz.
 
Mar 10, 2004
28,493
224
126
#4
Nvidia tells us that the die does come equipped with RT Cores, just like the desktop models, but that they will be useful for raytracing or VDI (Virtual Desktop Infrastructure), implying they won't be used for most inference workloads.
So it does have the RT cores.
 

Hitman928

Golden Member
Apr 15, 2012
1,755
187
136
#6
Wouldn't the FP16 numbers include the tensor cores?
Yes.

Edit: Without the Tensor cores, it would only provide 13 TFLOPS of FP16.
 
Last edited:


ASK THE COMMUNITY