Any thermal measurement needs calibration if you need accuracy.
Your main problem is that you don't know what the temperature on the die (semiconductor) is to calibrate it.
Graphics cards that provide temperature reading have a diode on the die as a part of the semiconductor chip itself. That diode is used for measuring the temperature of the die. There is a big difference between the die temperature and the temperature of the plastic surrounding it (that is what an external sensor will have access to). The best an external sensor can do is to measure the temperature of the plastic.
There is also a difference between the temperature of the package and the temperature on the other side of the PCB, where you say you have your sensor.
If you had the relationship between the two temperatures, you could calibrate it to show the temperature of the die by measuring the temperature on the other side of the board. But, you don't.
All that said, it is better than nothing. It can also give you relative information. I mean you can make measurements and find out what the plastic temperature is with a stable system. Then, if you make a chnage to your system (chnage the GPU cooler), you can see how the temperature of the package changed as a result.