what is double precision useful for?

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

Cerb

Elite Member
Aug 26, 2000
17,484
33
86
The inexactness of 1.2 has to do with the base 2 representation, not the floatingness of the point. And most people use a base 2 representation, both for floats and for fixed point numbers
It has to do with the combination of both. 1.2 in a fixed-point can be exactly represented, or not, depending (usually, it is, because the fixed point number exists for handling things like money, which are typically in decimal).

It's when you have IIIII.FFF, but then want to move to III.FFFFF, for smaller sets of values, or IIIIIIII00000, for larger sets of values, that fixed point falls on its face, just like its integer ancestors, regardless of base. Or, you may find that it's hard to get proper rounding behavior with one or another fixed-point implementation, due integer-like truncation-style rounding, the hiding of which incurs a significant performance hit that FP doesn't have.

Javascript, you were the chosen one! :( :( :(
Lua, as well. I've had to fight it with both of them. FP numbers are great, but they just aren't ints, and a programming language really should have ints available.

Scientific computations that need a lot of accuracy do use floats.
They need significant digits, rather than X places before/after the decimal. That's really the difference. Integers of a given size have equal or better accuracy, for their value range, but that range always includes a ones digit as the least significant. Great for managing data structures, inventories, text values, and money, but bad for...really, most everything else.
 

dipster

Junior Member
Jan 19, 2014
6
0
0
It has to do with the combination of both. 1.2 in a fixed-point can be exactly represented, or not, depending (usually, it is, because the fixed point number exists for handling things like money, which are typically in decimal).
And 1.2 in floating point can be represented, or not, depending. The thing both of those depend on is what the base is, not the fixedness of the radix point.

It's when you have IIIII.FFF, but then want to move to III.FFFFF, for smaller sets of values, or IIIIIIII00000, for larger sets of values, that fixed point falls on its face, just like its integer ancestors, regardless of base.
There are all sorts of ways they fall on their face. (Especially if you try to implement them yourself, then you're doomed.)
 

Wall Street

Senior member
Mar 28, 2012
691
44
91
It doesn't take thousands. 1/(a - b), if a and b are close -- oops!

I think a point could be made that the worst that could happen is visual artifacts, and probably only for a few pixels, and probably only for a single frame. Pretty harmless.

This algorithm returns a very very high number if a and b are close. Can you think of a case in a gaming graphics shader scenario where:

1) this is a useful calculation to perform?

and

2) it is very important to know exactly which very high number is returned?
 

Lonbjerg

Diamond Member
Dec 6, 2009
4,419
0
0
DP and ECC correct totally different types of errors, so it's strange to say one is useless without the other. There's certainly classes of problems that don't require ECC but simply aren't practical to solve using single precision. Not that it's been well adapted to GPUs yet, but linear programming for example.

Both are irrelevant for gaming though yeah.

Don't say that....some people insist on you need both....because they do...quite funny ;)
 

dipster

Junior Member
Jan 19, 2014
6
0
0
This algorithm returns a very very high number if a and b are close. Can you think of a case in a gaming graphics shader scenario where:

1) this is a useful calculation to perform?

and

2) it is very important to know exactly which very high number is returned?

It's not about "exactly". The number will be very wrong. In fact, the division doesn't matter for that; the error was introduced by the addition operation. Division just made it more noticeable.

Anyway, I'm not familiar with what formulas video game graphics tend to use. I think you're too quick to doubt that it'll come up, addition of things that can have opposite signs, and subtraction of things that can have the same sign, are very common in all fields. As is division. I do agree it's unlikely to matter much. Even if there's only one significant digit of accuracy, probably nobody will notice, since it'll probably be unimportant, brief, and small. The speed benefits of using singles are just too good.
 

Cerb

Elite Member
Aug 26, 2000
17,484
33
86
And 1.2 in floating point can be represented, or not, depending. The thing both of those depend on is what the base is, not the fixedness of the radix point.
In practice, fixed point numbers are still quite varied, though Decimal-like, BCD (just for its historical use), and bignum types dominate non-embedded use; while floating point basically means IEEE754 or a bastardization of it.

(Especially if you try to implement them yourself, then you're doomed.)
I did just that, going through some of Project Euler (a fixed width number with ANSI characters for digits, though variable total digits...the epitome of efficiency!). I never could get the hang of an efficient divide, but I was able to implement enough shortcuts to make it fast enough in non-JITed JS and Lua on a PIII, that even the problems I didn't know the right clever tricks for only took some seconds :) (each attempt was several months apart, and well after I had done the same with a custom-made fixed point type in Lua). Having read all kinds of rants by people hating on IEEE754, over the years, I came away with quite a bit more respect for it, and am convinced that it wasn't just Intel taking over the COTS PC market that made it the de facto standard. It really is a well-balanced format, exceptions and all.
 

Wall Street

Senior member
Mar 28, 2012
691
44
91
It's not about "exactly". The number will be very wrong. In fact, the division doesn't matter for that; the error was introduced by the addition operation. Division just made it more noticeable.

No. The number isn't very wrong after the addition/subtraction. If the base units for a and b are in miles, then the rounding after the after the addition/subtraction representing these distances in single precision floats will be in inches. In a given calculation where the units of the original a and b values being compared are significant, the error after a-b will not be significant because it is many orders of magnitude smaller. The exponent bit will become so small that any addition, subtraction or multiplication of the results will practically treat a-b as zero.

The error appears large in the 1/(a-b) for a~b is that you are asking for the inverse of an equation that approaches zero. This limits to infinity. Solving for values of discontinuous functions around their asymptotes is always error prone (because you are trying to map infinity to a real number).
 
Last edited:

Beavermatic

Senior member
Oct 24, 2006
374
8
81
good reply.:) i actually bought a 780 a few months ago (and i would've returned it except only i would have had nothing until i could've saved up $480 more for a titan) thinking it didnt have a fuse in it that cripples DP... that's just low class and not really necessary of nvidia to do that... it know it helped them make more money, but considering they wont open up their drivers much due to IP and that they have no competition (except maybe AMD, but AMD has always neglected image quality and extra features ever since R300 if not before then), it was rather amoral.

additionally, i am worried about price ceilings on processors of all kinds and that would be more favorable to intel, nv, and amd than IP repeal. corruption goes up as power is more consolidated and to get rid around that one needs to not be offered any IP, other regulations, or subsidies and then these things could be designed, made, and shipped out a place no larger than an 8 car garage. not saying that would work best for me if i was a businessman, but then i am not trying to be a businessman yet.

[mods: i dont mind this thread being closed now since i got good answers and may have just derailed it; although i wish i had the self-control to never make another DP thread again lol]

Actually just sold two Titans and bought two 780ti's. I'm a gamer, not a scientist. I'm using a single monitor at 2560x1600, so it was ridiculous to have the 6gb of VRAM (nor do I do any sort of graphic manipulation requiring such), and Im not doing compute or number crunching so no need for the double precision. I'll take the higher clock speed and more cores for a bit more game performance.

Was a hard decision though... the DP I really could careless about, but to walk away from 6GB of VRAM to a lower 3GB (albeit faster) in trade for slight performance gain was one of the hardest decisions of my life.
 

dipster

Junior Member
Jan 19, 2014
6
0
0
No. The number isn't very wrong after the addition/subtraction. If the base units for a and b are in miles, then the rounding after the after the addition/subtraction representing these distances in single precision floats will be in inches. In a given calculation where the units of the original a and b values being compared are significant, the error after a-b will not be significant because it is many orders of magnitude smaller. The exponent bit will become so small that any addition, subtraction or multiplication of the results will practically treat a-b as zero.

You are absolutely right, the absolute error will be nearly insignificant. The problem that a-b runs into for a and b within a few orders of magnitude, is that the relative error becomes enormous. The way scientists measure precision is with significant digits, and after a subtraction of numbers of similar size, you might be let with only one significant digit of precision, as the wikipedia article demonstrates.

That relative error can be scaled to any amount of absolute error, just by scaling up the number involved. That can and does happen; if it didn't, nobody would care ;)

Let me put it another way. If one float is 10097, representing the number 10097.XXXXXX , and another number is 10096, which represents the number 10096.YYYYYY, the result from this is 1.0000 -- we've lost almost all the precision we had available, there are no significant digits other than the first. You might say that in absolute terms, this doesn't matter -- the number is at most 1 unit away from where it should be. But one unit away is double the number.
 
Last edited:

Rakehellion

Lifer
Jan 15, 2013
12,181
35
91
why is 32bit single precision? why not 16bit or something? does this have to do with IEEE setting some kind of standard?

It used to be 16-bit back in the day, like the Geforce 2 era, but 32-bit is more standardized.

For 95% of graphics calculations, 32-bit is more than enough. Sometimes 16-bit is still used when space is scarce. Also, bear in mind that a 32-bit float is downsampled to an 8-bit int once it hits your screen.
 

Rakehellion

Lifer
Jan 15, 2013
12,181
35
91
Sounds like someone is trying to do some soul searching on whether they should buy a GTX Titan or a GTX 780ti.... double the vram with double precision, or more cores and better memory bandwidth (but half the vram) and higher "today" performance.

Buying a Titan won't make a difference because games are still programmed using 32-bit floats.
 

Rakehellion

Lifer
Jan 15, 2013
12,181
35
91
good reply.:) i actually bought a 780 a few months ago (and i would've returned it except only i would have had nothing until i could've saved up $480 more for a titan) thinking it didnt have a fuse in it that cripples DP... that's just low class and not really necessary of nvidia to do that... it know it helped them make more money, but considering they wont open up their drivers much due to IP and that they have no competition (except maybe AMD, but AMD has always neglected image quality and extra features ever since R300 if not before then), it was rather amoral.

There's no switch inside the GTX 780 chip that makes it slower at double precision math. That's like saying The GTX 580 is slower than the 780 because of a switch.
 

Rakehellion

Lifer
Jan 15, 2013
12,181
35
91
Using 64bit z-buffer (or 48Z/16S) would be easy way out of most z-fighting errors, but it would be costly and I'm pretty sure no GPU currently supports the format. (although coming 'dx12?' parts might fix this.)
It would also mean quite big increase in needed bandwidth.

Z-buffers use int, not float.
 

Cerb

Elite Member
Aug 26, 2000
17,484
33
86
There's no switch inside the GTX 780 chip that makes it slower at double precision math. That's like saying The GTX 580 is slower than the 780 because of a switch.
The Titan and K6000 offer several times the DP performance of the GTX 780 Ti, yet offer about the same SP performance, and they use the same chip, with similar clocks and core counts. Why do think it's not a "switch"?
 

Rakehellion

Lifer
Jan 15, 2013
12,181
35
91
The Titan and K6000 offer several times the DP performance of the GTX 780 Ti, yet offer about the same SP performance, and they use the same chip, with similar clocks and core counts. Why do think it's not a "switch"?

It's a completely different chip. Die space isn't free and they aren't going to put all that R&D in designing a product that 99% of their customers won't use.
 

Beavermatic

Senior member
Oct 24, 2006
374
8
81
Buying a Titan won't make a difference because games are still programmed using 32-bit floats.

So the Titans for gaming is pretty much just a waste of cash for gaming (with some exceptions ill list later)... that I can agree on and have experience with.

I honestly think Nvidia really should have marketed the Titans for what they are... compute/video & modeling workstation cards with lesser RAM quality than the Quadro's, but still the same size, at a fraction of the Quadro cost and still offers DP. I think labelling them as a gamer card raised a lot of expectations that this VRAM level was the next step for NVidia's gaming series cards when it really wasn't. So it kind of came back to haunt them with the release of the 780ti... everyone admired the extra cores and shaders and DX11.2 support, but were a bit confused on the serious plunge of half the VRAM.

I'll be honest, I bought my two titans out of fear of running out of VRAM at 2560x1600 with full AA enabled.

Never came ever came close to 6GB's. Not even close to 3GB's.

I chucked those two Titan's to a video/graphic designer, who can definetley stress that 6GB of VRAM through and through, and picked up two 780ti's to replace them. Now... to the naked eye... I notice no difference. All my games run as smooth as they did prior on the Titans. Of course when I start turning on benchmark tools and framerate counters, I do see 10 to 25fps gains on the 780ti's, but realistically, I think the difference between the two without looking at the benchmarks and counters, your not gonna see it without them.

I think going forward, Nvidia *really* needs to market whatever comes up next in the Titan series as a value Compute/Workstation/Media creation card for those who need such large buffers while sacrificing the need for ECC RAM yet still can do DP, rather than a gamer card. It's just kind of misleading, and too costly to be so. The only gamers I could possibly think of that need 6gb's of RAM are ones doing just outrageously high resolutions beyond 4k and or massive multimonitor setups, especially when using AA, though if your at such insane resolutions, you probably don't need AA to begin with.
 
Last edited:

Cerb

Elite Member
Aug 26, 2000
17,484
33
86
It's a completely different chip.
No, it's not. They even admit that much: they make the 780 Ti, Titan, K6000, K20, K20X, and K40 from the GK110. Every spec but DP performance matches that.
Die space isn't free
It's the same cost, being the same die. There may be added costs after cutting the die out for the Quadro and Tesla versions, but it's the same until then.
and they aren't going to put all that R&D in designing a product that 99% of their customers won't use.
Sure they would, to make more money from that 1%, by serving them specially-tailored products using the same core designs as the cheaper consumer versions. It's common practice. What do you think makes a Xeon E3, compared to a Core i5? Same thing. They have a vested interest in potential customers of those not opting for gaming versions of the same chip, should they not need the entirety of the Quadro or Tesla feature set. The Titan itself was an anomaly.
 

Exophase

Diamond Member
Apr 19, 2012
4,439
9
81
Furthermore single precision only has about a 10^6 precision.

There are actually 24 bits of precision in single precision floating point, not counting denormals. So you can store each unique integer up to about 16.7 million. Your number, 1 million, would only be around 20 bits.

Cerb said:
In practice, fixed point numbers are still quite varied, though Decimal-like, BCD (just for its historical use), and bignum types dominate non-embedded use; while floating point basically means IEEE754 or a bastardization of it..

I've never before seen someone use the term fixed point by itself to mean anything other than a plain old integer with an implied decimal point at some base-2 bit position.

It used to be 16-bit back in the day, like the Geforce 2 era, but 32-bit is more standardized.

Single-precision has meant 32-bit and double-precision 64-bit ever since IEEE determined that back in 1985. And for all intents and purposes, for quite a while before then,

16-bit formats have always been called something like half-precision, and I don't think anyone was really using it way back in the GeForce 2 era (back then the relevant parts of the pipeline were still fixed point)

Z-buffers use int, not float.

Not 100% of the time, eg Dreamcast and AFAIK some later PowerVR hardware used a 32-bit float z-buffer. Although only inside the tile, for most cases that was effectively the same thing.
 

Cerb

Elite Member
Aug 26, 2000
17,484
33
86
I've never before seen someone use the term fixed point by itself to mean anything other than a plain old integer with an implied decimal point at some base-2 bit position.
It's always been included with the likes of Decimal and currency types since way back starting CS classes ages ago, IME,.
 

GRAFiZ

Senior member
Jul 4, 2001
633
0
76
It's a completely different chip. Die space isn't free and they aren't going to put all that R&D in designing a product that 99% of their customers won't use.

Are you sure about this?

I don't believe the Titan and 780ti have completely different chips... I'm pretty sure it's the same foundation with different features enabled/disabled.

Financially it wouldn't make sense to design two entirely different GPU's that mirror each other in nearly every way possible.

In fact, no way Nvidia did this, it would cost them hundreds of millions of dollars.

Rather, they design one GPU with many features and have several variations fabbed for them.
 

Beavermatic

Senior member
Oct 24, 2006
374
8
81
Are you sure about this?

I don't believe the Titan and 780ti have completely different chips... I'm pretty sure it's the same foundation with different features enabled/disabled.

Financially it wouldn't make sense to design two entirely different GPU's that mirror each other in nearly every way possible.

In fact, no way Nvidia did this, it would cost them hundreds of millions of dollars.

Rather, they design one GPU with many features and have several variations fabbed for them.

The Titan and 780Ti use the same chip. With the 780ti, they just enabled the previously disabled SMX which now gives it another 192 cores and (I think) 19 more shaders, cut the VRAM in half, upped the clock speed thanks to a slightly new cooler (though resembles the Titan cooler), added DX11.2 support, and reduced DP to 1/24 from 1/3.

For all intents and purposes, they could have just kept everything intact, enabled the extra SMX, added DX11.2 support, and just called it a Titan 2. Though I believe they saw the Titan was too expensive and outside the reach of most gamers wallets, so while cranking up the horsepower, they had to neuter it in some areas that would be less of interest to a gamer to get the price down, and to still give relevant value of the current Titan lineup to those still wanting to drop $1200 per card.

Which is why I said the Titan should have never been labelled as a gaming card period, and just a value Quadro-series card. Though it'd had been a tough sell for those wanting to do compute on non-ECC ram of the Titan (and if so, risky despite its DP performance), it'd still be great for media/video/modeling/graphics design where ECC doesn't play such a significant role as quantity of VRAM compared to the Quadro line pricing.
 
Last edited:

GRAFiZ

Senior member
Jul 4, 2001
633
0
76
The Titan and 780Ti use the same chip. With the 780ti, they just enabled the previously disabled SMX which now gives it another 192 cores and (I think) 19 more shaders, cut the VRAM in half, upped the clock speed thanks to a slightly new cooler (though resembles the Titan cooler), added DX11.2 support, and reduced DP to 1/24 from 1/3.

For all intents and purposes, they could have just kept everything intact, enabled the extra SMX, added DX11.2 support, and just called it a Titan 2. Though I believe they saw the Titan was too expensive and outside the reach of most gamers wallets, so while cranking up the horsepower, they had to neuter it in some areas that would be less of interest to a gamer to get the price down, and to still give relevant value of the current Titan lineup to those still wanting to drop $1200 per card.

Which is why I said the Titan should have never been labelled as a gaming card period, and just a value Quadro-series card. Though it'd had been a tough sell for those wanting to do compute on non-ECC ram of the Titan (and if so, risky despite its DP performance), it'd still be great for media/video/modeling/graphics design where ECC doesn't play such a significant role as quantity of VRAM compared to the Quadro line pricing.

Yeah, thats' what I thought. No way is it entirely different deisgns.
 

Beavermatic

Senior member
Oct 24, 2006
374
8
81
I know this is straying it a little off topic, but it's still quasi-relevant due to the fact DP plays into pricing and interest.

I do believe the problem with the Titan was the fact its pricing was to try to lure more gamers into a workstation/compute-grade card, but still too high for most gamers to afford. And what it offered as its strongest points 99.9% gamers didn't need or could have cared less about (double precision, 6GB of VRAM).

Id say the sweet spot for most high-end gamers is $450 to $600 dollars (most being in the singly GPU board), with hardcore enthusiasts willing to dump $700 to $900 per card (most being in the dual GPU boards). At it's release, the Titan cost a whopping $1300 per card.... so it was way outside the ballpark of interest of even the elitist of gamers, and a single gpu board.

I remember hearing the arguments of 6GB of VRAM being the smoking gun for its sale for guaranteed future proofing, however in reality, for gamers on mainstream resolutions (and even 4k), 6GB is just ridiculous, and likely wont be needed as the standard for another few years at least. Which in that case, it was unnecessary futureproofing. By the time it was needed, there'd be 3 or 4 generations of newer cards that far outpower the Titan in every way/shape/form (and there already is... the 780ti, not even a year later). You had 3 years into the future worth of VRAM, but 3 years behind on the performance, it would have been humorously out of touch with what is required for ultra-level gaming as far as performance goes 3 years from now despite having sufficient VRAM. I mean the next few years will be 4k, which all of todays 2GB to 4GB cards will handle with ease (well, more so the 3GB and 4GB vram cards... I think the 2GB cards will start having issues with 4k especially when AA comes into play)

And never once did I hear gamers bring up Double Precision. I doubt many even know what it is. It literally plays no role in gaming as of current, or for a while (if ever). So there's a feature the target audience doesn't even use, nor care about.

If Nvidia marketing is listening (and I know they are... they browse these forums from time to time), I'd suggest going forward with the Titan line to drop the gamer strategy for it, and focus it more on budget Quadro hardware, less for compute (because of the non-ECC RAM), more for design/modelling/vid editing and especially data mining with that DP. Heck, with Quadro's going for $3000+, it could easily be sold for $2000 and still be considered a budget Quadro if marketed to the right group, and still making more sales at $2000 to the target audience vs. less sales at $1200 with the gaming audience.
 
Last edited: