• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Windows Calculator Scientific Notation Limit

S

sharkeeper

Lifer
Is there a limit?

I was working with a large number (143,000+ digits) and hit the n! key! I figured I'd let it go until it finished and that was over 5 hours ago, one CPU is at 100% working the problem! 🙂

-DAK-
 
the windows calculator does not accept numbers longer than 32 digits. I don't know how in the hell you think you got 143,000 digits in there.
 
Well, if you think about it, the maximum number a 32 bit processor can handle in one chunk is 4294967296 in one shot. After that it will need to run a loop to work on larger numbers. I guess you could probably do something to make it run an indefinite loop so it will continue until it runs out of a certain resource. Either that or there is an extremely high limit of the number of times the loop will run so that gives a limit...

This is me just thinking to myself though, I could very well be wrong.
 
the windows calculator does not accept numbers longer than 32 digits. I don't know how in the hell you think you got 143,000 digits in there.
Click to expand...

Scientific Notation

-DAK-

Why not RAID5 three mice? You could go twice as fast in a FPS, could stand losing one during battle and with today's fast cpu's, one would not take too much of a hit to calculate cartesian parity. Of course the third hand may be hard...

 
Originally posted by: notfred
the windows calculator does not accept numbers longer than 32 digits. I don't know how in the hell you think you got 143,000 digits in there.
Click to expand...

Scientific notation?

Edit: got beat to the punch
 
Originally posted by: shuttleteam
Is there a limit?

I was working with a large number (143,000+ digits) and hit the n! key! I figured I'd let it go until it finished and that was over 5 hours ago, one CPU is at 100% working the problem! 🙂

-DAK-
Click to expand...

It ain't gonna be finished anytime soon. I just did 25000! and it took about 8-10 seconds I figure.
 
Still going! It's gonna need to express the exponent in scientific notation! :Q

Either that or it will never finish. Thank goodness for SMP.

-DAK-
 
You must log in or register to reply here.

TRENDING THREADS

Back
Top