Floating Point Issues
- From: Bill Woessner <woessner@xxxxxxxxx>
- Date: Wed, 2 Jan 2008 07:55:54 -0800 (PST)
I need to evaluate an expression of the form AB-CD and I only need the
fractional portion of the result. Unfortunately, A and B are of the
order 10^6, C is of the order 10^9 and D is of the order 10^-1. So
when I go to form my sum, which is of the order 10^12, I only get a
few digits of precision in the fractional portion (which is all I care
about). Does anyone know of a trick to help alleviate this? I
suppose I could use a higher precision data type. I'm currently using
double precision but I could try a long double or some higher-
precision package.
Thanks in advance,
Bill
.
- Follow-Ups:
- Re: Floating Point Issues
- From: user923005
- Re: Floating Point Issues
- From: pixel
- Re: Floating Point Issues
- From: Martin Brown
- Re: Floating Point Issues
- From: Stephen Montgomery-Smith
- Re: Floating Point Issues
- Prev by Date: Can you plot your numeric data faster/easier?
- Next by Date: Re: Floating Point Issues
- Previous by thread: Can you plot your numeric data faster/easier?
- Next by thread: Re: Floating Point Issues
- Index(es):
Relevant Pages
|