Re: Accuracy of converting semicircles to degrees

From: Dale Andrews (dandrews_at_spam.bouncer.midwest.net)
Date: 11/28/04 

Date: Sun, 28 Nov 2004 14:10:58 GMT

On or about Sat, 27 Nov 2004 13:00:42 GMT, Bruce Stemplewski
<Bruce.Stemplewski@nospamstempsoft.com> wrote or did cause to be
written:

>Garmin's formula for converting degrees to semicircles to degrees is:
>
>d = s * (180.0D / 2^31)
>
>When s = 119304640
>
>d = 9.9999994039536
>
>But 10 was entered on the GPS. Does anyone know of a more accurate
>formula for converting semicircles to degrees?
>

d = s * (180.0D / 2^31)+0.000002
When s = 119304640 and D=1 then d = 10.000001403953552525650000000000

Does that look better?

This reminds me of an operator that was continually writing up a
system that was supposed to maintain 1500 pounds and the digital meter
kept reading 1499. Tweak the meter to read 1501 and all the complaints
stopped. How important is the difference other than it doesn't look
right when there are a string of 9's rather than a string of 0's?

You should never expect more accuracy from an answer to a formula than
the accuracy of the input. Therefore your d should be no more than 9
significant digits or 9.99999940. If we consider the 180.0 to really
be 180 and an infinite number of zeros then the real max accuracy of
the formula would be limited by 2^31 or again 9 digits. 9 digit
accuracy is VERY good. It is the difference between 30 years and 30
years plus 1 second. Do you really need more than that or is it just a
string of 9's isn't as personally satisfying as a string of 0's??

Relevant Pages

  • Re: How to convert extra long strings into their equivalent Hex Strings in VBA (Word 2K)
    ... numbers (upto 18 digits max) into its equivalent Hex String ... Public Function ExpressServiceCode(ByVal ServiceTag As String) As String ... 'the number dblTemp in the specified base, ... Dim lngTemp As Long ...
    (microsoft.public.vb.general.discussion)
  • Re: BigNum -- Floating Point
    ... The 'N' is the number of decimal digits. ... The internal representation is really just a string of bits. ... the number of shifts for various multiples of ten: ... The 'exponent' is very closely related to ...
    (comp.programming)
  • Patterns in pi, copyright law, and philosophy
    ... Whether the offset of a string found in pi can be used as a form on ... then calculate how many digits of pi one would need to raise the ... "An infinite series of numbers is not an exhaustive set of numbers. ... finite string of digits occurs within the decimal expansion of pi, ...
    (sci.math)
  • Re: Patterns in pi, copyright law, and philosophy
    ... The digits of pi are widely believed to be "normal", in the sense that every n digit combination of digits is equally likely, and pass all reasonable tests of randomness. ... The mean length of the offset must at least be equal or greater than the length of the string you are looking for. ... Pi has an infinite representation as a decimal. ... finite string of digits occurs within the decimal expansion of pi, ...
    (sci.math)
  • Re: Required Field for 7 Numeric digits only
    ... Function IsNumber(ByVal Value As String) As Boolean ... > works with verifying that it has 7 digits and is a numeric filled> textbox ... > Private Sub TextBox1_KeyPress ... > Dim IsValid As Boolean ...
    (microsoft.public.excel.programming)