orthogonalizing vectors

From: Mike Deskevich (mikedeskevich_at_gmail.com)
Date: 02/24/05 

Date: 24 Feb 2005 15:02:46 -0800

i was wondering if anyone knows of any ways to orthogonalize (and
normalize, but that's easy) vectors other than the gram-schmidt method?
 my problem is that i have a large number of vectors (in the range of
500-2,000) in a huge number of dimensions (10,000-70,000). i currently
have a gram-schmidt method implemented where i take vector number 2 and
orthogonalize it wrt vector 1 and then take vector 3 and orthogonalize
it wrt vectors 2 and 1 and so on. i'm not limited here by time, the
vector othogonalization goes pretty fast, what i'm limited on is
numerical precision of the vector. i have the vector stored using c's
long double on an intel processor (i think that's an 8 byte double - i
don't have my documentation with me right now).

anyway, when the vector size is on the small end of my range (500
vectors of 10,000 dimensions) the orthogonalizaiton works great.
unfortunately on the larger end of the range (2,000 vectors of 70,000
dimensions), the repeated dot products and subtractions do not exactly
orthogonalize the vectors (there's small amounts of different
components left over because of incomplete subtractions).

since in the end, i want all my vectors normalized too, i've tried both
(a) orthogonalizing all the vectors and then normalizing at the end and
(b) normalizing each vector as i orthogonalize it wrt all the other
vectors. both produce the same results.

i'm pretty sure that it's a numerical precision problem because before
i had this implemented as long doubles, i had it implements as doubles
and the problem showed up earlier (700 vectors of about 20,000-30,000
dimensions). switching over to long doubles appeared to have solved
the problem, then i went to even more dimensions and the problem showed
up again.

maybe there's another way to orthogonalize vectors? or is there some
code out there that was written by someone smarter than me that can
handle this?

thanks!
mike