Re: Comparing LS and TLS

On 2008-03-25 16:38:27 -0300, spasmous <spasmous@xxxxxxxxx> said:

I wrote some code to test the total least squares method. Results were
disappointing: for 10000 trials with different added Gaussian noise
the least squares produces the smallest error. When there are multiple
right hand side vectors there is a specific algorithm for TLS, which I
also compared (Ref Golub & van Loan). Results of this were worse than
LS and randomly better or worse than single RHS TLS.

I'm curious why is the special purpose TLS for multiple RHS not the
best? And why LS is superior to either TLS approach?!?

Numerical analysis will tell you HOW to compute it.
Statistics will tell you WHY you might want to compute it.

Why would you ask a statistics question in a numerical analysis newsgroup?
Try sci.stat.math for a meaningful response. Do not be surprised to find
that you have not asked the question in a sensible way. Operationally
that means that you may not have actually done the experiment you think
you did.

When do you think TLS is called for? Hint: What errors do you have in the
predictors?

% test what is best: ls, tls or tls with multiple RHS
function test_tls()

m = 10; % rows
n = 4; % cols
k = 2; % RHS vectors
sd = 0.01; % added noise std. dev.
trials = 10000; % number of trials

% matrices (condition 100)
Atrue = full(sprand(m,n,0.5,1/100));
xtrue = ones(n,k);
btrue = Atrue*xtrue;
err = zeros(n,k,trials,3);

% loop here
for i = 1:trials

% add noise
A = Atrue + sd*randn(m,n);
b = btrue + sd*randn(m,k);

% least squares
err(:,:,i,1) = pinv(A)*b - xtrue;

% tls separate RHS
for j = 1:k
err(:,j,i,2) = tls(A,b(:,j)) - xtrue(:,j);
end

% tls multiple RHS
err(:,:,i,3) = tls(A,b) - xtrue;

end

% analysis
err = reshape(err,n*k*trials,3);

disp(['LS error = ' num2str(norm(err(:,1)))])
disp(['TLS1 error = ' num2str(norm(err(:,2)))])
disp(['TLS2 error = ' num2str(norm(err(:,3)))])



% total least squares function
function X = tls(A,B)

% Solves the linear equation AX=B using
% total least squares.
% Ref: Golub & van Loan 3rd ed (p 596)
%
% A is the matrix (m x n)
% B is the RHS matrix (m x k)

[m n] = size(A);
if size(B,1)~=m
error('A, B size mis-match')
end

% augmented matrix
C = [A B];
[U S V] = svd(C,0);

% solve
V12 = V(1:n,1+n:end);
V22 = V(1+n:end,1+n:end);
X = -V12/V22;


.

Relevant Pages

  • Comparing LS and TLS
    ... I wrote some code to test the total least squares method. ... LS and randomly better or worse than single RHS TLS. ... sd = 0.01; % added noise std. ...
    (sci.math.num-analysis)
  • Re: Comparing LS and TLS
    ... the least squares produces the smallest error. ... LS and randomly better or worse than single RHS TLS. ... the "true" system is compatible and then the expectation value of the ... level of propability for a error level ...
    (sci.math.num-analysis)
  • Re: A Diophantine Equation
    ... Hmmm ... ... where the RHS is a difference of 2 squares. ...
    (sci.math)
  • Correction
    ... meaningful way is called "Total Least Squares" After looking at ... some TLS literature, I have come to the conclusion that my formulation ... LS solver. ...
    (comp.graphics.algorithms)