Re: prior distributions of estimated parameters

Hi.

"Leslaw Bieniasz" <nbbienia@xxxxxxxxxxxxx> a écrit dans le message de news:
Pine.GHP.4.58.0507131528560.15769@xxxxxxxxxxxxxxxxxxxxxx
>
>
> Thank you very much for thoughtful comments. I would like
> to understand them better. Is there any literature where
> I could read about this?

What I am describing is the standard Bayesian approach to the problem. There
is a huge amount of literature on Bayesian methods (which are really just
applications of probability theory), but if you want an interesting read,
look at Jaynes' book called 'Probability theory as logic', or his papers,
many of which are online at http://bayes.wustl.edu. He is a very clear
writer. You will not find the 'solution' to your problem in this book, but
you will find out how to construct your own solution along the lines I
described.

> On Mon, 11 Jul 2005, illywhacker wrote:
>
>> to high probability. There really is no harm in assuming a gaussian
>> distribution here based on the width of the error bar. A Gaussian
>> distribution is the maximum entropy distribution for a given variance,
>> and
>> so you are in effect assuming as little as possible. Or we could assume
>> that
>> the error bars represent an absolute limit, but that within these limits
>> the
>> distribution is uniform. Note, however, that this is assuming *more* than
>> is
>> assumed by a Gaussian distribution.
>>
>> So, given the parameters of your model, you can predict the true value at
>> a
>> sample point, and thereby calculate the probability of the datum given
>> the
>> parameters. Assuming that the errors at the different points are
>> independent, you can there by calculate the probability of all the data
>> given the parameters. (Note that if the errors cannot reasonably be
>> assumed
>
> This is somewhat unclear to me. Do I actually assume that
> the distrubution of data given parameters is normal, or
> I calculate this distribution from the model, by assuming
> the prior distribution of the parameters?

No - you do not calculate it; it is part of your model. You assume a
Gaussian distribution, or rather you describe your knowledge of the data
given the parameters by a Gaussian distribution. This is because for given
noise energy (related to the size of the error bars), the Gaussian
distribution is the maximum entropy distribution; that is, it assumes as
little as possible given the information provided (noise energy). If you
have more information than this, you should try to incorporate it into the
distribution of the data given the parameters: it will help.

The distribution for the data given the parameters, P(data | params),
coupled with a prior distribution for the parameters P(params) - which I
assumed was constant - enables you to construct, via Bayes' theorem, the
distribution for the parameters given the data:

P(params | data) = P(data | params) P(params) / P(data)

where P(data) = \int_{params} P(data | params) P(params) is the normalizing
constant (constant wrt params that is).

>> Now you need a prior distribution for the parameters. There are ways to
>> construct these, but for the moment assume that the prior distribution
>> for
>> the parameters is uniform. The distribution for the parameters given the
>
> Isn't a uniform distribution arbitrary? What will be the effect
> of this assumption on the final distribution of parameters given the data?
> Actually, I would be most happy to find the "true" distribution of
> parameters, given the data, no matter what the distribution of the data
> is. I think that even if we don't know the true distribution of the
> data, the information about it is contained in the data.

The uniform distribution is not arbitrary: it expresses the fact that you
know nothing about the parameter values before you see the data (actually
this is not the whole story, but that is the idea). If the data provides
fairly precise information about the parameters, that is, if P(data |
params) is quite strongly peaked around a certain value of params, then the
prior distribution will not make much difference provided it varies slowly
over the region around this value. If you do know something about the
parameter values, then you should try to incorporate this knowledge into the
prior distribution for the parameters; it will help.

illywhacker;


.

Relevant Pages

  • Re: FFT test with few kbits
    ... The output of the DFT is 20 complex values. ... >> to test that they have the same gaussian distribution as the real ... For each complex output of the DFT, ...
    (sci.crypt)
  • Re: Fourier Derivation of Heisenberg Uncertainty is Nonsense
    ... by taking the Fourier transform of the Gauss distribution ... Gaussian distribution in mathematical probability-statistics is like ... optimality, Gaussian/Normal and other graph-symmetric real line ... Dimitriyev gives a second alternative derivation of the Heisenberg ...
    (sci.physics)
  • Re: Random delay as a countermeasure to timing attacks
    ... - the signal has a Gaussian distribution, ... the noise ... random queries, or queries chosen according to the cipher, or ...
    (sci.crypt)
  • Fourier Derivation of Heisenberg Uncertainty is Nonsense
    ... by taking the Fourier transform of the Gauss distribution ... Gaussian distribution in mathematical probability-statistics is like ... optimality, Gaussian/Normal and other graph-symmetric real line ... Dimitriyev gives a second alternative derivation of the Heisenberg ...
    (sci.physics)
  • Re: Students Error Bars
    ... > When you just have a few data points but have reason to believe they come ... > distribution to estimate your uncertainty. ... if you were measuring the density of some object, ... it may be the case that the error bars are widened a bit ...
    (sci.physics)