Re: how to find the best ADC step size?

From: Randy Yates (randy.yates_at_sonyericsson.com)
Date: 12/22/04 

Date: 22 Dec 2004 15:21:21 -0500

"Clay S. Turner" <Physics@Bellsouth.net> writes:

> "Randy Yates" <randy.yates@sonyericsson.com> wrote in message
> news:xxpllbqjpjf.fsf@usrts005.corpusers.net...
> > "Clay S. Turner" <Physics@Bellsouth.net> writes:
> >> [...]
> >> The idea is to try to make each symbol contribute equally to the overall
> >> process. Imagine looking at your data after a huge number of symbols was
> >> received. The idea is to make the info provided by each type of symbol
> >> contribute equally.
> >
> > But this Huffman coding won't do that. Choosing a representation for a
> > symbol doesn't change the probability of the symbol occurring. It
> > does, however, minize the average symbol rate - I certainly see
> > that. Perhaps I'm being blind?
> > --
>
> Hello Randy,
>
> The Huffman problem and the quantization problem are related in they both
> implement methods to effectively flatten out variations. True they are
> different in that the Huffman problem is trying to minimize the average
> number of bits per sample and the quantization problem is trying to maximize
> the information per sample. The differences arise from the constraints and
> what you are starting with. In one case we have a fixed amount of
> information, so how few total bits can we fit all of the info in. The
> theoretical answer is the entropy, but the practical answer (comes from
> requiring whole numbers of bits in a symbol is the Huffman entropy.) In the
> other case we have a fixed symbol size, so how can we get the most info per
> symbol. The connection is the entropy and its maximization requires a flat
> distribution.
>
> In Huffman coding we sort the symbol probabilities into descending order and
> then we make multiple passes throught the list each time combining the two
> lowest probilites together. So we are essentially trying to make each path's
> probability be the same or at leat similar. In the quantization we are just
> diving up the total probability into N equal regions.
>
> You are not being blind, you are just asking how two seemingly different
> things are related. And the relation is through a flattening out of the
> probabilty function.
>
> I hope this helps to clear some things up.
>
> Clay

Intriguing stuff! I'm not sure I follow the sorting description, but
that's OK. I need to go back and read fully and with undrstanding
the seminal Shannon papers. 

-- 
Randy Yates
Sony Ericsson Mobile Communications
Research Triangle Park, NC, USA
randy.yates@sonyericsson.com, 919-472-1124

Relevant Pages

  • Re: how to find the best ADC step size?
    ... > symbol doesn't change the probability of the symbol occurring. ... The Huffman problem and the quantization problem are related in they both ... theoretical answer is the entropy, but the practical answer (comes from ...
    (sci.math)
  • Re: how to find the best ADC step size?
    ... >>The Huffman problem and the quantization problem are related in they both ... >>number of bits per sample and the quantization problem is trying to maximize ... >>diving up the total probability into N equal regions. ...
    (sci.math)
  • Re: The core of the method examined
    ... Now the math behind the huffman is known to me. ... The probability that a 0 follows a zero is higher than that that a one follows a zero in my input. ... The file size of the huffman output is not given by that, but rather by the average output size, multiply the probablity of the input (which cannot be derived from the probability of a zero or one alone) times the encoded size. ... However ANY BINARY COMPRESSION SYSTEM CAN BE USED THERE. ...
    (comp.compression)
  • Re: Still on the Arithmetic coder
    ... >> I assume the right probability for the symbol is known ... ... >> arithmetic coding would just produce the right amount of bits. ... For the file where A is .5 there is zero error the huffman ... My Compression code http://bijective.dogma.net/ ...
    (comp.compression)
  • Re: Still on the Arithmetic coder
    ... >>> I assume the right probability for the symbol is known ... ... >>> arithmetic coding would just produce the right amount of bits. ... >>> Huffman would produce full bits and in an optimal case this could ... > of rounding are we using here. ...
    (comp.compression)