Re: Rookie having problems with some filter code. Any help?

In article <dno3mv$bae$2@xxxxxxxxxxxxxxx>,
davem@xxxxxxxxx (Dave Martindale) wrote:

> Don Bruder <dakidd@xxxxxxxxx> writes:
>
> >However, you bring up a valid point - one which I've been struggling
> >with - How to cope with (A) color values that go negative while being
> >"cooked", and (B) values that overflow the available 8 bits while being
> >cooked. For situation B, a simple "value % 256" leaps to mind as the
> >proper treatment. But coping with values that land in "negative-ville"
> >as a result of being cooked, whether they are or are not possible to fit
> >into 8 bits, is something that has me completely stumped.
>
> If you're just going to write out the image again, negative values
> should be clamped to zero, and values greater than 255 should be clamped
> to 255 - not wrapped around to zero.

Hmmm... So simple clamping is, despite the counterintuitiveness
involved, the way to go? Without chasing every possiblity, it seems
likely to me that would end up with a whole slew of pixels whose
color-component values are pegged at either 0 or 255, with comparatively
little "middle ground".

> But if you're going to do a chain
> of operations, and you don't have to convert back to 8 bits unsigned
> between each operation, there is sometimes a benefit to carrying the
> unclamped values through the whole sequence of steps and clamp only at
> the end.

OK, that does make at least SOME sense, but it still hits the same wall
mentioned above - At first glance, it would seem that clamping,
particularly in a "many-control-one" situation needing many
multiplications (some of them possibly involving negative multipliers)
and additions to arrive at a value, whether it happens "in the middle"
or "at the end" of an ops sequence, would tend to skew things pretty
strongly to the extremes of the range, eating the bejeebers out of the
"middle" values in the process.

> There are occasions (e.g. differencing) when keeping the result mod 256
> is the right thing to do - but that's not usually the case.

Are you saying "differencing" to mean what I think you are? As in "XOR",
subtract, or otherwise combine two frames (or segments of frames) -
let's call them "F1" and "F2", and the "combined" version that results
"R" for convenience - so that if a pair of pixels, F1[x,y] and F2[x,y],
share a common RGB value - let's say they're both r=101, g=30, b=222 -
then R[x,y]'s color will be set to <some arbitrary color value>? (which
will probably, but not necessarily, be white or black due to common
practice, but whatever it actually is, the intent is to convey a
"nothing to see here" message.)

Or am I *WAY* out in left field?

--
Don Bruder - dakidd@xxxxxxxxx - If your "From:" address isn't on my whitelist,
or the subject of the message doesn't contain the exact text "PopperAndShadow"
somewhere, any message sent to this address will go in the garbage without my
ever knowing it arrived. Sorry... <http://www.sonic.net/~dakidd> for more info
.