Re: Math research, passion is important

From: Quinn Tyler Jackson (quinn-j_at_shaw.ca)
Date: 06/19/04 

Date: Sat, 19 Jun 2004 18:03:50 GMT

I said:

> > I suspect (though I may be way off -- I'm feeling under the weather)
that
> > you, Will, have a sense that if I claim to have invented such an
algorithm,
> > it's likely that it's *not* a bluff. But that's neither here nor there,
I
> > suppose. Nobody wants to test my outrageous claim -- I can handle that.
It's
> > always good to have one unorthodox ace in the hole. ;-)

Will Twentyman said:

> I don't find anything outrageous about claiming to invent algorithms.
> It's part of being a programmer.

Yup. Claiming to invent efficient Type < 2 parsing algorithm, however, is
going a bit far. ;-) Not impossible -- but highly unlikely that someone like
me would have come up with it.

> I find it odd that you describe testing various algorithms as "taking a
risk",
> however. I'd just call it experimentation.

I've seen mathematicians abandon paths of research because they didn't see
promise of it being fruitful in their quest for publication. If they had
just stuck to it a bit longer, they may have found ways around the
obstacles.

For instance, back some years ago, I was challenged to produce an efficient
Type < 2 algorithm that could parse a test case FASTER than the Type 2
equivalent with ad hockery used to handle the context-sensitive parts. My
algorithm showed exponential time complexity, the challenger's showed the
expected O(n) complexity.

It was a daunting challenge. The "risk" was that I might waste a huge amount
of my valuable programming and theorizing time trying to make my algorithm
O(n) for the test case.

Slowly I worked out the kinks in my algorithm. To do this -- I had to fail
many, many times, and each time I failed, I had to judge the risk of
committing more time to the problem. But to do it, I also had to formalize
my work and look at my algorithms formal properties. This required me to
invent the $-Calculus. This required me to submit my theory to peer
scrutiny. This required me to risk being taken for a fool and/or crank.

And then, one day, I hit the wall: my algorithm was linear, but the constant
portion was still slower than the test case. Years later, I'd bent the curve
flat, but that damned constant portion was still there. More risks of time
and ... finally, my parser was faster with its Type < 2 grammar than the
challengers Type 2 version.

Plenty of risks there. Very, very few formal theorists encouraged me --
those who did only did so with due caution.

The final result was an algorithm that works for many Type < 2 languages,
and a formal theory that introduced a fifth computational machine into the
Chomsky Hierarchy. (The PDA-T.) The risk hasn't gone away. There's still a
"chance" I've wasted my time. But at least I have a parser that is formally
Type 0, and can literally "parse anything" that can be parsed, entirely in
the formalism, without ANY ad hockery and marginalia, and can do so in what
appears to be minimal time for the language in question.

Experimentation can entail risk. My observation about risk taking is that
many pure theorists avoid such risk and discourage others from taking them.
If you take a risk -- you may be shown to be wrong -- and if you are shown
to be wrong -- you can't get published -- and if you can't get published --
you can't get tenure. 

--
Quinn

Relevant Pages

  • General Cardiovascular Risk Profile for Use in Primary Care
    ... General Cardiovascular Risk Profile for Use in Primary Care. ... "Background—Separate multivariable risk algorithms are commonly used to assess risk of specific atherosclerotic cardiovascular disease events, ie, coronary heart disease, cerebrovascular disease, peripheral vascular disease, and heart failure. ... The present report presents a single multivariable risk function that predicts risk of developing all CVD and of its constituents. ... Conclusions—A sex-specific multivariable risk factor algorithm can be conveniently used to assess general CVD risk and risk of individual CVD events. ...
    (misc.health.diabetes)
  • Re: How good an encryption algorithm is this?
    ... You're saying that none of the examples of using CryptoAPI work? ... >> own algorithm to start with. ... > have known not to bother trying to invent their own", ... I don't think we need any more reasons not to invent your own ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: How good an encryption algorithm is this?
    ... You're saying that none of the examples of using CryptoAPI work? ... >> own algorithm to start with. ... > have known not to bother trying to invent their own", ... I don't think we need any more reasons not to invent your own ...
    (microsoft.public.vc.language)
  • Re: How good an encryption algorithm is this?
    ... > own algorithm to start with. ... have known not to bother trying to invent their own", ... > attempt to crack it doesn't mean it's more secure than one which many ... > CryptoAPI in the first place? ...
    (microsoft.public.vc.language)
  • Re: How good an encryption algorithm is this?
    ... > own algorithm to start with. ... have known not to bother trying to invent their own", ... > attempt to crack it doesn't mean it's more secure than one which many ... > CryptoAPI in the first place? ...
    (microsoft.public.dotnet.languages.csharp)