Re: Due Diligence

From: Major Zed (MajorZedMusic_at_hotmail.XSPAMX.com)
Date: 08/17/04 

Date: Tue, 17 Aug 2004 09:30:22 -0400

Richard Ulrich wrote:
> Major,
> There are a lot of different specialties in statistics, and there
> are a lot of different problems in each specialty. 'Bits and
> pieces' like data quality or regression diagnostics arise in
> all the different areas, but with (sometimes) vastly different
> relevance. If we know that you are analyzing credit reports
> (for instance), that puts a particular slant on what is useful,
> necessary, or expected by convention. And, what is known
> about the area.

> When I hear 'financial data,' I immediately wonder if *you*
> are invoking all the problems of time-series analyses. Frank
> Harrell's textbook (which Mike B recommended) is good for
> logistic regression (which was explicitly mentioned), and good
> for model building, not only if that is the kind of model you
> end up with. But it may not say as much as you need about
> time series, and your consultant *may* see your own
> problems as something different.
>
> So, "What is the problem?" - can make a lot of difference to
> how it is *apt* to be approached. I figure that the particular
> approach varies a lot with the N (sample size); and whether
> the problem has precise prediction or has fuzzier relationships,
> possibly because of weak measurement (can the measures be
> improved?).

Let's say we're analyzing credit quality as the criterion and one or two
year's worth of financial statement data as the predictors. No time
series analysis.

> How do you measure good consulting? You might ask for a
> consultant as a number cruncher, but then you probably
> won't get a real statistician, or be in a position to hold this
> consultant responsible for much.
>
> Are you asking for help in defining the problem? - A good statistical
> consultant will ask you to re-think the problem out loud, for his
> benefit; and you should either learn something from his questions,
> or recognize that he has covered the known area.

The problem is fairly clear-cut. The goal is a model; input=financials,
output=probability of default in the next year. The consultant is
expected to be knowledgeable about / experienced in the domain already.

> The 'work product' depends on various things. Is there
> an in-house expert (you?) who is expected to understand
> the details of printouts?

Yes (yes)

>The Harrell book might help for that.
> Or, do you get a working formula, while the consultant stays
> indefinitely on-call?

yes, that too.

>Well, the formula ought to work, and
> even before that, it ought to seem sensible -- If there are
> screwy variables included, that *might* be a bad sign.

To be sure. I can perform this kind of oversight based on my
experience, but it would be guided more by intuition than by something
formal. Looking for a more comprehensive, formal, set of guidelines.

> Hope this helps.
>

I truly appreciate your taking the time to engage in this dialog.

M*Z

Relevant Pages