Re: Quartile Algorithms, Samples and Data Filtering
- From: "David A. Heiser" <daheiser@xxxxxxx>
- Date: Sat, 23 Jul 2005 15:39:06 -0700
"IanC" <i_cheyne@xxxxxxxxxxxx> wrote in message
news:1121952241.246609.148840@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>
> I'm looking for some guidance as to how much weight to put on
> statistical analysis of this issue.
>
> The question I think is whether statistical analysis and techniques
> apply to a situation where the data is extracted using selection
> criteria. I'd call this filtering rather than sampling - but I'd
> appreciate your comments.
>
> First of all I'd like to declare I am in ICT - with just enough
> basic knowledge of stats to be dangerous - and I'm sorry but I'll
> have to describe the issue at length in some detail.
>
> We have a typical commercial arrangement where we can test the price of
> services we receive by going to an independent third party - the
> Benchmarker - who selects peers from their database of studies, and
> using the costs of the selected peers provides a benchmark price for
> comparison with the price we pay. Of course there a number of agreed
> normalisation and adjustment steps involved in turning peer costs into
> price but that is where the benchmarker's expertise comes into play.
>
> The difficulty is that we have now agreed that the price will be set
> at the quartile (Q1) of the peer organisations - and have only
> subsequently become aware of the different algorithms used to calculate
> quartile. There seems to be a variety of algorithms that range from
> n/4 to (n+3)/4 and with the small number of peers selected there can be
> a significant variation in the resulting price. For your information,
> applying the selection criteria will typically result in around ten
> organisations selected as peers
>
> The issue is really about whether or not we should use statistical
> analysis to select an appropriate algorithm for a small number of
> peers, or simply treat this a commercial discussion to select a formula
> that meets the contractual definition (which just says split the peers
> 25% / 75% using interpolation).
>
> Where I am having difficulty is the way peers are selected. The
> benchmarker has a database containing studies of many organisations
> that go back over a number of years. These studies are of all sorts of
> organisations, many are not suitable to be compared against us.
>
> The selection criteria is to find those that match in size and
> complexity, then look at performance criteria so we are looking at
> similar services and service levels, preferably from the same industry
> group, and also the date of the study so we align with our study
> period.
>
> The ten or so that make it through this filtering exercise are
> designated the peer organisations, and the benchmark price is
> calculated using data from these peer organisations.
>
> It seems to me that with this type of filtering it would not be correct
> to treat the peers as samples and the benchmarkers database as the
> population, and attempt to analyse the algorithms as if this were the
> case - for example trying to look at variation and bias between the
> quartile of the sample and the quartile of the population.
>
> What are your views? Does the filtering exercise make statistical
> analysis invalid, weaker or what? Can the peers selected be regarded
> as samples? Any insights would be most appreciated.
>
> Regards,
>
>
> IanC
>
Establishing a business plan from a set of data is entirely an arbitrary
process. You can't say that any element in the logic structure is true or
false, right or wrong. Choice of a statistical process again is arbitrary,
since what is desired is not "statistical properties of a sample/population"
using statistical methods described in an introductory statistics book.
Because the terms "sample" and "population" and "quartile" are used, does
not in any way mean that the definitions in a statistics book apply.Your
description clearly describes "non-statistical" usage of these terms.
Therefore there is no mathematical structure to arrive at gate B from your
starting point.
In the business and political world there is a great deal of these
"arbitrary structures". The only thing that holds them together is some
explanation and rational that all users accept. The only way these change is
when some "smart..." figures a way sell a method that shows lower costs and
ensures that his company always gets the bid".
In your model, there is no criteria for a measure of quality or successful
project completion anywhere..
DAH
.
- Follow-Ups:
- References:
- Prev by Date: Re: Estimating variance with limited data?
- Next by Date: trimmed mean
- Previous by thread: Quartile Algorithms, Samples and Data Filtering
- Next by thread: Re: Quartile Algorithms, Samples and Data Filtering
- Index(es):
Relevant Pages
|
