For example, an agency was comparing its producers with a study showing the average producer's book was $300,000. Was its $250,000 producer a poor performer? Possibly, except she made $250,000 in half of the time. How about another producer with $350,000? Is he great? Perhaps, except his book was given to him. Possibly the biggest problem with averages is the general assumption that every statistic can be plotted along a normal curve. Some sophisticated users exacerbate this mistake by assuming normal levels of confidence and variation at either tail. The truth is not all curves are normal and we know that in reality, especially in financial markets, even when using a normal curve, the extreme tails do not always work the way a normal curve predicts. The lesson: When a study only shows averages, very little weight should be placed on the results if one is trying to determine cause and effect. Context is critical
Changing the context enables people to purposely mislead others. For example, if I wanted to pump up an agency owner, I could expound on how his agency's 88 percent retention rate was great, without mentioning that most of his competitors are doing even better. At 88 percent retention, the reality is the agency is likely doing something wrong. Another great example is, "Our producers are awesome! They each wrote more than $200,000 in new commissions last year!" Are the producers new or established? Is this a small or large agency? Does this include program business? And most importantly, what difference does it make how much new business the agency writes if all of it goes out the back door? I've met awesome new business producers who could write the equivalent of 30 to 40 percent of their books in new business each and every year. Of course, their retention rates were around 65 percent. New business only counts if it is retained. These are important issues that greatly affect the terrific-sounding "$200,000 in new business" statistic. The lesson: Check the context of the statistic. Do factors exist that might mitigate the statistic's usefulness? Mixing & matching
Be cautious of studies that compare apples to oranges. A common and very misleading mismatch is to use EBITDA (earnings before interest, taxes, depreciation and amortization) to compare companies growing organically to those growing by acquisition. The problem is while EBITDA excludes almost the entire cost of acquisition growth, it does not exclude the cost of organic growth. I do not have space to go into all the details here, but suffice it to say that when comparing the two types of growth using EBITDA as a measure, the acquiring firm's EBITDA is virtually guaranteed to look hugely better. The key is to look at cash flow. When firms grow organically, their cash flows and profitabilities usually are very close to the same. When a firm grows by acquisition, cash flow often is much less than profits. But because too many people do not understand the implications of EBITDA versus cash flow, a lot of bad acquisitions, growth decisions and even loans have been made.

Special Reports /

For the Manager- Good statistics are key to management

Related Stories

Recommended For You