Wednesday, February 23, 2011

How Not to Abuse Statistics

This past weekend, former Speaker of the Montana House of Representatives wrote a guest column that appeared in the Bozeman Chronicle. Read it here.

As a social scientist, I stress to my students the importance of using statistics correctly when developing an argument. To use statistics properly and to use them well, it is incumbent upon the person making the argument to place those statistics in their proper context.

What is notable in Mr. Sales' column is how he uses context when it serves his argument, and ignores context when it does not. I should make it clear that Mr. Sales is not alone in doing this; folks on both the right and the left are guilty of cherry picking statistics.

In any case, I wrote the following response which appeared today in the Chronicle:

To the Editor:

I read with interest Mr. Sales’ guest column yesterday. Although Mr. Sales provides a lot of information suggesting that the state of Montana is profligate, he fails to put those figures into their proper context. One example makes the general point: Mr. Sales makes much out of the fact that 780 state employees earn more than $100,000, a number that has increased by 278 since 2005. This is not justifiable, he claims, given that Montana ranks in the bottom 25 percent in personal income.

Mr. Sales fails to explain, however, who exactly receives such “outrageously” high pay. A quick perusal of the one hundred highest paid employees in the state (a search readily available at the Montana Policy Institute’s website at reveals that nearly all are doctors, psychiatrists, lawyers, judges, and top administrators—some of whom have worked for the state for more than two decades. The accurate yardstick with which to judge these salaries are national or state medians for these positions—not the median salary of all Montanans. These positions require extensive education and experience, and the amount Montana provides in compensation is not out of line with private sector salaries. In fact, these state salaries are likely much lower than the amount such professionals could obtain in private practice. Many likely serve the state of Montana out of a sense of civic duty and pride.

I do not disagree that Montana faces some tough budgetary choices; however, factual arguments divorced from their proper context certainly do not return us to the “values that [have] served us so well.”

Context matters. Indeed, it is everything.


Matthew N. Paine said...

As eloquent and cognizant as ever, Dr. Parker. I have strayed from the blog recently and don't know why, as it is indispensable in a town void of neutral fact representation. Thanks for writing, hope all is well!

-Matt Paine

Anonymous said...

Most striking to me in Mr. Sales' editorial was the lead sentence to his summation: "We must remember that before government can give to one citizen it must take it from another, thus gaining control of both."
It's a dizzying leap of ideological faith to go from the simple and bias free logic of the part before the comma to the blatant bias of that part after the comma. Insert "church" or "marketplace" for "government" and the fallacy of the Sales' conclusion is evident.