The Problem with Statistics
January 11th, 2011
I’m sure you’ve heard the saying “90% of statistics are made up on the spot” or “You can prove anything with statistics.” To some extent it’s true. Unlike other forms of math, a problem in statistics doesn’t always yield a single answer. Subjective isn’t exactly the right word to describe the science, but it is true that it’s by no means objective like calculus.
The problem with statistics, and even more specifically the form of statistics applied to economics, also called econometrics, is that even when we try our best, with the very best intentions of objectivity, it’s still very hard to get right.
Take the unemployment rate, a particularly contentious and critical statistic in U.S. policy at the moment. The unemployment statistic measures the percent of the world force that does not have a job, is actively searching for employment, and is available to work. As many newspapers and pundits have noted, this leaves some pretty gaping holes, including: part time workers who are seeking full-time employment, students who are unemployed and “discouraged” workers who have been unemployed for so long they are no longer looking. This last category has become particularly significant in the last few months, as the relatively optimistic unemployment statistics may unintentionally picking up an increase in the number of discouraged workers, not a rise in the number of available jobs.
Another good example of a contentious statistic is the Consumer Price Index (CPI), which measures inflation by tracking the changes in prices paid by consumers for a basket of goods and services. In November when Sarah Palin suggested inflation was on the rise, using the hypothetical example of grocery shopping, a number of economists objected. Among them was Sudeep Reddy, a Wall Street Journal reporter, who countered that in fact “grocery prices haven’t risen all that significantly” and he pointed to the CPI for evidence.
The truth is it’s possible for either of them to be right. The CPI is by no means perfectly representative sample of consumer goods and services in the country, which by definition makes it a biased estimate. While there is no reason to believe the CPI systematically or significantly under- or over-estimates the price level, even a small margin of error either way could have significant implications for public policy. That said there is no reason to believe the CPI is grossly underestimating inflation, as sensationalist groups like the National Inflation Association have insisted and this ridiculous YouTube video (which as of today has received almost 4 million hits) has claimed. Economists can and do get it wrong sometimes, and with significant consequences, but when discussing U.S. national economic policy let’s not throw around words like “conspiracy” and “scam.”
Studying statistics like the unemployment and the inflation rates is not just an academic exercise. These statistics have vital consequences for the economy, and ultimately citizens’ well-being, as they provide benchmarks by which policy-makers construct public policy. For example, when the Federal Reserve announced its $600 billion plan to battle unemployment it did so in light of looming deflation (as measured by the CPI) and stagnating unemployment. If either of those statistics is wrong, then the Federal Reserve just made 600 billion mistakes.
The point of this story is that economists make mistakes. They are not intentional mistakes. Trust me they’re all doing their best. But because economics is not a perfect science (and because statistics is not a perfect math) differences of opinions do exist. And so economists make mistakes, which are often not without huge consequence.
When it comes to the study of illicit financial flows (IFFs), this example could not be clearer. In traditional models, economists net illicit inflows from illicit outflows, resulting in a lower “net” estimate of capital flight. This is a skewed picture because illicit outflows, because they are illegal by definition, are not supplementing the domestic economy in the same way an illicit outflow is detracting from it. For example, if you were an economic advisor in Colombia, would you tell President Juan Manuel Santos that the money flowing into his country as a result of the drug trade is offsetting the negative effect of the money flowing out? No. If anything the two numbers should be added.
Economists often get caught up with the status quo or as we call it the “established literature.” Because all of the previous models of capital flight have netted outflows from inflows, economists who follow will continue the tradition. This kind of herd mentality is damaging to economics and damaging to international development as it drastically understates the damage to developing countries as a result of IFFs. Thankfully, we can look to organizations like Global Financial Integrity who are bucking the trend and are following reason instead of the crowd.