Chetty's Article Falls (Uncharacteristically) Short

Raj Chetty offers an article in the NY times arguing that economics should be considered a science. Chetty points out that there are unknowns in many fields that are indisputably "scientific": 

"It is true that the answers to many “big picture” macroeconomic questions — like the causes of recessions or the determinants of growth — remain elusive. But in this respect, the challenges faced by economists are no different from those encountered in medicine and public health. Health researchers have worked for more than a century to understand the “big picture” questions of how diet and lifestyle affect health and aging, yet they still do not have a full scientific understanding of these connections. Some studies tell us to consume more coffee, wine and chocolate; others recommend the opposite. But few people would argue that medicine should not be approached as a science or that doctors should not make decisions based on the best available evidence."

Chetty basically gets it right in my opinion. Economics should be considered a science. However, Chetty's own article provides an example of why so many find it hard to view economics in this way.

Chetty brings up three examples of what a "scientific" approach to economics has supposedly taught us:

"Consider the politically charged question of whether extending unemployment benefits increases unemployment rates by reducing workers’ incentives to return to work. Nearly a dozen economic studies have analyzed this question by comparing unemployment rates in states that have extended unemployment benefits with those in states that do not. These studies approximate medical experiments in which some groups receive a treatment — in this case, exten ded unemployment benefits — while “control” groups don’t.

These studies have uniformly found that a 10-week extension in unemployment benefits raises the average amount of time people spend out of work by at most one week. This simple, unassailable finding implies that policy makers can extend unemployment benefits to provide assistance to those out of work without substantially increasing unemployment rates.

Other economic studies have taken advantage of the constraints inherent in a particular policy to obtain scientific evidence. An excellent recent example concerned health insurance in Oregon. In 2008, the state of Oregon decided to expand its state health insurance program to cover additional low-income individuals, but it had funding to cover only a small fraction of the eligible families. In collaboration with economics researchers, the state designed a lottery procedure by which individuals who received the insurance could be compared with those who did not, creating in effect a first-rate randomized experiment.

The study found that getting insurance coverage increased the use of health care, reduced financial strain and improved well-being — results that now provide invaluable guidance in understanding what we should expect from the Affordable Care Act.

Even when such experiments are unfeasible, there are ways to use “big data” to help answer policy questions. In a study that I conducted with two colleagues, we analyzed the impacts of high-quality elementary school teachers on their students’ outcomes as adults. You might think that it would be nearly impossible to isolate the causal effect of a third-grade teacher while accounting for all the other factors that affect a child’s life outcomes. Yet we were able to develop methods to identify the causal effect of teachers by comparing students in consecutive cohorts within a school. Suppose, for example, that an excellent teacher taught third grade in a given school in 1995 but then went on maternity leave in 1996. Since the teacher’s maternity leave is essentially a random event, by comparing the outcomes of students who happened to reach third grade in 1995 versus 1996, we are able to isolate the causal effect of teacher quality on students’ outcomes.

Using a data set with anonymous records on 2.5 million students, we found that high-quality teachers significantly improved their students’ performance on standardized tests and, more important, increased their earnings and college attendance rates, and reduced their risk of teenage pregnancy. These findings — which have since been replicated in other school districts — provide policy makers with guidance on how to measure and improve teacher quality."

Chetty's presentation of these findings is a perfect example of why it is hard to see economics as a traditional science. Take his analysis of the literature on unemployment benefits. As far as I can tell, his first claim, that a 10 week extension only modestly raises the time workers choose to stay out of work, is correct.

However, let us look at a very new NBER paper on the effect of unemployment benefits that also uses what Chetty would identify as a scientific approach. Here is the abstract:

"We exploit a policy discontinuity at U.S. state borders to identify the effects of unemployment insurance policies on unemployment. Our estimates imply that most of the persistent increase in unemployment during the Great Recession can be accounted for by the unprecedented extensions of unemployment benefit eligibility. In contrast to the existing recent literature that mainly focused on estimating the effects of benefit duration on job search and acceptance strategies of the unemployed – the micro effect – we focus on measuring the general equilibrium macro effect that operates primarily through the response of job creation to unemployment benefit extensions. We find that it is the latter effect that is very important quantitatively." (emphasis added)

This is very important because this paper actually does not dispute the empirical claim that Chetty made, but it does dispute the policy conclusion he came to based on that empirical claim. The fact that unemployment benefits only modestly increases the time workers want to stay out of the market does not automatically imply that dramatic increases in the duration of unemployment benefits will not have significant negative effects on employment. Indeed, as this paper shows using just the kind of method Chetty recommends, looking at other channels through which unemployment benefits can affect employment yields a much less positive result for advocates of expanding these benefits.

With regards to Medicaid, Chetty also paints a surprisingly incomplete picture of the Oregon Medicaid experiment. As you will recall, Chetty is correct in pointing out that expanding Medicaid seems to have increased usage of health care, decreased financial strain, improved mental health, and improved self reported well being, but he, quite surprisingly given the caliber economist Chetty is, leaves out the less flattering (for supporters of the ACA) part of the study that found no statistically significant increase in objective measures of physical health for patients who received Medicaid.

At best, the Medicaid study was a mixed result for supporters of expanding the Medicaid program (which the ACA does quite dramatically). At worst, the study is a sad demonstration of how bad Medicaid (and perhaps insurance in general) is at improving objective physical health. Why Chetty presented this study as an unambiguous victory for the pro Medicaid crowd is a mystery to me (although I suspect support of ACA has something to do with it)?

Of course, the benefits that Medicaid did bring were quite large, but one would imagine a smaller Medicaid program that relied more on cash transfers would still have those benefits while perhaps leading to more efficient use of health resources, but that is a topic that deserves it's own post.

As for the study on the benefits of good teachers, I have not looked into this particular issue and thus have no comment on this.

My goal here is not to "take down" Chetty's arguments. Instead, I would like to offer an explanation for why economics is not often thought of as a traditional science: it is too political. Chetty is among the most respected economists in the world (as he should be). If any economist could be politically objective, Chetty would fit the bill, but we still see political bias seeping through in this article.

Economics is a science, but it is a very politicized science. The Medicaid study, with its ambiguous results, offered justification for the policy proposals of both supporters and opponents of ACA, for example. Both sides were offering an incomplete picture of the study in this debate, but both sides were also correct the claims they made even if they strategically left out inconvenient findings.

Economics certainly can be seen as a science when it comes to making observations about the world, but when it comes to recommending certain policies, economics is only as scientific as the biases of the economists allow it to be.