Bad Outcomes is Using Bad Logic

Charlie Clarke and his co-blogger, Robert H, are both sure that I'm either committing some sort of statistical fallacy or suffering from some misunderstanding. The thing is that I'm not. Instead, both Robert H and Charlie Clarke are using very faulty logic and misunderstanding my position.

They are both convinced that the only relevant results from the Oregon Medicaid study are the positive results on mental health, self reported health, and financial hardship. In other words, the only relevant results are the ones that reinforced their preexisting beliefs about Medicaid. The results that are less convenient for their worldview (the ones on physical health) should be thrown out.

The justification for this ideologically convenient analysis of the study is that the lack of positive results on physical health supposedly tells us absolutely nothing about the effect that Medicaid has on physical health. Obviously, that is a rather absurd view. In fact, a correct reading of the results on physical health do make claims of Medicaid improving physical health dramatically much weaker.

Typically, people are guilty of overstating their case. Oddly enough, as I read back on this exchange and look at the evidence, it seems that I have been understating my case. The lack of positive results on Medicaid is indeed a rather poor result for Medicaid. This was not a small sample size by any means. The study was not, by any reasonable measure, underpowered. If Medicaid made substantial improvements in health, it should have shown up this study.

As I have said, this does not mean that this experiment disproves significant health improvements stemming from Medicaid. It does not. However, it does significantly weaken the justification for belief in such effects.

The more I read Clarke and H, the more it seems that what they are really saying is "the results don't matter unless positive results show up". In other words, heads they win, tails we can't draw any conclusions from.

If these results did turn up significantly positive, there is only one likely conclusion that we could come to:

1.) Medicaid significantly improves physical health relative to being uninsured.


If they turned up with no statistically significant effects, there are really only two likely conclusions:

1.) Medicaid does not significantly improve physical health relative to being uninsured.

2.) Medicaid does significantly improve physical health relative to being uninsured, but not by enough to have significant results in this study.

Clarke and H seem to have come to the rather absurd conclusion that the results would only have mattered if they came out statistically significant in some direction. That is incorrect. Virtually nobody would have predicted that Medicaid would have significantly reduced physical health. Instead, the critics of Medicaid would predict a lack of major improvement.

I'll also add that I have let Clarke off the hook for ignoring the raw numbers that actually show Medicaid reducing certain measures of health. Of course these numbers are not statistically significant, but they are closer to statistical significance than most of the positive results on physical health. If Clarke is going to claim that these raw numbers suggest improvements in physical health, he needs to also claim that they suggest Medicaid hurts the cardiovascular health of the already sick and increases smoking. (more on that here). Of course, I think it is obviously wrong to suggest that this qualifies as evidence that Medicaid hurts cardiovascular health among the sick or increases smoking, but that is the logical conclusion that Clarke's arguments should lead him to.

By the way, there's been another round of results from the Oregon study that don't find statistically significant changes in employment resulting from Medicaid. I'm waiting for Clarke and H to explain to the NYtimes that they really shouldn't be reporting on these results as they don't really tell us anything about the effects that Medicaid has on employment. I would agree with the NYtimes here. But, if the folks at Bad Outcomes are to be logically consistent, they should be accusing them of committing all kinds of statistical fallacies. Don't hold your breath.

Yet Again on Oregon Medicaid Study

Charlie Clarke responds again. One excerpt:

"In Anon Ymous's  latest response, it seems he feels he's being misunderstood.  Yet, he keeps making statements that demonstrate exactly the view I'm arguing against.

"The Oregon study did not find evidence of statistically significant improvements in physical health. That is a pretty significant result."

No, this is exactly what I've criticized.  If you run a test that has low power, it is NOT significant if you find no effect.  Suppose you have a magic pill that stops you from dying, you give it to 10 undergrads and use 10 as a control group and measure the effect after one year.  You find no statistically significant effect.  Is that finding significant?  NO!!!  Even if the pill prevents 100% of all deaths, you'd have to give it to thousands of undergrads to get statistical significance, because undergrads just don't die very often."

I don't think Clarke understands that I am agreeing with his core point, that the Oregon study did not prove Medicaid does not improve physical health. Based on this study, we can't rule out the possibility that Medicaid might improve health. However, we can say, based on this study, that even if Medicaid improves health, it does not improve health dramatically enough to be captured in what was a fairly large study. Indeed, it was done, as Clarke suggests in his hypothetical, on thousands of individuals, and there was still not statistical significance. 

It is possible that they would have found statistically significant improvements in physical health if there were more individuals in this study. It is possible they will find statistically significant improvements in physical health when looking at a larger time horizon. However, that is not the study we have. For now, we have a study that has a sample size in the thousands and a time horizon of a few years. In this study, there was not evidence found of statistically significant improvements in health. 

Clarke would be well served in this argument to just come out and admit that there wasn't evidence, in this study, of significant improvements in physical health. Instead, Clarke resorts to arguing that the numbers provide just as much support for the view that Medicaid substantially improves physical health as to the view that Medicaid does not substantially improve physical health:

"For high blood pressure, there is a fall from 16.3% of people to 14.97% of people.  That's an 8% reduction in incidence of high blood pressure.  It's more consistent with a "bold claim" of up to a 16% reduction of high blood pressure than it is a zero percent reduction.

For diabetes, there is a fall from 5.1% to 4.17%.  That's an 18% reduction in the incidence of diabetes.  That's more consistent with a "bold claim" of up 36% reduction in the incidence of diabetes than no reduction in the incidence of diabetes."

 This is a point I responded to a few days ago in a previous post:

"The Oregon Medicaid Study did indeed show no statistically significant improvement in objective measures of physical health. I am a bit surprised that so/ many people have taken issue with this assertion. The common response is that there were positive results, but they just weren't positive enough to be statistically significant. In order to be statistically significant, results have to have less than a 5% chance of being statistical noise. In the Oregon study, the positive results on elevated blood pressure had a 65% chance of being statistical noise, the positive results on elevated blood sugar had a 61% chance of being statistical noise, and the positive results on high cholesterol had a 37% chance of being statistical noise. In other words, not only were the results not statistically significant. They weren't even close.

It's actually worse than that. If we just look at raw numbers and ignore statistical significance, we can also say that Medicaid increases smoking and decreases cardiovascular health of those that were already sick. The raw numbers suggest that. Of course, neither of these results are statistically significant, but they are closer than any of the other measures. There is only a 24% chance that the results showing Medicaid reducing cardiovascular health are statistical noise, and there is only an 18% change that the results showing Medicaid increasing smoking are due to statistical noise. These still are far from statistically significant, but if one is going to argue that the raw numbers that are statistically insignificant are useful in telling us the effect of Medicaid, then they should also be arguing that Medicaid increases smoking and worsens cardiovascular health of the already sick.

I would argue that we shouldn't read too much into any of the statistically insignificant results and instead focus on the statistically significant ones. By the way, that data is from Megan McArdle who had an excellent post on this very topic."

The raw numbers would also suggest that Medicaid increases smoking and decreases cardiovascular health for the already sick. Of course, these numbers aren't statistically significant, but they are closer to statistical significance than most of the positive results in the raw numbers. 

Using Clarke's logic, we could say that the study provides more evidence that Medicaid increases smoking and reduces cardiovascular health among the sick than evidence that Medicaid improves any of the other measures of objective physical health. 

For the sake of clarity, I want to lay out my exact position on this study:

1.) The Oregon study found statistically significant positive results when looking at self reported health, mental health, and financial strain. The Oregon study did not find statistically significant positive results when looking at objective measures of physical health.

2.) This does not rule out the possibility that a larger study over a longer time horizon would have found statistically significant positive results on objective measures of physical health.

3.) It does mean that this sample size of thousands over the given time horizon was not able to find evidence of statistically significant improvements in physical health.

4.) Now, while this does not rule out the possibility that a larger study would have found statistically significant improvements in physical health, it is also true that if there were very substantial improvements in physical health (anywhere near the size of the improvements seen in other areas) resulting from Medicaid, they would have likely been captured in this study.

5.) This study did capture significant improvements in other areas (financial strain, mental health, self reported health). This suggests that, at the very least, Clarke can admit that this study suggests that the improvements in objective physical health are not anywhere near as large as the improvements in the aforementioned areas.

As for the implications for public policy:

1.) This study does suggest that there is much value in having a program that provides health care for the poor. We can gather this from the results on mental health, financial strain, and even self reported health.

2.) However, this study does not tell us that Medicaid does a particularly good job of being this program. It does not tell us that a smaller Medicaid program that takes a very different form could not offer the same benefits without all the costs.

3.) This is why I say that the study, at the very least, slightly boosts the case for reform.

Now, let us go back to what Raj Chetty said:

"Other economic studies have taken advantage of the constraints inherent in a particular policy to obtain scientific evidence. An excellent recent example concerned health insurance in Oregon. In 2008, the state of Oregon decided to expand its state health insurance program to cover additional low-income individuals, but it had funding to cover only a small fraction of the eligible families. In collaboration with economics researchers, the state designed a lottery procedure by which individuals who received the insurance could be compared with those who did not, creating in effect a first-rate randomized experiment.

The study found that getting insurance coverage increased the use of health care, reduced financial strain and improved well-being — results that now provide invaluable guidance in understanding what we should expect from the Affordable Care Act."

Nothing Chetty says here is actually incorrect, as I said before. However, I would have to say that a vast majority of readers that know little about the actual Oregon study would finish reading this article with the impression that the Oregon study simply told us that Medicaid "reduced financial strain and improved well-being" and was therefore a vindication of Medicaid.

Clarke says that this a fair characterization of the study. I disagree. The study didn't prove that Medicaid did not improve physical health. However, it did look to see if there was any evidence that it did in any significant manner, and they did not find any. That result does matter. 

Clarke is wrong to say that we should just leave out that part of the study. It may not prove that Medicaid doesn't improve physical health, but it does tell us that a study that apparently was large enough to allow us to conclude that Medicaid brings about significant improvements in financial strain, mental health, and self reported health was not able to detect any significant improvements in physical health.

That does tell us something about Medicaid. And, it is not unambiguously positive. An economist of Chetty's caliber writing about why economics should be considered a science should have made a stronger effort to convey this point. After all, that's what any good scientist would do.

Even More on The Oregon Medicaid Study

Charlie Clarke offers up some criticisms of my interpretation of the Oregon Medicaid study (here and here). An excerpt:

"In a follow up to the post that started a long argument between Scott Sumner and I, Anon Ymous demonstrates exactly what you shouldn't conclude from the Oregon Health Study, "To put it simply, the Oregon study showed that Medicaid does a good job of protecting the poor from crushing medical expenses, but it doesn't make them healthier or save lives."

The correct statement is, "the Oregon study showed that Medicaid does a good job of protecting the poor from crushing medical expenses, but we don't know if it makes them healthier or saves lives."

Actually, the correct statement would be "the Oregon study showed that Medicaid does a good job of protecting the poor from crushing medical expenses, but it does not provide evidence that Medicaid improves physical health".

That isn't too different from what Clarke said, but we need to remember that the folks did look to see if there was evidence in this study that Medicaid improved physical health and found none. They didn't find evidence to the contrary either, which Clarke goes to great lengths to point out.

Let me also put Clarke's quoting of me in fuller context. This is from my last post on the subject:

"I also think many people assumed that I was making an argument based on the Oregon study that I was not. I was not trying to argue that the Oregon study was an indictment of Medicaid. I was simply arguing that it was not a vindication of Medicaid. We can quibble about the results, but this study was not a big win for Medicaid. That much should be clear. Chetty seemed to imply that this study was a big win for Medicaid in its current form, and that was the claim I took issue with.

What the study did indicate was that spending money on health care for the poor helps the poor (which should be rather obvious). The statistically significant results on financial security and mental health show that having a program of this sort is worthwhile.

To put it simply, the Oregon study showed that Medicaid does a good job of protecting the poor from crushing medical expenses, but it doesn't make them healthier or save lives.

This result does, I would argue, bolster the case for reforming Medicaid. If we, for example, moved towards a system where Medicaid provided catastrophic insurance plus a subsidy that can be used to purchase supplemental private insurance or start some sort of health savings account (as some states are already doing), this program would still protect the poor from crippling medical expenses. It also would offer them more choice, improve the quality of care they receive (which is big problem with Medicaid), and help constrain costs.

Of course, there are many other proposals for reforming Medicaid (Tyler Cowen has a interesting one as well). The main takeaway from this study is that there is nothing special about the form Medicaid currently takes, and that, therefore, we should not be afraid of looking at serious reform."

Here is more from Clarke:

"As far as the politics of Medicaid, this certainly doesn't mean that supporters are immune from criticism.  In a more perfect world, Scott and Anon Ymous would be arguing the effects estimated by this study are too small to be worth the money or much smaller that liberal so and so would have thought going in.  If even the high end of the estimates are not worth the cost, then that is a big feather in the cap of Medicaid opponents.

Alas, the commentary we get is on the order of: Study proves Medicaid doesn't make people healthier.  And then we get attacks on Raj Chetty for not parroting that false interpretation."

My issue with Chetty's interpretation of the study was the unambiguously pro Medicaid result he implied came with it. In fact, the study did not find evidence of improvements in phyiscal health. That doesn't mean that Medicaid doesn't improve physical health, but it does mean that fairly high caliber study found no evidence that it did. 

I don't know how many different times and different ways I have to explain that I am talking about the absence of a positive result and not the presence of a negative one. The Oregon study did not find evidence of statistically significant improvements in physical health. That is a pretty significant result. Obviously, it is not as bad as positive evidence that Medicaid does not improve physical health. Still, if we had the critics and supporters of Medicaid in its current form predict the results on physical health in advance, I think that it is fairly clear that the critics would have been closer.

We spend hundreds of billions of dollars on this program every year. Any attempt to reform or reduce the rate of spending growth in this program is met with claims that Medicaid saves lives and makes people much healthier. My response isn't: "the Oregon study proved that Medicaid doesn't save lives or make people healthier". Instead, my response is: "where is the evidence? The Oregon study didn't find any. Most other studies haven't either."

The defenders of Medicaid seem to think that the burden of proof is shared on this question. It is not. It is squarely on the shoulders of those making these bold claims about Medicaid, as they are the ones defending the status quo of a program that commands a lot of resources at least partially based on very bold claims about what good the program supposedly does. Claims that, thus far, are quite scant on evidence. 

The Oregon study wasn't proof that Medicaid doesn't make people healthier. If Clarke thought I was implying otherwise, I apologize. Even though I tried to be clear that this was not the point I was making, I was still not clear enough. However, the Oregon study was yet another example of a study that failed to provide evidence for the bold claims about Medicaid.

Why Conservatives Like the Singaporean Health Care System?

Matt Yglesias asks what USA conservatives like about Singapore's health care system. He makes a few points:

"— Singapore has "multiple tiers of protection to ensure that no Singaporean is denied access to basic healthcare because of affordability issues."

— "The first tier of protection is provided by heavy Government subsidies of up to 80% of the total bill in acute public hospital wards, which all Singaporeans can access."

— "The second tier of protection is provided by Medisave, a compulsory individual medical savings account scheme ... Singaporeans and their employers contribute a part of the monthly wages into the account to save up for their future medical needs."

— As best I can tell, these Medisave accounts are deposited into the Central Provident Fund, a government-run investment pool, rather than constituting private savings as we would understand them.

— "The third level of protection is provided by MediShield, a low cost catastrophic medical insurance scheme" supplemented if like by private insurance called Integrated Shield plans and "Singaporeans must subscribe to the basic MediShield product before they can purchase the add-on private Integrated Shield Plans."

— "Finally, Medifund is a medical endowment fund set up by the Government to act as the ultimate safety net for needy Singaporean patients who cannot afford to pay their medical bills despite heavy subsidies, Medisave and MediShield.""

Many liberals have argued Singapore's system is not very market oriented. That's true when looking at the system by itself. However, comparing Singapore's system to the health care systems of other developed nations makes the system look much more market oriented relative to these other systems. 

The Singaporean system has a much higher share of its health care spending coming from private sources than other developed nations (including the USA). Indeed, it has an unusually high proportion of its spending coming from the private sector and an unusually cheap health care system. Another feature is the extent to which Singapore's system is financed by out of pocket expenditures. Out of pocket just means that the patients pay the providers directly instead of paying through a third party (like the government or a private insurance company). Over half of health care spending in Singapore is out of pocket. That compares to about 10-20% in most other developed nations (again this includes the USA).

Still, the Singaporean system does have universal coverage, lots of forced savings, lots of public hospitals, and price controls. So, it is hardly a free market paradise. Of course, neither is the USA system (even in its pre ACA form).

Debating the extent to which Singapore's system is "market oriented" seems a bit redundant to me. It is a very market oriented system in some ways (the way they pay for health care) and a fairly socialized system in others (the way they provide and regulate health care). What is clear is that the Singaporean system does not utilize third party payment systems for non emergency medical expenses to anywhere near the same extent that other health care systems do.

That is to say that they treat medical insurance like any other type of insurance. If you have a medical catastrophe, insurance is their to protect you. For routine expenses, on the other hand, individuals generally pay their own way (even if it is through a forced savings account). This gets rid of the incentive to overutilize medical coverage that exists in other nations. 

Replacing the USA system with the Singaporean system would not be practical as the Singaporean system is tailored for their particular society. That is a society which is smaller, wealthier, and healthier than us. It is also a society that has much lower taxes than us, less debt than us, and a welfare state that relies heavily on non socialized mechanisms to finance itself.

Still, Singapore, I would argue, is evidence that moving towards a system with catastrophic health insurance with individuals covering their own routine expenses (perhaps through health savings accounts) would be a smart move. We can, for example, reform public health programs to move in this direction and change tax laws to encourage more health savings accounts. 

Conservatives tend to be more favorable to these sort of proposals than liberals (who often would rather have comprehensive, universal public health care), and that is why some conservatives praise the system. It's less about the big government versus small government argument in health care, and more about the catastrophic insurance versus comprehensive coverage argument that is less publicized but very real.

By the way, here is an excellent overview of the Singaporean system.

More on the Oregon Medicaid Study

My last post responding to Raj Chetty's NYtimes column on economics as a science has received a good deal of attention (at least compared to my previous posts) largely thanks to Tyler Cowen, Scott Sumner, and Bob Murphy linking to it on their blogs. I obviously would like to extend my thanks to these economists for helping get this blog more attention.

I also noticed that the main criticism of my post in the comments on both this blog and other blogs is my handling of the Oregon Medicaid Study. A few points:

1.) The Oregon Medicaid Study did indeed show no statistically significant improvement in objective measures of physical health. I am a bit surprised that so many people have taken issue with this assertion. The common response is that there were positive results, but they just weren't positive enough to be statistically significant. In order to be statistically significant, results have to have less than a 5% chance of being statistical noise. In the Oregon study, the positive results on elevated blood pressure had a 65% chance of being statistical noise, the positive results on elevated blood sugar had a 61% chance of being statistical noise, and the positive results on high cholesterol had a 37% chance of being statistical noise. In other words, not only were the results not statistically significant. They weren't even close.

It's actually worse than that. If we just look at raw numbers and ignore statistical significance, we can also say that Medicaid increases smoking and decreases cardiovascular health of those that were already sick. The raw numbers suggest that. Of course, neither of these results are statistically significant, but they are closer than any of the other measures. There is only a 24% chance that the results showing Medicaid reducing cardiovascular health are statistical noise, and there is only an 18% change that the results showing Medicaid increasing smoking are due to statistical noise. These still are far from statistically significant, but if one is going to argue that the raw numbers that are statistically insignificant are useful in telling us the effect of Medicaid, then they should also be arguing that Medicaid increases smoking and worsens cardiovascular health of the already sick.

I would argue that we shouldn't read too much into any of the statistically insignificant results and instead focus on the statistically significant ones. By the way, that data is from Megan McArdle who had an excellent post on this very topic.


2.) I also think many people assumed that I was making an argument based on the Oregon study that I was not. I was not trying to argue that the Oregon study was an indictment of Medicaid. I was simply arguing that it was not a vindication of Medicaid. We can quibble about the results, but this study was not a big win for Medicaid. That much should be clear. Chetty seemed to imply that this study was a big win for Medicaid in its current form, and that was the claim I took issue with.

What the study did indicate was that spending money on health care for the poor helps the poor (which should be rather obvious). The statistically significant results on financial security and mental health show that having a program of this sort is worthwhile.

To put it simply, the Oregon study showed that Medicaid does a good job of protecting the poor from crushing medical expenses, but it doesn't make them healthier or save lives.

This result does, I would argue, bolster the case for reforming Medicaid. If we, for example, moved towards a system where Medicaid provided catastrophic insurance plus a subsidy that can be used to purchase supplemental private insurance or start some sort of health savings account (as some states are already doing), this program would still protect the poor from crippling medical expenses. It also would offer them more choice, improve the quality of care they receive (which is big problem with Medicaid), and help constrain costs.

Of course, there are many other proposals for reforming Medicaid (Tyler Cowen has a interesting one as well). The main takeaway from this study is that there is nothing special about the form Medicaid currently takes, and that, therefore, we should not be afraid of looking at serious reform.

Chetty's Article Falls (Uncharacteristically) Short

Raj Chetty offers an article in the NY times arguing that economics should be considered a science. Chetty points out that there are unknowns in many fields that are indisputably "scientific": 

"It is true that the answers to many “big picture” macroeconomic questions — like the causes of recessions or the determinants of growth — remain elusive. But in this respect, the challenges faced by economists are no different from those encountered in medicine and public health. Health researchers have worked for more than a century to understand the “big picture” questions of how diet and lifestyle affect health and aging, yet they still do not have a full scientific understanding of these connections. Some studies tell us to consume more coffee, wine and chocolate; others recommend the opposite. But few people would argue that medicine should not be approached as a science or that doctors should not make decisions based on the best available evidence."

Chetty basically gets it right in my opinion. Economics should be considered a science. However, Chetty's own article provides an example of why so many find it hard to view economics in this way.

Chetty brings up three examples of what a "scientific" approach to economics has supposedly taught us:

"Consider the politically charged question of whether extending unemployment benefits increases unemployment rates by reducing workers’ incentives to return to work. Nearly a dozen economic studies have analyzed this question by comparing unemployment rates in states that have extended unemployment benefits with those in states that do not. These studies approximate medical experiments in which some groups receive a treatment — in this case, exten ded unemployment benefits — while “control” groups don’t.

These studies have uniformly found that a 10-week extension in unemployment benefits raises the average amount of time people spend out of work by at most one week. This simple, unassailable finding implies that policy makers can extend unemployment benefits to provide assistance to those out of work without substantially increasing unemployment rates.

Other economic studies have taken advantage of the constraints inherent in a particular policy to obtain scientific evidence. An excellent recent example concerned health insurance in Oregon. In 2008, the state of Oregon decided to expand its state health insurance program to cover additional low-income individuals, but it had funding to cover only a small fraction of the eligible families. In collaboration with economics researchers, the state designed a lottery procedure by which individuals who received the insurance could be compared with those who did not, creating in effect a first-rate randomized experiment.

The study found that getting insurance coverage increased the use of health care, reduced financial strain and improved well-being — results that now provide invaluable guidance in understanding what we should expect from the Affordable Care Act.

Even when such experiments are unfeasible, there are ways to use “big data” to help answer policy questions. In a study that I conducted with two colleagues, we analyzed the impacts of high-quality elementary school teachers on their students’ outcomes as adults. You might think that it would be nearly impossible to isolate the causal effect of a third-grade teacher while accounting for all the other factors that affect a child’s life outcomes. Yet we were able to develop methods to identify the causal effect of teachers by comparing students in consecutive cohorts within a school. Suppose, for example, that an excellent teacher taught third grade in a given school in 1995 but then went on maternity leave in 1996. Since the teacher’s maternity leave is essentially a random event, by comparing the outcomes of students who happened to reach third grade in 1995 versus 1996, we are able to isolate the causal effect of teacher quality on students’ outcomes.

Using a data set with anonymous records on 2.5 million students, we found that high-quality teachers significantly improved their students’ performance on standardized tests and, more important, increased their earnings and college attendance rates, and reduced their risk of teenage pregnancy. These findings — which have since been replicated in other school districts — provide policy makers with guidance on how to measure and improve teacher quality."

Chetty's presentation of these findings is a perfect example of why it is hard to see economics as a traditional science. Take his analysis of the literature on unemployment benefits. As far as I can tell, his first claim, that a 10 week extension only modestly raises the time workers choose to stay out of work, is correct.

However, let us look at a very new NBER paper on the effect of unemployment benefits that also uses what Chetty would identify as a scientific approach. Here is the abstract:

"We exploit a policy discontinuity at U.S. state borders to identify the effects of unemployment insurance policies on unemployment. Our estimates imply that most of the persistent increase in unemployment during the Great Recession can be accounted for by the unprecedented extensions of unemployment benefit eligibility. In contrast to the existing recent literature that mainly focused on estimating the effects of benefit duration on job search and acceptance strategies of the unemployed – the micro effect – we focus on measuring the general equilibrium macro effect that operates primarily through the response of job creation to unemployment benefit extensions. We find that it is the latter effect that is very important quantitatively." (emphasis added)

This is very important because this paper actually does not dispute the empirical claim that Chetty made, but it does dispute the policy conclusion he came to based on that empirical claim. The fact that unemployment benefits only modestly increases the time workers want to stay out of the market does not automatically imply that dramatic increases in the duration of unemployment benefits will not have significant negative effects on employment. Indeed, as this paper shows using just the kind of method Chetty recommends, looking at other channels through which unemployment benefits can affect employment yields a much less positive result for advocates of expanding these benefits.

With regards to Medicaid, Chetty also paints a surprisingly incomplete picture of the Oregon Medicaid experiment. As you will recall, Chetty is correct in pointing out that expanding Medicaid seems to have increased usage of health care, decreased financial strain, improved mental health, and improved self reported well being, but he, quite surprisingly given the caliber economist Chetty is, leaves out the less flattering (for supporters of the ACA) part of the study that found no statistically significant increase in objective measures of physical health for patients who received Medicaid.

At best, the Medicaid study was a mixed result for supporters of expanding the Medicaid program (which the ACA does quite dramatically). At worst, the study is a sad demonstration of how bad Medicaid (and perhaps insurance in general) is at improving objective physical health. Why Chetty presented this study as an unambiguous victory for the pro Medicaid crowd is a mystery to me (although I suspect support of ACA has something to do with it)?

Of course, the benefits that Medicaid did bring were quite large, but one would imagine a smaller Medicaid program that relied more on cash transfers would still have those benefits while perhaps leading to more efficient use of health resources, but that is a topic that deserves it's own post.

As for the study on the benefits of good teachers, I have not looked into this particular issue and thus have no comment on this.

My goal here is not to "take down" Chetty's arguments. Instead, I would like to offer an explanation for why economics is not often thought of as a traditional science: it is too political. Chetty is among the most respected economists in the world (as he should be). If any economist could be politically objective, Chetty would fit the bill, but we still see political bias seeping through in this article.

Economics is a science, but it is a very politicized science. The Medicaid study, with its ambiguous results, offered justification for the policy proposals of both supporters and opponents of ACA, for example. Both sides were offering an incomplete picture of the study in this debate, but both sides were also correct the claims they made even if they strategically left out inconvenient findings.

Economics certainly can be seen as a science when it comes to making observations about the world, but when it comes to recommending certain policies, economics is only as scientific as the biases of the economists allow it to be.

 

Public Health Coverage and Private Health Coverage

First, a study linked to by Tyler Cowen. From the abstract:

"We found that privately insured patients had lower risk-adjusted mortality rates than did Medicare enrollees for twelve out of fifteen quality measures examined. To a lesser extent, privately insured patients also had lower risk-adjusted mortality rates than those in other payer groups. Medicare patients appeared particularly vulnerable to receiving inferior care."

Yes, that says Medicare not Medicaid. The evidence that Medicaid patients have less access to care and lower quality of care is so overwhelming that even opponents of Medicaid reform (and advocates of Medicaid expansion in its current form) admit that Medicaid doesn't offer nearly as good care as private coverage. More on Medicaid from Avik Roy if you are interested.

The important finding here is that even Medicare, which does not have the same reputation as Medicaid with regards to quality, offers significantly lower quality coverage than private insurance. This is likely because of lower provider reimbursements as well as the additional paperwork required to get paid by Medicare (not that private insurance companies are easy to deal with either).

Second, despite the fact that everyone is entitled to "free" (tax financed) health care in Ireland, almost half still pay for private health insurance. This is, by the way, also true of China where many people still find it necessary to pay for private coverage despite having already paid for public coverage through taxes. I visited China some time ago. I asked a college age Chinese girl why the Chinese do this and she said (in broken English) that the public health coverage offered such long waiting times and such low quality that anyone who could afford not to use it would pay the extra to get decent health care. 

Finally, Canada is moving towards private medicine. Sweden is doing the same. Both are doing so out of necessity. Public health care simply doesn't offer the quality health care that patients in 21st century developed nations demand.

The main point here is that private medicine offers substantially better quality than public or government medicine. We see this when comparing public systems and private systems within the USA. We see this when we look at nations with very governmental systems that have patients that still pay additionally for private insurance despite having already paid for public coverage through taxes. We also see this in nations where the government is trying to beat the consumers to private medicine by moving towards health privatization.

In nations with public health care systems, private health care has been quite taboo, but that is changing as citizens and governments move towards it. People demand high quality health care, and government simply can't deliver. The private sector, quite simply, can.

This is not meant as a criticism of public health care, but one cannot escape the conclusion that, given all of these realities, public health coverage is woefully inadequate in providing for a 21st centur population that is wealthier and more demanding than ever.

Despite claims that moving towards single payer or government run health care is inevitable or necessary, the experience from both within our nation and from other nations may suggest that real progress means more private medicine, not less.