On the whole, federal workers are overpaid. A common response to this is that the only reason that federal workers are paid more, on average, is because they are better educated and thus higher pay is justified. That is feasible, but it isn't true. Even after taking such differences into account, a significant compensation gap remains (here).
However, there could be a larger problem in public sector compensation. Not only is the public sector, as a whole, overpaid. It is also too compressed. Bryan Caplan reports on this dynamic here. A chart is provided as well:
Overall, the public sector is overcompensated by 16% relative to the private sector. However, this gap varies widely by education level. For workers with high school diplomas or less, the federal government pays an incredible 36% more. However, for workers with professional degrees or doctorates get paid 18% less in the federal government than the private sector.
To put it simply, a worker with a professional degree or doctorate can expect to get paid about 87% more than a worker with a high school diploma or less in the federal government compared to 210% more in the private sector.
There are two key forces at work here: public sector unions and public opinion. Public sector unions create a huge pressure to overpay low skilled workers. These would be workers with some college or no college. These workers are paid 30 to 40% more in the federal government than the private sector.
Then there's medium skilled workers who tend to have a college degree. These workers are also overpaid in the federal government, but to a lesser degree. Something more to the tune of 5 to 15%. Here there is some pressure from public sector unions as well.
At the top, workers with professional degrees get paid about 20% less in the federal government than the private sector. This is where public opinion plays a big role. I don't know this for fact, but I would think that the private sector tends to pay much better for upper management. The higher the level of management, the better the private sector pays both overall and relative to the public sector.
Case in point is the president of the united states. The pay is "only" $400,000. That may sound like a lot, and it is a lot. But, for the most powerful position in the entire world, it is actually astonishingly low. CEOs of any major company are paid millions, if not tens of millions, every year. By comparison, the commander and chief is very underpaid.
In the private sector, a job like that would pay an incredible amount. Luckily, most people who run for president do it for public recognition, prestige, and (hopefully) to improve the world. However, any large corporation has an entire group of upper management who are paid well above six figures. Indeed, this is necessary for the success of any large organization. Management cannot be undervalued. Most people with that sort of high level managerial experience go to the private sector rather than the federal government simply because of the pay differential as well as mobility.
This is, by the way, why you should ignore people who argue for single payer health care because "administrative costs are lower". Low administrative costs aren't necessarily a good thing. This is especially true if they represent underinvestment in management, which is the case with our federal government and most nationalized health care systems.
How does public opinion lead to underinvestment in management in the public sector?
It's simply because the public would go nuts if there were a large group of federal government employees making six figure incomes. The public won't be too upset if someone who's market wage would be something like $20,000 gets paid $25,000, but there would be outrage if there were a group of people getting paid $200,000 or more in the federal government even if they could command a higher wage in the private sector. The medium skilled workers have both of these pressures on them (public unions and public opinion) which is why they have a smaller (but still significant) wage premium in the federal government.
One could argue that it is the private sector underpays low and medium skilled workers and overpays highly skilled management. Indeed, this is basically the argument that advocates of caps on CEO pay and large hikes in the minimum wage make. I just don't find this very plausible. Individual private sector companies may overpay upper management, but it is hard to imagine that the entire private sector is radically overpaying upper management. It is much more plausible that upper management really is a scarce skill that commands a lot of pay because it is so difficult and important.
Indeed, some evidence suggests that CEOs are underpaid. They get paid a lot, but they may add even more to their firms in value. Even if CEOs are overpaid, pay caps are a bad idea for a lot of traditional economic reasons and practical policy reasons, but that is for a different post.
The point here is that management is really important and the public sector really underpays management relative to other public workers and the private sector. This is one reason that contracting out is necessary for so many projects: the private sector has the managerial talent necessary to run big operations that the public sector just doesn't have.
It is also yet another reason that nationalizing industries like health care or banking are bad ideas. These industries require a lot of managerial talent to run and the public sector can't deliver here.