You're very right. We will hear some say that it's not a matter that they have contracts that are too rich in the public sector; it's that they're not good enough in the private sector. But go and tell your boss, if you're in the private sector, that you want a better deal.
You've put your finger on the precise problem. The boss in the private sector has competition. It may be foreign competition. It may be competition down the street. Nevertheless, they're constrained by that competition. The public sector unions are dealing with employers that are monopolists, mostly, and do not have customers but taxpayers funding the revenue they collect.