If they were privately owned, the expenses would come from the owners / investors, not the tax payers - at least not in a capitalist republic. Of course politicians' cronies often get favors...
I agree that the prison system is on a cusp of becoming the biggest money maker for all venders involved. I even pondered about what supplies my company could provide. I feel apprehensive about participating in an industry that is not for the good of society.