This is basically the purpose of asking for a deposit or advance rent. You basically charge them for cleaning up the place and clearing of the bugs that they had brought in. You have to make sure though that you are charging fair for the service. You would not want to deduct from the deposit more than what you actually spend for for the service.
Kick them out. This is why you prepare in advance for this situation. Sometimes you have to be willing to just cut your losses and accept that it will be a learning lesson.
This is such a tough question because if they brought bedbugs along, it would take a lot of effort to get rid of them. The only option is to talk to the tenant so that you can both come up with a proper solution to eliminating the problem. Another problem is cockroaches that can take over a household.
I completely agree with kicking them out, I'll just add that if you collect a month's rent in advance you will always have some money to pay for repairs when you kick them out.
Sure that he has to be kicked out or support the expenses for the cleaning, I mean, it's general law or it has to be in the contract that the place needs to be in habitable condition.