Yes, and you're right; I think most Prime Ministers sitting for a reasonable period of time in New Zealand would have been thrown out of the chamber at least once.
In fact, the current Prime Minister, since he's been Prime Minister, which has now been eight years, hasn't been made to leave the chamber. Before that, when he was an opposition member, he did go out a few times. In this instance, he was required to leave because Standing Orders say that when the Speaker is on his feet and calling for order, members must sit down and be quiet. The day before that, the Speaker had warned him about the same thing. He tends to get up, turn his back to the Speaker and then address the rest of the chamber. Sometimes when the Speaker is on his feet, the Prime Minister says that he can't see him. He did that three days in a row. He had been warned the day before, and the Speaker did that for the same reason he would ask any other member to leave, which was just defying the Chair.