In my opinion, when an official is appointed to serve during good behaviour or during pleasure, an effort has already been made to find the best candidate for the job. These people appear before selection committees. The unsuitable ones are weeded out. That means that there must be a very good reason to remove someone from office, unless it is a matter of conflicting ideologies, or unless the mandate was misunderstood from the outset.
In that case, removing an appointee would be a very serious matter. If we parliamentarians have no power and if all of this authority rests with the governor in council, then a removal from office could well be arbitrary and unfair.