I agree with the first part of your remarks, but in the end it isn't true that the decision can be unfair or arbitrary. The decision needs to be reasoned, and if it isn't, there will be a judicial review and the courts can reverse the decision of the Governor in Council.
On March 6th, 2008. See this statement in context.