Legally speaking, you're right: the Governor General has full power and discretion as far as that is concerned but one must add that this power is framed by conventions that are not legal but rather political.
On September 26th, 2006. See this statement in context.