If I may, I'd like to be more specific. I didn't want to refer to the province of Quebec as a bilingual region. I was talking about bilingual regions like northern Ontario, eastern Ontario, the Montreal metropolitan area and most of the Eastern Townships.
Apart from those regions in Quebec, and I should point out in passing that Quebec City is a unilingual French region, the language of work in federal institutions apart from the bilingual regions of Quebec is always French.
For example, you do not have a right to be supervised in English or to write your briefing notes in English in Quebec City, because the language of work in federal institutions in Quebec City is French, just as it is English in places like Toronto and Vancouver.