Ultimately, people vote with their feet, and if they don't feel valued within the society where they live, intelligent people will find a way to use whatever skills they have. I'm probably less concerned, frankly, about a brain drain from places like India and China, which are the two dominant sources of immigrants now, because they are making massive investments in education in growing engineers and skilled people in numbers that far exceed what we could ever use in Canada.
Your fundamental point is absolutely right. You have to worry, as a matter of fundamental social policy, about stripping the best and brightest out of other countries. But often they have a willingness simply because they cannot find a way to use their talent at home.