That's correct. That's primarily the reason Canada Post was in fact able to exploit these workers for so many years, because that was their response: “If you don't like it, I have plenty of people in this rural community who are willing to take your job”.
That varies based on the region of the country you live in. In Alberta these days, Canada Post can't get people to work at all, because there are better paying jobs in the oil industry, for example. But you're absolutely right, in many communities where you have high unemployment rates, people will work for a lot less than what they'll work for in urban centres.