I am wholly in favour of that idea. I would even recommend a sector-based and region-based work permit. It's clear from closed work permits that it would be very beneficial if workers could change jobs within a given sector or a predetermined occupation. The current system sorely lacks flexibility, and in most cases, employers suffer just as much as the workers do.
Businesses often call for a system like that, particularly because it would help them manage peak production capacity. Workers need to earn enough income to live adequately, so they often have to work overtime. They are authorized to work for only one employer, which can be a barrier for them. For instance, instead of having to put in 200 hours a week, they could work for two employers, as per a specified agreement. That would be one way of giving employers and employees more flexibility.
As you suggested, employees would no longer be tied to a single employer; rather, they would be tied to a sector. The issue of who would cover the employee's travel costs would have to be dealt with, among other things, but that could be worked out among the employers, perhaps in consultation with the worker.