It's a tricky situation, especially because, as Matt said, the Olympics are a huge draw. People get to know a couple of names, so they'll pick a country. If the Canadians have a relay going in, the relays are really exciting.
Swim meets usually end off with a 50-metre freestyle, a really exciting race, but the reality is that the main races in the sport of swimming are a 200-metre individual medley or a 400-metre IM. You're looking at 1,500-metre freestyle, which really isn't that exciting to watch.
Not to be pessimistic about the sport itself, but to answer your question of where we start, it's in elementary schools. It's implementing swimming and all sports into a physical education curriculum.
If we want to mimic what they have in the United States, at a university swim competition the building is crowded and overflowing with people cheering on the team. We see that a little bit starting in CEGEP. If we can bring that to high school and bring a culture into our high schools, that would help. If every high school or one in the area had a pool, that would be a huge step in the right direction towards developing kids who want to watch swimming.