No, you don't. That's the whole beauty of it.
The votes are fixed at the time of the general election. On going across the aisle, you're still representing your same riding and you're still fixed with your same vote, so you get no advantage from crossing the aisle.