It depends a little bit on why interest rates have to go up. If the economy is stronger, there's more revenue, people are spending more and that's building inflationary pressures, and the government will be getting more revenue, yes, it will also have higher interest rate costs.
The more worrisome scenario is one where the economy is weaker but inflation is stronger. Then it's tougher on Canadians and it's tougher on governments.