If you're spending 10ยข out of every tax dollar that you collect on servicing past debts, that obviously leaves you with less to spend on providing current goods and services.
The problem that we collectively face now and going forward is that we're going to have growth of the economy or growth of the base revenues of government that will be equal to or perhaps even less than the interest rates. The burden of past debt will increase year after year rather than being eroded.