That is correct, and it's consistent with the basic principles that money borrowed for the purposes of earning income is deductible. Interest on money borrowed for the purpose of earning income is deductible and other types of interest are not.
For example, if you had an individual who earned interest from a deposit in a deposit-taking institution and at the same time paid interest on, say, a car loan for a personal car, the interest on the car loan would not be deductible, but the interest on the moneys placed with the deposit-taking institution would be taxable. The same principles apply in the case of interest paid and received from the Canada Revenue Agency.