I'll start, but Gordon is really the expert.
If it is a designated national historic site within Parks Canada, there is legislation through the Parks Canada Agency Act and through policy to protect it. If it's a designated national historic site within the federal family, not under the custodianship of Parks Canada, such as the Central Experimental Farm, that's another department and the other department has full authority, as Gordon explained, so the minister has no real authority on that. If it's a federal heritage building, it's under Treasury Board policy. The ultimate penalty for doing whatever you want with a classified federal heritage building is a rap on the knuckles by the secretary of the Treasury Board. That's it.