Your summary is correct. We found two gaps in the oversight. First, Transport Canada didn't assess the effectiveness of the safety management systems. Second, it didn't assess whether all of its additional oversight activities had improved overall safety.
Why is this important? It's a matter of common sense. When you spend time and human resources on something, you want to know if that investment is working. We also want to know whether we're inspecting and checking the right things, verifying compliance with the appropriate requirements or whether we should adjust the approach.
Self-assessment is therefore always very important, but the effectiveness of all monitoring activities must first be assessed before determining if corrective action is required.