I do agree, and I would say the reason I agree is that there we're talking about what people label time, place and manner restrictions that don't talk about the content of the speech. Therefore, it's far more likely to say that it's legitimate restriction, because we're not making this discrimination on the kind of speech that is involved.
I think what you're pointing to is the kind of thing that we should be crafting rather than talking about the particular types of speech or the intentions behind speech.