If you haven't given your consent, then it is contravening the rules. A member cannot interrupt and ask questions of the person who has the floor. However, I'll just say that in my position as chair, when there are points of order, points of clarification, or relevancy orders being made, I sometimes have to give them a minute and hear them out in order to see what the member is saying. That sometimes is enough time for the member to put that question out there. Of course, though, you do not by any means have to respond.
That was, I guess, not a point of order. That was not in order. I hope that resolves that and gives you some clarification.