That would be true only if it were effective. We have studies showing that the N95 masks do not reduce the risk of infection to health care workers working with influenza. Whether they use a surgical mask or an N95 mask, the risk is exactly the same. The precautionary principle would apply only if it actually made a difference, but in this case, it doesn't.
You can always find disagreement among experts. But the vast majority of experts, those dealing with infectious diseases on a regular basis, would say that (a) it makes no sense, (b) it's difficult, and (c) it may actually increase the risk if it gives a false sense of security.