The key problem here is not so much where goods are being shipped, because usually sales taxes are applied there. It's where there are services or things such as software, movie rentals, e-books, and so on that the biggest unfair competition takes place. Those services—advertising is the other huge one—are not being taxed, and that provides unfair competition.
You're quite correct that part of the whole thing that's going on here is that Netflix, Google, Amazon, and Facebook are setting up many parts of their businesses in tax havens, thereby avoiding even U.S. taxes, so it's even worse in terms of unfair competition, because they're not paying anything at all in many cases.