It all depends on how much effort it takes the predators to prey upon their favourite food items.
For many fish in the ocean, you are absolutely right. There will come a point when the search effort, the handling effort.... What's the point? For seals, there are salmon. Fine, it is a nice, fatty fish, but there are not that many salmon, compared to herring, mackerel, and many other species they can exploit.
The difference with Atlantic salmon and Pacific salmon is that at a very predictable time of year, you have a lot of bite-sized fish coming out of a source point. Marine mammals are not foolish, so it doesn't take that many marine mammals to have a demonstrable impact on the mortality rate, because they don't have to expend that much effort in order to obtain the food.
I would say that in general you are absolutely right, but salmon, with their migratory patterns, are perhaps a special case in this regard.