Will A mount lens work well without OSS?

I'm considering purchasing EA2 with some A mount lens because the e mount ones are just equally expensive even including the adapter. So I'm just wondering if all the lens don't have OSS isn't that a huge downgrade from the speed point of view? I'm looking at losing 2-3 stops for each lens I purchase?

So for example, if I were to purchase a really awesome A mount 50mm1.4 Zeiss it would technically be way slower than the SEL50F18? Which is counter productive?