"Compact" and "fast" are mutually exclusive.
Is it though?
Yes.
The norm is as follows:
f/2.8 is considered fast for zooms. Even so, such zooms are not compact, even for APSC, where filter sizes are around 77mm for wide zooms like the 16-55. For ultra wide zooms, take a look at the 8-16mm from Fuji as an example. For FF, wide angle zooms normally have a filter of 82mm. Only in m43 systems one can consider f/2.8 zooms as truly compact, but that is because he sensor size is much smaller. Going back to FF, Canon has a 28-70mm f/2.0 zoom lens, look at the size of it. And that is a standard lens, not wide angle zoom.
For primes, f/2.8 is not considered as fast. For that, you need to go to f/2.0 and faster.
You wrote in another reply that f/2.8 is fast enough for you; if that is the case, you should consider the new Fuji 8mm prime lens; after all, the difference between f/2.8 and f/3.5 is negligible.