You are right: large apertures have many advantages other than brighter viewfinder.
First, there is certain non-binding law that 2.8 aperture in zooms means the very highest overall quality. Companies could, in theory, build poor 2.8 zooms and excellent 3.5 zooms, but they don't.
24-105/4 and 70-200/4 are rare L-level exceptions, but even they AFAIK don't have the same feature as the 2.8 versions: almost full IQ at the largest aperture. 70-200/2.8 is practically perfect at 2.8, but most other zooms have the first high quality f-stop at about 5.6 or so.
Second, not every f-stop is created equal. You could say that 2.8 and 1/60 is the same as f/4 and 1/30. In theory, yes, but in practice, no.
There is an old joke from the film era that f/11 and 1/125 is ALWAYS the right exposure. The meaning of the joke is that when shooting in sunlight with 100 ISO film, this really comes close to being true.
In similar manner, many situations in real life photography dictate certain steps or barriers that your tools must achieve. 2.8 aperture is one of those. Very often you really, really need the f-stop to be 2.8 and you cannot substitute it with f/4 and longer time.
This comes from a complicated combination of the typical interior light, typical DOF needs and typical shutter time needs. 2.8 is actually on the very edge of usability, so f/4 will not cut it at all.
This is why professionals seldom use zooms anyway when shooting indoors. They pay top dollar to get stuff like 85/1.2, so they can achieve the often necessary combination of totally-sharp-but-selective-DOF f/2, facial-movement-stopping 1/60 and technically-good-enough 800 ISO. There is no way that you could do the job with theoretically "similar" f/5.6, 1/30 and 1600 ISO.
2.8 zoom is also crucial in indoor sports. It just happens to be so that most arenas have "always-right" exposure of 2.8, 500, 1600 ISO. You cannot use f/4 and 250, because 250 doesn't stop a moving human being. 3200 ISO is (or has been) out of bounds for high quality pics, and f/4 also gives you so deep DOF that the background becomes too sharp and distracting.
Regards,
Ravalls