m43 vs. "big" aperture P&S's

Like probably many others, I'm weighing the small-bodied m43 guys (PEN's & GF's) against the bright-lensed P&S's (Canon S95, Oly XZ1) for a carry-anywhere camera.

I've read posts here claiming that big-apterure P&S's are better in low-light than m43's with standard-aperture (f/3.5-5.6) kit lenses. Which leaves me wondering if that's really true.

The m43 sensor is what, like 5x larger than the largest P&S sensors? If you compare m43 shot at, say f/4.0 (or even f/5.6) vs. a P&S at f/2.0, the P&S is working at a faster F-stop but wouldn't the m43 lens still have the larger aperture (in absolute terms) due to being scaled to a larger sensor, and thus be letting more total light?

I don't pretend to know completely what Im talking about, but it just seems to that the much larger sensor, even with a slower lens, would be off??