I think that the reasons Roy and Erik gave you are the good ones.
As an example the lens next to the 17mm, is the smallest one could adapt to M4/3, the Industar 69, a Tessar triplet from the Russian Chaika half frame camera. Of course it has no motor, no AF.
The Tessar triplet Could be the simplest devised: 4 lenses in 3 groups. Works with the M39 mount, which is smaller than m4/3.
However I also have the old 9-18 which has a huge front element, or an old 135/2.8 weighing 500 gr. The mount can still take it but I am carefult to hold the kit by the lens. Some have mounted bigger lenses, but one must be careful of the structural integrity of the mount, possibly with SHG zuiko lenses. Larger diameter helps to better distribute the weight.
So it's a matter of compromises. Some lenses can be small and still have big front elements, like some Leicas and CV, but I suppose that the exit lens will still be relatively small since it is so close to the film/sensor plane.
RF mounts like Leica, or half frame cameras' ones were smaller, but they accomodated only a limited range of lenses, not at the extremes of the focal range like a SLR had to do. It is interesting to see that the m4/3 mount is something in between the two.
--
Photostream:
http://www.flickr.com/photos/amalric