Whatever happened to the "size" factor?

To preface my remarks, please remember that this (like any) thread is meant for discussion. Love to see varying viewpoints - no insults needed, or threads ending with "Your Momma" would obviously be way out of line here. 

With that being said...

I first jumped into the world of mirrorless cameras when Micro Four Thirds started to gain a foothold - specifically with the likes of the Panasonic GF1 (which I still use), and the Panasonic GH1.  I was intrigued to say the least and the promises of having smaller body styles that translated into a lighter form/weight factors was enticing to say the least. As more and more models were released, I became hooked - no turning back now with mirror-less.  Even the older regular Four Thirds cameras that I was previously using were huge and heavy weight compared with this new mirror-less technology.  Since that time however, the cameras have gotten larger (i.e. Olympus E-M1 series and the Panasonic GH series).

Then I subsequently added Fuji with the X-E1. Granted, the Fuji cameras were a tad larger, but their form factors and IQ levels were, in my book, to die for and I soon became a Fuji X aficionado also.  But since that time they've obviously (gradually) gotten larger with the likes of the X-T1/2 series with battery packs, and their X-Pro models, etc.  Now enter the X-H1 and you have a camera the size of a DSLR - damnnnnn, now go figure?  What happened to one of the largest draws towards mirrorless - the size factor?

Well, then I added the Samsung NX series with the NX300.  I immediately became hooked on that line, subsequently adding the NX30 and NX500.  All of these cameras had a superb form factor, excellent IQ characteristics, and they were extremely light for the genre camera they were part of.  Samsung really seemed far ahead of its tme.  I never had the bucks to get the NX1 - and that camera, along with battery pack, and S lenses were as large as DSLR's - again, what happened to the size factor?

Then I accidentally came upon the EOS M series of cameras.  What a delight they were (still are).  Interestingly enough, even though they have an APS-C sized sensor, their form factors are very small.  It was a great feeling - at least for me.  In fact, every time I look at the likes of my EOS M10 or EOS M100, I sort of shake my head in amazement as I wonder how they could fit DSLR-sized electronics and sensor inside such a small form factor.  Kudo's to Canon on the M series.  Now if only they had added a dozen or so lenses since the M was first released - damnnnnnnn. 

But now - with the release of the Nikon Z6/7 and the pending arrival of the Canon R (and even Panasonic waiting in the wings with their FF model), the sizes all appear to be almost as large as a DSLR again.  Sure, they are mirrorless designs, but when you add those huge lenses - well, we're back to being large again, not to mention weighing more.

So I ask - with all of the technological advances we've made over the past decade - astonishing really - why couldn't maintaining a smaller size factor be a major factor in their camera design planning?  I realize that FF sensors will require a somewhat larger form factor - but what I'm seeing here is that we've witnessed the yo-yo effect (up and down) with regards to camera size factors in mirrorless designs over the past decade.

OK - I'm done now...