How many leave the Big Dog (D700/D800) at home while traveling and take a smaller camera instead?

I had the RX100 and quickly sold it.  Not much reach and slow on the long end of the zoom.

Too often I go hiking, traveling, or even to the ballpark where a larger camera becomes a liability.  At night carrying a $3000+ camera/lens on a dim street seems like a bad idea.  Same with climbing a steep rocky trail.

Currently I slip an E-PM2 in one pocket and a 40-150mm lens in another.  I may even take one of the smaller Nikon D lens too...they work perfectly together.