I have owned the E1, E3 and E5. Each was a progression. Importantly, as I look back at the pictures I have shot in the last 10 years, I am even happier with the upgrades. The E5 simply works better at taking difficult pictures. I don't have to fight to get rid of blown highligts so much and the focusing is certainly good enough for what I shoot.
All that said, the lenes are great. I still have my original 14-54, but the 12-60 is better.
I also have hte 50-200 and the 50. This is all I need for excellent photographs.