I have. I went from a Nikon D300 to the EP1. I had enough of the bulk. What's a DSLR good for if you're thinking twice about carrying it around (in my case, anyway; some people don't mind the bulk and backpack-full of gear). In the end, it's the picture that counts. And for a picture to be taken, one must, first and foremost, have a camera at hand. Since I got the EP1 I'm carrying with me everywhere I go. Of course, this meant that I had to re-evaluate my shooting habits: do I need 6fps? Rarely, in my case. High-Iso? I don't really like digital noise, so I rarely take high Iso pictures. Additionally, the EP1 gives me surprisingly good results at 3200 Iso, maybe not as good as the D300, but good enough, given the trade-off in size.
In short, I am not giving-up DSLR features that are crucial to me. What the EP1 gives me is good enough for my needs, actually, it exceeds my needs, in terms of colours reproduced, style, and the use of great (and cheap) legacy lenses. Come think of it, the exorbitant price of current pro Nikkor lenses, such as the 70-200 f2.8 Nikkor, played also a significant role in my switching. I can get great results by using some really nice legacy lenses on my EP1, for a fraction of the Nikkor lenses's prices. Lenses such as Canon 55mm f1.2, or Zuiko 50mm f1.4, Pentax Super Takumar, and so on.
No regrets whatsoever.