It really depends on how you use the SP-590UZ, what your budget is, etc. Right now, there is not really a replacement in the Olympus line up that will perfectly match your SP-590UZ. So, you have to judge what features are must haves and what features you can live without.
Now, the SP-590UZ has an 35mm film camera equivalent focal range of 26-676mm. If you take a lot of pictures at the long end of the zoom range, then you probably want one of the long zoom cameras (SH-50 for $250US, SZ-16 for $230, SP-820UZ for $280, SP-100 for $400).
If you find yourself need more range than you already have, then you should look at the SP-100, which will give you 24-1200mm equivalent zoom. It isn't available yet, but it should be next month.
If you want that range in an interchangeable lens camera (ILC) like the OM-D or Pen, you will need at least two lenses, the 12-50mm (24-100mm equivalent), and the 75-300mm (150-600mm equivalent). The E-M10 + 12-50mm + 75-300mm will set you back $1,650 (US) or so. Now, if you are willing to give up a little bit of wide angle, you would switch the 14-42mm (28-84mm equivalent) kit lens for the 12-50mm, and it would drop the price to $1,350. Bear in mind that you will have to switch lenses to cover the field of view, and that there is a hole in the middle of the range (100-150mm or 84-150mm equivalent) that you would need to shoot with the wide angle lens, and crop your image to achieve the framing you want. If that is a problem, you would need to switch to the 14-150mm (28-300mm equivalent) for your wide angle lens for $1,750, or add the 40-150mm lens (80-300mm equivalent) to your package for $1,500.
Be sure to look at how big and heavy the entire package is. Also note that you will need to shoot two handed (left hand zooms the lens, right hand does the shutter). I find at times, I prefer to shoot one handed, and using the super-zoom cameras with the zoom lever on the shutter button is very handy.
If you are willing to have only 1/2 the telephoto range and a little less wide angle (28-300mm equivalent), then the Stylus1 becomes a candidate as well ($700). Likewise, in that range, you can drop the $550 75-300mm lens from the ILC cameras.
If price is an issue, Olympus is having a fire sale on the E-PM2 + 14-42mm + 40-150mm lens for $300. The E-PM2 does not have an electronic viewfinder as part of the camera, but you can add one for $145 for the cheap one, and $280 for the better one. I have the E-PM2 camera, and I really like the images. I recently upgraded to the E-M5 in order to get weather sealing and the electronic viewfinder. The E-PM2 is the low end Pen, which means it doesn't have as many buttons/dials for direct shooting, and the LCD doesn't tilt.
Next up is the size of the camera. I don't have direct experience with the SP-590UZ, but I had a SP-550UZ for a while. The SH-50, SZ-16 are smaller, the SP-820UZ should be about the same size, the Stylus1 is probably wider but not as deep as the SP-590UZ when folded up, and the SP-100 and ILC cameras will be bigger (particularly when you consider bringing multiple lenses). The E-PM2 + 14-42mm lens is perhaps the same size as the SP-590UZ, providing you remove the flash and/or electronic viewfinder. The extra lenses though might be a problem in terms of size.
The SP-590UZ took 4 AA batteries. Except for the SP-820UZ, all of the cameras take propriety batteries, so you might want to figure in the cost of getting extra batteries to the total price with the other cameras. I tend to buy third party propriety batteries and chargers from bluenook.com.
The SP-590UZ has an electronic viewfinder (EVF), which is very helpful in shooting in bright sun. For some people, it isn't an issue whether the camera has an EVF or not, for others it is very important. The SH-50, SZ-16, and SP-820UZ do not have an option for an electronic viewfinder, so on these cameras you would need to shoot with the LCD only. You can get hoods to block out the sun for the cameras (look at clearviewer.com). The SP-100, Stylus1, and E-M10 all have builtin EVF's. The E-PM2 can take an optional EVF, but you will not be able to use the flash when using the EVF. I also find I cannot use the VF-2 on my E-PM2 when I use polarized sunglasses, and wanting the EVF was a big reason I went for the Stylus1/E-M5.
The SP-590UZ used xD memory cards which are now obsolete (and good riddance to them). All of the current cameras take SD, SD-HC, and SD-XC memory cards. This means you will have to buy at least least one memory card. FWIW, I just got some SanDisk 16GB SDHC Extreme Class 10 UHS-1 cards for under $20. When getting memory cards, be sure to get at least class 10 cards if you plan to shoot video. If you have an older SD card reader, you probably will need to replace it, as the modern cards need newer readers.
I believe the SH-50 and SZ-16 will charge batteries via the USB cable (I'm not sure about this). If you want to charge batteries outside of the camera, you can get 3rd party chargers (and extra batteries).
The SP-590UZ had a f/2.8-5.0 lens. With the exception of the Stylus1 and SP-100, all of the cameras mentioned have slower lenses. However, in the 4-5 years since the SP-590UZ was designed, the sensor technology has improved, and you can shoot at higher ISO's, so it likely will be a wash. The E-PM2 and E-M10 will have much better image quality, particularly at the higher ISOs, but it is a matter of do you want to carry the bigger lenses, etc. in order to get the bettery quality of image that the larger sensor affords. Everything is a tradeoff, and it is a matter of what you want. I go back and forth whether I want to carry all of the gear to get that extra quality, or whether I want a small package that fits in a jacket pocket.
In terms of accessories, the Stylus1, E-PM2, and E-M10 will support external flashes and wired shutter releases. The Stylus1 and E-M10 will also support connecting to the camera from a smartphone via wifi.
I do not tend to shoot movies with my Olympus cameras. If movies are important to you, then Henry is the person to talk to, since for him movies are very important.
So, if I were in your shoes, the SP-820UZ is probably the closest to a direct replacement for your SP-590UZ. The only thing is the lack of the EVF, and it might make sense to get something like a clearviewer. I have the clearviewer, and it does what it claims, but I find it a little fiddly for using it as a vacation camera, and I haven't used it in awhile. The SH-50 is also a good camera -- it is a little smaller and maybe it doesn't have all of the controls the more expensive cameras have, but maybe you don't use a lot of those extra bells and whistles the SP-590UZ had. If you want the EVF, I would say go either for the Stylus1 or SP-100. Go for the Stylus1 if you want pocketability, more advanced shooting features, at a cost of much less zoom range. Go for the SP-100 if you don't mind a slightly bigger camera, but that it still is an all-in-one camera. Go with the E-PM2 or E-M10 if you want to get into more control, and have better image quality for large prints (showing images at web resolutions or printing 4x6 pictures likely won't show up the better quality).
