Well - if this was 4 years ago - i would have recommended the Oly c5050 with oly case - it was a killer combination with a super bright lens. These days, its a little harder. Oly, sony, fuji and canon all make housings for their P&S cameras. That is by far the cheapest way to go - housings by pro housing companies are quite pricey (and worth it but not a great starter package).
There are a few key things to look for though - most importantly i would say is the ability to take raw photos. The canon G7 would have been great if it did raw, but i suspect it doesnt cause its very noisey. With raw shots, underwater pics can be fixed in ways that just isnt possible with jpg alone. In fact, if you take raw pics, you can even eliminate a step of white balancing underwater if you so desire - and that can be tedious to do and nice to avoid. Another key thing is the built in wide angle - is it wide enough? Basically, there are two kinds of shots underwater - shipwreck pics/diver pics and then macros. The main thing is to get as close to the subject as possible so that you can eliminate as much floating stuff in the water as you possibly can. The closer you are, the less stuff you're going to see. Thus, wide is very good, and wider is better. You can get external wide angle lenses as well, but its nice to not have to worry about those either. If your water is reasonably clear, again its not much of a concern.
For macro, you want a macro that gives you a long enough lens to not scare things away as you get closer. On this trip, i had the 60 and the 100 - i found the 60 required me to get too close and the 100 felt just right (on a 1.6 crop camera). Creatures might be different elsewhere. But, again, from a point and shoot point of view, try to get the equivalent zoom of a 60-100 range plus decent macro ability.
There are also two types of lighting situations - flash and natural. I love natural shots but it requires either high iso or a fast lens. You likely wont have the fast lens option at the zoom end on a point and shoot, but it might be decently fast at the wide end. In the absense of that (or in addition to that) a quality high iso is also beneficial - noise is the enemy here.
So basically, look for those features in a point and shoot - good wide, decent zoom/macro, good quality high iso and a decently fast lens equivalent and the ability to take raw shots - and you're on your way. Another thing to consider is the shutter lag and/or flash refresh time - these are key to getting fast moving creatures as they tend to flit around alot. Otherwise, there will be lots of fish tale shots in your collection.
So, find the camera, then verify that it has a cheap housing by the same manufacturer and you're off to the races.
steve
I just find it hard to justify the fact that some of these housings costs more than the camera and lens 
