85mm 1.8 or 17-85mm IS

Newbie with a limited budget and a 30D, so I'm collecting my lenses slowly. Here's what I have now:

50mm 1.8
18-55mm IS
55-250mm IS
...and 17-85mm IS

I shoot family portraits, family candid shots, architecture, nature, and the standard novice wannabe artsy shots.

I just picked up the 17-85mm IS last month and was going to get rid of the 18-55m IS, but was discouraged by the distortion at 17mm with the 17-85mm after shooting 200 images. Of course, the upside of the 17-85mm is the range. Perhaps the new 15-85mm will perform better.

I love the shots and DOF I've seen from the 85mm, but am afraid I won't use this lens as much as I expect. It'll also be great in low light situations of course.

At this point, I'm planning to keep my 18-55mm, and sell off the 17-85 to buy an 85mm 1.8. Is this a good idea? I know the answer will depend on what I shoot and realize that I'm comparing apples to burgers, but perhaps there's a consensus among you experienced users as to whether either the 17-85mm or the 85mm is an "essential" lens.

Any thoughts?