15-85mm 'L' Upgrade?

Hi, last year I changed from an Olympus system to Canon after receiving lots of advice on here. I went for the 7D and have never looked back. I only have two lens at the moment, the 15-85mm and 70-200 f4 L.

In three months time I am off to Cuba and really only want to carry one lens. I was going to upgrade the 15-85mm for the 24-105mm F4 L series lens purely for the reason that I want to have the best optical quality I can for the trip as I may never get the chance to return.

After reading various threads about the 24-105mm I am not so sure about upgrading now! Also due to the crop factor I am a bit worried the 24mm might not be wide enough for general street/walkaround photography whilst in the likes of Havana.

Interested in hearing some opinions/options on what I could upgrade to or should I just stick with the 15-85mm? I don't think I can cover that focal range in L series glass with a single lens!