Thinking of upgrading the Canon 70-300 IS

The 70-300 IS isn't particularly sharp at 300mm, nor does it have much contrast. These appear to be your chief objections.

The 70-300 IS L is a major improvement on both points. It's such an improvement that for the purposes of retaining detail, it may as be a longer focal.

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=358&Camera=453&Sample=0&FLI=4&API=1&LensComp=738&CameraComp=453&SampleComp=0&FLIComp=4&APIComp=0

It's also roughly the same size as the non-L and therefore portable. Lenses with more reach start to get unportable in a hurry. The 100-400/L is your next best bet, and it's even heavier, not as sharp, two inches longer, and has a less effective IS system.

As to the others:

I had a 70-200/4L IS. I did miss the 200-300mm range and tended to use it exclusively with a teleconverter. If you're already short at 300mm, dropping to 200mm won't help.

The only advantage of the 300/4L IS over the 70-300/L is that extra stop at 300mm. The newer IS system of the zoom will make that up with stationary targets. Everything else also favors the zoom; size, weight, versatility, and even sharpness.