Nikon D4 vs D800 ISO

Here is my question. I know the D800 spans from 100-6400 iso with boost up to 25600. The D4 spans from 100-12800 with boost up to 204000. Without boosting (which I have always seen as unacceptable image quality in any camera I've used) the D4 has only 1 stop better ISO performance. I have a D7000 right now and don't ever use above 3200. Is there a really a BIG difference in ISO performance between the two or would only people shooting in very dim light benefit from the additional stop???