Just take both 400 lenses, shoot the same real film, not MTF, and
judge the difference.
Both these lenses are optically superbe, definitely of the highest
caliber. For the record, the Canon 400/2.8L IS scored a 4.4 at
photodo, a truly excellent score. In fact, there are no other
Canon lenses that exceed that score, except for one: the Canon
200/1.8L with a score of 4.8, which photodo claims is the highest
score of any lens they have ever tested. The Nikon 400/2.8 AFS,
unfortunately, is not tested. Of the Nikon lenses that photodo has
tested, there are no scores higher than 4.4.
But testing in controlled conditions aside, in the field you will
see the real-world performance edge going to the Canon simply do to
the vibration-cancelling effects of IS. Additionally, the Nikon
400/2.8 AFS suffers from one of the poorest tripod collar designs
in recent history, as discussed here:
http://www.naturfotograf.com/tripod_collar_rev00.html
Mounting a lens onto this lens is somewhat like being on the
business end of a diving springboad. Compare that with the Canon
400/2.8L IS lens's design, which has a much shorter, lower tripod
neck, that is positioned in a position of better balance, closer to
the camera body. Additionally, this effects how it handles with a
monopod, too, because it allows you to position the head and
monopod much closer to your body rather than farther out, as the
Nikon 400/2.8 tripod collar design requires.