Return my refurbished D4 or keep it?

of a brand new D4 is worth it given the issues you have raised.  Nikon describes their refurbished cameras as "top quality" on their web site and it doesn't sound like the camera they sent you meets that criterion.  If Nikon refuses to accept the camera back then I would dispute the charge with my credit card company and go from there.

Just as an aside, I think that Nikon ought to take the approach that other retailers use, e.g., KEH, and actually grade the used products that they sell.