I don't use OVF when shooting close.
I don't use OVF when I need faster AF. AF is faster with EVF.
I don't use OVF when framing is critical. The framelines really could be more accurate. When I use a legacy 28mm on the X-Pro1, I tell the camera it is a 26mm lens and the framelines are much better.
I don't use OVF with the zoom lens. The OVF and the zoom lens work together, but not very well.
I have read many comments that the OVF is great because you can see what is outside the framelines and about to enter the shot. But, the truth is - all that stuff outside the framelines is already in the shot, it's just not in the framelines (w/35 lens). I do think this is a great argument for using the LCD though, then you can see everything that is outside the shot.
I currently have an one X-Pro1 and two X-E1s. While I can appreciate a good OVF (like on my FE2), I don't think I will be keeping the X-Pro1 much longer. If the OVF were better, I would use it more.