X-Pro1: not having much luck with video. Tips?

Not getting great results from my video: focus hunts constantly. Here's an example from the Virginia Renaissance Fair. Camera set to AF-S:

http://player.vimeo.com/video/42078185

I didn't buy the Fuji for video--the iPhone 4S is always in my pocket and does just fine. But I'm curious to hear whether other people are getting better results and if so, how.

Thanks.
--
Marc A. Garrett
http://flickr.com/photos/since1968/