Keep my 20mm f1.7 with this gear...?

When I say that video is important to me, than I don't mean that I want to produce professional video's. I just want to shoot casual video's of my family without constant blurring video's, that's all. When filming with the 20mm, the video is often blurred because it's focus hunting. Now that I'm using the 12mm, it doesn't occur anymore.