Please advice

This has nothing to do with focusing. You used a much too slow shutter speed with a much too long focal length amplifying any motion blur from either yourself or the subjects. With a wider angle and IS I think this would have worked ok, but not at 80mm.