Do you use polarizing filters for wildlife photography?

My family and I are headed to Australia for a month. I'll be (over) packing the usual kit:

bodies: D2X, D1x, D80,
nikkor glass: 300 2.8, 17-55 2.8, 70-200 2.8, 35 2.0, 12-24 4.0,
teleconverter, tripod, computer, etc.

While chatting about the trip with a co-worker, and fellow enthusiast, he strongly suggested that I use polarizing filters since the sun is so bright, and "they have holes in the ozone."

I have a polarizing filter for the 300 2.8, but I'm wondering if anyone has any POV on the use of the polarizing filter before I decide to use it and/or quickly acquire filters for the rest of the glass.

Obviously, I give up some light, and a full stop or 2, when I use the filter. The filter "manual" also suggests not using matrix metering. Any other issues? Distortion?

Thanks everyone,

--
George