Manual exposure variations - why?

This is a problem defined by physics and easily ascertained. Either the light levels changed, the panning changed (longer in one spot than another, or there is mechanical defect. Take a series of pics with fixed, known light levels (hint: use a light meter) and see if they vary. If yes then get your camera or lens serviced.