24-105mm focusing question

I am new to DSLR, especially zoom lenses.  I just purchased a 6D with the 24-105L kit lens.  Last night I was trying to take some photos of the stars and I noticed that if I merely put the manual focus all the way to infinity, the pics came out way out of focus... To achieve a suitable sharpness, I had to just barely touch the beginning of the infinity indicator.  What puzzles me is this:  what would I use the remaining focus adjustment on? Or, is my lens maybe out of alignment?  It is only a few days old.  Is this natural with these lenses?

--
You're never lost, because no matter where you go...there you are