All these things factor into the decision for a given venue.
Copyright: NCAA Div I and pro players have their image copyrighted
by the organizations. You can't take a snap of one of those players
and sell the image (except for news purposes). Similarly with video
- you can't shoot video often because it violates copyright.
Money: Look they want to make money off the images. For people that
want quality photos of athletes they want you buying merchandise
where they get a piece of the pie.
cramped quarters: we're talking about being in the stands here, not
on the sidelines for pro venues. The last thing I want in the stands
is someone behind me with a 400mm 5.6 lens smacking me in the head.
Very often many venues will have a policy limiting the lens size
allowed in. This allows the casual fan to take a snap and be happy
but none of the above 3 problems come into play (you're not going to
get a sellable image or even a close up and a 5" lens isn't likely to
interfere with the people around you).
This policy is the most common I've seen. Of course, policy is one
thing - then you add very low paid ushers/security into the mix and
they have their own interpretation of the policy. If you run into
one of these individuals you're likely out of luck.
Having said all that I've found baseball is the most forgiving and
camera friendly of the pro sports. For college sports depending on
the stadium and your seating it's almost not worth it anyways. Most
major college stadiums have seating that sweeps back so you're pretty
far from the action. Unless you've got a 1200mm lens or sitting
front row you're not going to get many shots.