I have been shooting primes almost exclusively for several years now and love it. My 35 f/2 lives on my camera at least 80% of the time and I love it!
To answer your question better, it is important to know if weight and size matters to you. One of the main reasons I love my 35 f/2 is because it is so small and light, I find that I am willing to carry my camera with me most of the time now - so I actually have it with me when there are pictures to be taken! When I used to shoot big heavy zooms, I almost never carried my camera.
Re: 35 f/2 IS vs 35L - I am considering upgrading to the 35L - not because I don't like the pictures from my 35 f/2, but because I like to shoot moving subjects in low light and want that extra stop. I love how compact and light my 35 f/2 is, along with how inconspicuous it is and it's high quality images. But I think the extra stop would be perfect for me.
Re: 135L for sports - I have kids in basketball, volleyball, and soccer. For the indoor sports I am very happy with my 100 f/2! However, I have been considering getting an 85mm for indoors instead. I find that with 100mm, I often don't get the ball in the shot with the person, or the other players working with or challenging my player, and would like that in order to show the intensity of the situation. Notice your last shot of the basketball team huddled on the opposite side of the court. Even though you took the shot with the 70-200 lens, you had it set to 135mm, and they almost fill the entire frame - plenty close - point being 135 is fine for indoors.
For fun, I shot my last volleyball game with my 35 f/2 and loved the pictures! I must admit that I was allowed to move up and down the court, but I got action, players, and the ball. And with 21 megapixels, I can easily zoom in digitally in post processing if needed.
I use my 100 for portraiture as well, but find that it is too long for some of the smaller rooms I find myself in sometimes. For this reason, and the others listed above, I am considering trying an 85mm, to see how this works for me as a 35 / 85 two primary lens combo. I just think I would use the 85 more than I am using my 100.
For outdoor soccer during the day, I like my 70-300 f/4-56 IS. I don't mind having a bigger heavier zoom for outdoor field sports because taking a camera to something like this is purposeful, and think it would be difficult to eliminate a longer fl lens for field sports. I get great results from this lens, even though it is not an "L". The only time I wish I had the 70-200 f/2.8L IS II (I wouldn't get 200mm lens without IS) is for night games, where f/4-5.6 doesn't cut it for me. But since soccer is the only thing I use this lens for, it is just not worth to me for the L at this time.
So my final advise based on what I do, and my value for small, light and fast is the 35L, either 85, and whichever 70-200/300mm IS zoom is worth it for you outdoors.