What's technically the most challenging angle for stray light and filters? In the corner of the frame?
I had bad flare around indoor lights due to a filter once. It's true that I didn't try with the filter off, but then again I never saw the problem anywhere else with any other lens either.
You really can't know how a filter is affecting a particular situation unless you test both with and without a filter. A few years ago I did a quick test of the same scene with and without a filter. The image with the filter looked fine. I didn't see any filter related issues. When I took the filter off, the image got better. Overall contrast went up, and a "reflection" in the original image went away (it turned out it was flare). Obviously, one shouldn't extrapolate a single test to all situations. But my point is that sometimes you don't realize you've lost some image quality until you compare.
Of course a reasonable perspective is that if you are happy with your quality when you use a filter, then you don't have a problem that needs to be solved.
====================
There is more than one type of flare, and the problem angles will vary.
One issue is that the flat filter parallel to the highly reflective sensor can produce an "
infinity mirror" effect. This is more of a problem for light sources that are actually in the frame. Film cameras did not have this issue as unexposed film emulsion is not very reflective.
Canon mentions this issue in their excellent "Lens Work" book. They point out that some of their high end telephoto lenses have a built-in protective element. The protective element is specifically not flat in order to avoid these internal reflections.
My expectation is that the further the light is from the field of view, the less of an issue it will be. This is true whether or not you are using a filter, but filters can make the issue worse.
If the light is outside the field of view, then you may wish to shade the filter/front of the glass. Use a lens hood if you can. If you are outside, and the sun isn't behind you, use something (a magazine, black cardboard, your hand) to put the front of the lens in shade.
====================
Remember, not all filter use is "protective". Polarizing filters and Neutral Density filters all you to do things that would be very difficult in post processing. These filters have the same flare and contrast issues as "protective" filters. No matter why you have a filter on your lens, you should be aware of these issues.