Biltmore is the least important destination on the trip. In fact if it turns out to be too difficult we may skip it. We toured 5 mansions in Newport, RI last fall in about 4 hours. Two were even Vanderbilt properties. It was fun but not worth any excessive issues. The mountains and trees are the reason for the trip.
How crowded is the park and is there anything I can do to avoid it?
I can't compare biltmore to any of the mansions in Newport, but don't be so quick to dismiss it.
Regardless, Clingmans Dome is VERY popular since you can drive to it. We hike to Mt. Lecont Lodge every October. Nice hike (I'd recommend Alum Cave Trail), but if you have 2 cars, you can do Alum Cave and The Boulevard. The Boulevard is the longest trail to the lodge, but it easily has the best, sweeping views of the 5 trails going up to (or coming back from) the lodge. Anyway, after coming down off the mountain we've tried to drive to Clingman's dome and the line of cars was so long we finally said eff it and turned around. So if you want to do Clingmans dome, like everything else in a NP, do it early before the late risers get into the park.
Another cool hike in the park is to do part of the AT trail up to Masa Knob I believe. That will give you a great view if the leaves are in color.
Another great hike, although not in the park, is Looking Glass Rock. You can see it from the Blue Ridge and it's a fine view, but the hike is pretty good too.
Mount Mitchell is always popular, what with it being the tallest mountain this side of the Mississippi, but last time I was up there there were a lot of dead trees. Don't know if it's recovered.
But the single most important thing you need to bring is patience, especially on the Blue Ridge Parkway. If you find yourself getting frustrated at the traffic, pull into one of the many overlooks and enjoy the views. No way to avoid the traffic. Sorry, it's just the way it is. And avoid Gatlinburg at all costs! It screams touristy, and believe me, there are a ton of tourists there. Have fun.