I think it was mainly a compromise. Smaller film was really grainy, larger film, ie medium format was inconvinient for the average person. With modern digital, a 1.6x sensor can have the same resolution as a piece of 35mm film, but have less noise at equivalent ISO.
I saw somebody put full resolution comparison between the 40D and 5D on the forum months ago, and the differences were mostly negelegable. Yes, the 5D has slightly more resolution, slightly less noise, and requires less out of glass to make the most of itself. For the average photographer, is it worth the cost? That is their call. But I doubt crop is going anywhere.
Like others said earlier, it is much less expensive to produce, bodies and lenses can be smaller, and the average consumer is starting to realize they rarely need more than 10MP in a camera. A 10MP dSLR can easily produce clear, stunning, well detailed 15x20s, and most people don't even go that large.
Even a good point and shoot can make good prints at 8.5x11. The 6MP fuji 1/1.7 SCCD series made some awesome medium sized prints. The later generation, ie F10, F11, F30, F31fd of this sensor could produce awesome medium prints even at ISO400.
And of course a dSLR could produce better photos than the P&S I mentioned before. My point was that the detail and resolution they are capable of is enough for medium prints, ie 8x10 or 8.5x11.
I hope to soon get some 20x30 prints from my 20D for a gallery in my photography club. I may upsample them in PS to try and maximize IQ, but I doubt the 8MP of my 1.6x sensor will be of severe limitation, especially at standard viewing distances. And how much better would those shots look at 20x30 if taken with a 5D and a comparable lens? My guess is slightly better if you looked very close, possibly with a magnifying glass.
And considering Olympus went the route of no possiblity of FF with their 4/3rd system, they will continue to produce some of the smalles dSLRs, and Canon could not come close to an Olympus size camera if they eliminated crop sensors.
--
Main Albums:
http://picasaweb.google.com/Carskick/
Older Albums:
http://www.flickr.com/photos/carskick