I think what you have found is the way the X- cameras are.
There is need for a slight shift in thinking with the X system. All the film simulations are colour filters. Filters are not made with colour neutrality in mind; just the opposite. For any filter, some pictures will look great, others will look terrible. They are simply not neutral.
The filter that I find is the closest to neutral is Pro Neg High or Pro Neg Std. Not Provia, despite it being labelled as STD on the camera. I have the Olympus XZ-1 and XZ-10. They are not quite neutral as well, but the Pro Neg's are only off by similar amounts compared to the Oly cameras. The cameras that I have found to be the most neutral, (in the sense that I do not have to make any colour adjustment in post processing) are the old Olympus CCD Camedia cameras - C5050,C5060,C5070,C8080,C7000, all of which I still have except for the C5050. With the newer Olympus and the Fuji X (X30,X-E1), I have to make colour adjustment in some pictures, especially the reds.