Judging the camera or judging processing?

I'm afraid you (and the OP) are kidding yourselves. You seem to be under the impression that (for example) getting colours, contrast and sharpening right in the camera is fundamentally different from adjusting them afterwards at the computer. It isn't: it's identical, except that you have more control on a computer (you are not restricted to a limited range of settings like +1, +2, +3 etc.)

But what exactly are you comparing? Colour fidelity? That depends on how good the auto-WB is as much as anything else, not to mention the demosaicing algorithm used by the camera, as well as the default saturation settings etc. Sharpness and resolution? that depends on the lens as much as the sensor. Highlight detail, shadow detail, contrast? Depends on the tone curve used by the camera at the jpeg processing stage.

Yes, because it's true.

But that is not judging the camera's performance: that is comparing the results of your choice of jpeg settings against somebody else's, which has very little to do wit the camera. Do you know how much variation you can get out-of-camera by playing with these settings? Try turning all of the settings that you have control over (contrast, sharpness, saturation, noise reduction, vibrance, highlight protection, etc, etc., to their minimum settings. take some pics in varied lighting conditions. Then turn all the settings to maximum and take some more pics. Compare them. Which pictures best reflect the 'performance of the camera'?  is that even a meaningful question?

If you compare the appearance of your out-of-camera pics against those from a different camera the single biggest variable is this choice of jpeg settings which is up to you and not the camera. Yes, you could try and set up cameras X and Z to produce output as similar as possible for side-by-side comparison, but there will still be differences (what does '-2 noise reduction' on a Nikon mean compared to a Canon?) And unless the lenses are identical, differences between one lens and another will be just as significant as anything else.

That's great. but 'as good as I can' is a subjective judgment... what you mean is 'so that pictures come out how I like them to look'. Others may have different opinions about how pictures should look out-of-camera and will set their camera accordingly, so any comparison you can do is just a comparison between the different personal preferences of two people.

As pointed out below the best way to compare cameras would be to shoot RAW images from different cameras (with the same lens!) and then use a RAW convertor with the same settings for output from each camera. Then - and only then - would you have a chance of comparing sensor output without the camera's own software or lens differences getting in the way... but this requires some post-processing at a computer which completely defeats the point of the OP's query.

Best wishes

--
Mike
http://www.pentaxphotogallery.com/artists/mikeward