Memory requirements (2009 Mac Pro) for Photoshop/Lightroom

What are the optimal memory requirements for Photoshop CS4 and Lightroom 3 on a Mac?

I'm looking to buy a 2.66 GHz quad-core Mac Pro (2009) and have read conflicting advice from the Mac Performance Guide where the author states that 12 or 16GB is enough, but somewhere else advises to upgrade a Mac Pro to 24 or 48GB right away. I understand that configuring and setting up separate fast internal SATA drives as well as adding an SSD counts a lot for increase in performance.

I consider myself a light/medium Photoshop user (usually 20 layers or less, but in time I might become a heavier user) and a "regular" Lightroom user. How much memory do you suggest I get for a productive work environment? And would I be better off using just 3 memory slots instead of all 4? I've read that this can increase performance as the first 3 slots go directly to the CPU while the last one gives slower memory access.