Capture 4.3.1 speed/memory analysis (quick and dirty)

[Joseph S Wisniewski]

I spent a while watching memory and timing Capture 4.3.0 and 4.3.1 performing different operations on D100 and D2X files. And I've come to the conclusion that the speed difference isn't that big, if you feed Capture enough memory. Perhaps 30% faster (and highly dependent on machine. One computer I tried had the exact same speed, within 5%). The big difference is in memory handling.

4.3.0 appears to open a full floating point buffer for every image that you open. That means that a single D2X image takes up 369 megabytes in Capture (12.3mp x 3 colors x 10 bytes for a double precision float). And, if you use high quality mode of Dlighting on an image, it took another full 369 meg floating point buffer. So a machine with 1 gig of memory would start swapping to hard drive if you did dlighting on a single D2X image. Open 2 images, use dlighting on one, or open 3 images and don't do anything elses, and NC ground to a near halt.

4.3.1 shares the floating point buffer among all images, and only grabs a 16 bit integer for each image that you open, without demosiacing the raw to RGB. So, 393 megs for the first D2X NEF, 24 megs for each additional. About 2 megs extra per image for dlighting, so it actually looks like Nikon cleaned up their software.

Smaller NEFs use proportionally less memory.

It's nice being able to open a dozen NEFs at a time

--

Salvage troll posts! When you see a thread started by a troll, post something useful to it. It will drive the trolls up the wall.

Ciao!

Joe

http://www.swissarmyfork.com

 

[Andreas Berglund]

I tested it on a rudimentary level this morning and here is what I got sofar

1. Grapichs handling was on average 22% faster, not much to write home about really seems they haven't rewritten the grpahics engine just tuned it. They still seems to be using the CPU for graphics operations not the GPU (BAD!)
2. Loading NEF marginally faster
3. Saving JPG's slightly slower (!)

Memory handling I haven't tried yet, i'm encouraged by your tests

--
Andréas Berglund
delapsus resurgam
(email and equipment in profile)

 

[Jon Beanland]

I am running dual 246 opterons with 4 gigs of ram. Nef files open in 4 seconds from 7 and tiffs save in 7 seconds from 15. Batch processing speed seems much faster now. Capture seems to be utilizing both cpu;s in task manager more now. When opeing files both processors will utilize 100% and when bathcing files they both are running around 70% in task manager. Seems like pretty good improvment on my machine. Jon

 

[Rich H]

I see a 30% improvement consistently on opening a single D2X NEF. With 4.3.0 it took 13 seconds from clicking edit in Nikon View to having the NEF come up in Capture completely (assumes capture was already running). Now it takes 9 seconds. P4 2.4Ghz, 2GB RAM.

Rich H

 

[Jason Elander]

This is great for me. I have a laptop with a modest 512 megs of RAM, and previously struggled with my D100 NEFs, much less D2X files. Using the multi-image window was nothing but a cruel joke. I upgraded last night and it's like a whole new program! Individual NEFs open up much faster, and in the multi image window at least 5-10x as fast. Simply fantastic...

Jason

 

[devenh]

I have a dual Xeon 2.2 ghz system with 2gb RDRAM, RAID 5 hard drive. Upgraded from NC 4.3.0 to 4.3.1. To load four D2x NEF took 51", now it takes 34".

Saving four D2x NEF files took a wopping 2'25" (appeared to be swap file related), now takes 21" -- almost a 7x improvement!!!

Deve

 

[Colin196788]

...can be found here, FYI:
http://nikonimaging.com/global/news/2005/0822_01.htm

--
^^^ Colin. ^^^

 

[kccheng]

the GPU won't help you on photo retouch.
--
Ka-Chun Cheng
Hong Kong, Nikon 5K7, D70
http://www.kachun.com

 

[Jeff Klofft]

will make a bigger difference on memory starved machines. I believe that is why we are see the differences.
--
Jeff

Check out my gallery at http://www.pbase.com/jklofft

 

[peetnik3]

Hi Jon,

How do you like your dual opteron 246 system w/4MB RAM for processing D2x NEFs through Capture? Ron Reznick recommended the same setup to me and I've been contemplating building a system around those specs. What is your motherboard and your hard drive setup if I may ask? Thanks very much in advance.

Andrew

 

[Jon Beanland]

I like it and alot better now since capture is faster.
Tyan 2885 board
Corsair memory 4 dimms for a total of 4 gigs
Matrox p/750 graphics card

2 10000 rpm scsi drives running on adaptec 2120s controller in raid 0 for opertating system

2 10000 rpm scsi drives running on 2120s controller raid 0 for Photoshop scratch disk
western digital 250 gig for data with 2 hot swap WD 250's for dual backup
WD Raptor 10000 rpm drive for windows swap file

This system is about the same perfomance as my p4 system opening and saving single files in NC, but where you see the difference is when you go to opening multiple files like four or more at a time in Capture. For example. Opening 5 d2x nefs in capture takes about 40 seconds on the P4 3.2 presott while on the opteron it only takes about 20 seconds before all 5 files are fully loaded. Also you can batch process files and work in photoshop at the same time with little performance decrease. This system has been very stable and with the ability to work with multiple applications at the same time, i think it has been worth the upgrade. Jon

 

[Ed]

I can think of no reason they would create buffers of floating point values.

How did you come to this conclusion?

 

[Joseph S Wisniewski]

I can think of no reason they would create buffers of floating
point values.

Really? I can think of dozens of reasons, and I only dabble in image processing software. Unless you want to spend outrageous amounts of time optimizing algorithms for fixed point, floating point is the way to make large area algorithms (virtually anything in the frequency domain, cepstral deconvolution, etc) run predictably.

How did you come to this conclusion?

As I mentioned earlier, an educated guess, based on the amount of memory that Nikon capture is consuming to process images. 10 bytes per pixel, which doesn't leave much besides floating point.

And the fact that Nikon Capture heavily loads the floating point unit (aren't debuggers wonderful) when it's doing just about anything.

--

Salvage troll posts! When you see a thread started by a troll, post something useful to it. It will drive the trolls up the wall.

Ciao!

Joe

http://www.swissarmyfork.com

 

[Joseph S Wisniewski]

will make a bigger difference on memory starved machines. I
believe that is why we are see the differences.

That's where I see the big difference. I was watching 4.3.0 and 4.3.1 on the debugger. 4.3.0 exhausted the physical memory and started swapping virtual memory under some pretty simple situations (just one D2X NEF open with dlighning, or two open without any processing at all) while 4.3.1 never swapped, even with 20 D2X NEF files open at once.

--

Salvage troll posts! When you see a thread started by a troll, post something useful to it. It will drive the trolls up the wall.

Ciao!

Joe

http://www.swissarmyfork.com

 

[Joseph S Wisniewski]

I have a dual Xeon 2.2 ghz system with 2gb RDRAM, RAID 5 hard
drive. Upgraded from NC 4.3.0 to 4.3.1. To load four D2x NEF took
51", now it takes 34".

Saving four D2x NEF files took a wopping 2'25" (appeared to be swap
file related), now takes 21" -- almost a 7x improvement!!!

That's the strong point of 4.3.1. I had 20 D2X ENF files open at the same time with it, and it never ran out of physical RAM and never once touched the swap file. I wouldn't even do four D2X NEF files at the same time, for exactly the reason you said, it swapped.

--

Salvage troll posts! When you see a thread started by a troll, post something useful to it. It will drive the trolls up the wall.

Ciao!

Joe

http://www.swissarmyfork.com

 

[devenh]

With the improved performance looks like I can postpone the installation of a dedicated 10k SATA RAID 0 swap drive.

Deven

 

[Ed]

Well, I write software, and yes one will use floating point math - even buffers as large as several rows of an image being processed - but a buffer the size of the image? There is no need for that. I would have a hard time believing Nikon would be quite that silly.

I have worked on software that processes images well over 10K pixels in at least one dimension, and there was no advantage in creating an image buffer that deep.

Plus, I have watched NC on both platforms, and I have never seen them create blocks of memory large enough to account for that. I'll have a look at the latest, though, out of curiousity.

I can think of no reason they would create buffers of floating
point values.

Really? I can think of dozens of reasons, and I only dabble in
image processing software. Unless you want to spend outrageous
amounts of time optimizing algorithms for fixed point, floating
point is the way to make large area algorithms (virtually anything
in the frequency domain, cepstral deconvolution, etc) run
predictably.

How did you come to this conclusion?

As I mentioned earlier, an educated guess, based on the amount of
memory that Nikon capture is consuming to process images. 10 bytes
per pixel, which doesn't leave much besides floating point.

And the fact that Nikon Capture heavily loads the floating point
unit (aren't debuggers wonderful) when it's doing just about
anything.

--
Salvage troll posts! When you see a thread started by a troll, post
something useful to it. It will drive the trolls up the wall.

Ciao!

Joe

http://www.swissarmyfork.com

 

[Ilkka Nissilä]

Well, I write software, and yes one will use floating point math -
even buffers as large as several rows of an image being processed -
but a buffer the size of the image? There is no need for that. I
would have a hard time believing Nikon would be quite that silly.

I suppose Nikon does this in order to avoid rounding errors when a sequence of operations is applied on the image interactively. Photoshop doesn't do it like this by default and so I think you need to do all operations as layers in one shot so that you get a similar effect - and layers do use memory.

 

[Andreas Berglund]

No but it does with Zooming if you use the GPU for it is supposed to be used for. When one compares the pitiful zoom times to 100% with the times it takes for PSCS (1 and 2) one can see the difference it makes if one uses the GPU.
--
Andréas Berglund
delapsus resurgam
(email and equipment in profile)

 

[Joseph S Wisniewski]

Well, I write software, and yes one will use floating point math -
even buffers as large as several rows of an image being processed -
but a buffer the size of the image? There is no need for that. I
would have a hard time believing Nikon would be quite that silly.

I have no trouble believing Nikon can be really, really silly.

Look at the D100 version 2.0 firmware upgrade. Rather than putting it up for download, and having most users get it right and a small percentage screw it up and need to send their cameras in for service, they required 100% of users to send their cameras in to service for the firmware upgrade. So many people sent their cameras in that it was more than the regular Nikon service centers could handle, and they had to set up temporary service facilities just for firmware upgrades.

Or look at the D2X encrypred white balance issue, which was basically Nikon's response to Dr. Jaewook Chung finding a flaw in the way Nikon computed white balance.

And what's the difference between an SB-28 and an SB-28DX? SB-28 hardware is fast enough to generate the preflash sequences for D-TTL. It's just firmware, with a new model number, and a $40 increase in price.

I have worked on software that processes images well over 10K
pixels in at least one dimension, and there was no advantage in
creating an image buffer that deep.

Agreed, if it's production software.

Plus, I have watched NC on both platforms, and I have never seen
them create blocks of memory large enough to account for that. I'll
have a look at the latest, though, out of curiousity.

Which was the last NC you watched? A version 3.x, perhaps. They were much better with memory than 4.X. That's when things really started to go to heck.

--

Salvage troll posts! When you see a thread started by a troll, post something useful to it. It will drive the trolls up the wall.

Ciao!

Joe

http://www.swissarmyfork.com