K5: Testing RAW Continuous Shooting

Started Oct 20, 2010 | Discussions
richardplondon
richardplondon Forum Pro • Posts: 10,762
Re: PEF slower?

dlacoutu wrote:

Problem is that the Adobe Raw Converter strips information from the PEFs while converting them to DNGs...

The black pixel stripe, for instance, is removed, but it can be helpful (even essential on a K20!).

In a free choice, I would choose either compressed PEF, or camera-native compressed DNG.

From memory, it does not so when going from a K10 DNG to a compressed DNG.

K10d's uncompressed DNG takes up a lot of space on the card), and carrying PEF through the whole workflow has no real downside that I can determine - so that is what I do. If PEF needed to be turned to DNG for some reason, I could do that later. Harder to convert back, though

DNG is not yet any more software compatible, than PEF - except with brand new camera models, which don't yet have all the final calibration profiles etc anyway.

RP

 richardplondon's gear list:richardplondon's gear list
Pentax K-5 Sigma 10-20mm F4-5.6 EX DC HSM Pentax smc DA 21mm F3.2 AL Limited Pentax smc DA 70mm F2.4 AL Limited Pentax smc DA 35mm F2.8 Macro Limited +6 more
MightyMike Forum Pro • Posts: 38,599
Re: I'm pretty sure the raw buffer is at least 256 MBytes...

Thanks Gordon, It looks like we're getting to the bottom of this, if we haven't reached there already
--
Mike from Canada

'I like to think so far outside the box that it would require a telephoto lens just to see the box!' ~ 'My Quote :)'

http://www.airliners.net/search/photo.search?sort_order=views%20DESC&first_this_page=0&page_limit=180&&emailsearch=mighty_mike88%40hotmail.com&thumbnails=

Jeff Charles Veteran Member • Posts: 7,514
Re: 128mb buffer in my estimation...

MightyMike wrote:

... I do understand that one can fire off smaller burst while waiting for the buffer but you can't review the initial burst set until the buffer is empty...

Does the entire buffer need to be cleared before any of the photos can be reviewed? If not, a bigger buffer seems desirable. The larger buffer will support a longer initial burst, and once the "20-second" buffer is half cleared after 10 seconds, it will have the same capacity for another burst as the "10-second" buffer.

When I shoot a burst with my D90, the first images are available for review right away.
--
Jeff

'Yeah, well, you know, that's just, like, your opinion, man.' The Dude

 Jeff Charles's gear list:Jeff Charles's gear list
Sony RX100 III Fujifilm X100T Olympus E-M1 Fujifilm X-E2 Olympus M.Zuiko Digital ED 40-150mm 1:4-5.6 R +5 more
MightyMike Forum Pro • Posts: 38,599
Re: 128mb buffer in my estimation...

Just checked on the K-7... yes the entire buffer has to be emptied before i can review a photo
--
Mike from Canada

'I like to think so far outside the box that it would require a telephoto lens just to see the box!' ~ 'My Quote :)'

http://www.airliners.net/search/photo.search?sort_order=views%20DESC&first_this_page=0&page_limit=180&&emailsearch=mighty_mike88%40hotmail.com&thumbnails=

Jeff Charles Veteran Member • Posts: 7,514
Re: 128mb buffer in my estimation...

MightyMike wrote:

Just checked on the K-7... yes the entire buffer has to be emptied before i can review a photo

Thanks for checking. It's too bad that it works that way, but probably a only a minor inconvenience in most situations.

-- hide signature --

Jeff

'Yeah, well, you know, that's just, like, your opinion, man.' The Dude

 Jeff Charles's gear list:Jeff Charles's gear list
Sony RX100 III Fujifilm X100T Olympus E-M1 Fujifilm X-E2 Olympus M.Zuiko Digital ED 40-150mm 1:4-5.6 R +5 more
ET2 Veteran Member • Posts: 4,110
CompactFlash

That's why pro models (1IDIV D3s) use CF. It's (reliably) faster.

glanglois Contributing Member • Posts: 987
Re: 128mb buffer in my estimation...

Are we certain that the bottleneck is found in the physical size of the buffer and/or electrical characteristics of the card interface?

I have this suspicion, based upon burst speed when certain auto corrections are enabled, that at some point the image processor may be unable to keep up with developing, compressing, bit shifting, as well as various personal hygiene activities (protocol handling, error checking, self-diagnostics, etc.).

Could it be we're asking the PRIME II and its hardware implementation to do entirely too many thing to too many bits?

If you've kicked this around, Gordon, I'm afraid I didn't find it.

 glanglois's gear list:glanglois's gear list
Panasonic Lumix DMC-FZ300
MightyMike Forum Pro • Posts: 38,599
Re: 128mb buffer in my estimation...

Even though you are right that those settings slow things down a lot that is not the issue with the buffer we are discussing, those testing it don't have the corrections turned on.
--
Mike from Canada

'I like to think so far outside the box that it would require a telephoto lens just to see the box!' ~ 'My Quote :)'

http://www.airliners.net/search/photo.search?sort_order=views%20DESC&first_this_page=0&page_limit=180&&emailsearch=mighty_mike88%40hotmail.com&thumbnails=

Dale108
OP Dale108 Veteran Member • Posts: 7,358
Re: K5 vs Kr: Testing RAW Continuous Shooting

I purchased a Kr today to go along with my K5. For the Kr continuous shooting is better than the K5! I could shoot 12 RAW DNG files before slowing down; this compares to 6-7 K5 RAW files. It looks like the Kr will be part of my wildlife kit!

Dale
--
http://www.pbase.com/abundant108

http://www.pentaxphotogallery.com/home#section=ARTIST&subSection=272176&subSubSection=1787360&language=EN

 Dale108's gear list:Dale108's gear list
Sony RX10 III Olympus TG-5 Nikon Coolpix P1000 Olympus E-M5 II Pentax K-1 +1 more
GordonBGood Veteran Member • Posts: 6,308
Re: 128mb buffer in my estimation...

glanglois wrote:

Are we certain that the bottleneck is found in the physical size of the buffer and/or electrical characteristics of the card interface?

I have this suspicion, based upon burst speed when certain auto corrections are enabled, that at some point the image processor may be unable to keep up with developing, compressing, bit shifting, as well as various personal hygiene activities (protocol handling, error checking, self-diagnostics, etc.).

Could it be we're asking the PRIME II and its hardware implementation to do entirely too many thing to too many bits?

If you've kicked this around, Gordon, I'm afraid I didn't find it.

We've had our testers turn off all JPEG processing that can be turned off, including distortion and aberration correction, and JPEG noise reduction, with the results as to to raw buffer depth as posted and as specified by Pentax Japan: about eight frames to buffer full (actually should be specified as "for ISO sensitivities below ISO 6400"). No doubt turning on some of those things reduces the buffer depth from this.

Regards, GordonBGood

GordonBGood Veteran Member • Posts: 6,308
Re: K5 vs Kr: if only...

Dale108 wrote:

I purchased a K-r today to go along with my K-5. For the K-r continuous shooting is better than the K-5! I could shoot 12 RAW DNG files before slowing down; this compares to 6-7 K-5 RAW files. It looks like the K-r will be part of my wildlife kit!

Dale, if my computations are correct, Pentax have increased the raw buffer size of about 64 MBytes for the K-x up by about a factor of three to about 192 MBytes for the K-r. This makes sense as they could do it using the new body and circuit cards. However, it also likely means that it will take about three times as long to clear a full buffer, or about 15 seconds for the K-r rather than the about 5 seconds for the K-x. If the memory writing algorithms have been tuned a bit, the raw buffer clearing time may be a little less than this.

If Pentax were to offer a similar 12 bit raw depth mode for the K-5 using my computed use of an estimated 256 MBytes of raw buffer RAM, they would be able to offer about the same 11 to 12 raw buffer depth at full high continuous shooting speed and a typical 11 to 12 second full raw buffer clearing time, which would likely keep everybody happy. This would put the K-5 more in line with the Nikon D7000 and Canon 60D raw buffer depth, (although the Canon depth is for 14-bit raws, it will likely take about 16 seconds to clear for low ISO images and much longer for the larger high ISO ones). Hopefully, they may consider and be able to implement this as a firmware update.

Regards, GordonBGood

Dale108
OP Dale108 Veteran Member • Posts: 7,358
Re: K5 vs Kr: if only...

Hi Gordon:

Lets hope that Pentax can improve the K5 buffer write times with a firmware update as it is surprising that the "flagship' camera doesn't have the best burst in the range.

Dale
--
http://www.pbase.com/abundant108

http://www.pentaxphotogallery.com/home#section=ARTIST&subSection=272176&subSubSection=1787360&language=EN

 Dale108's gear list:Dale108's gear list
Sony RX10 III Olympus TG-5 Nikon Coolpix P1000 Olympus E-M5 II Pentax K-1 +1 more
GordonBGood Veteran Member • Posts: 6,308
Re: K5 vs Kr: if only...

Dale108 wrote:

Hi Gordon:

Lets hope that Pentax can improve the K-5 buffer write times with a firmware update as it is surprising that the "flagship' camera doesn't have the best burst in the range.

Hi Dale, just to clarify, I don't think that Pentax can actually improve "write times" with a firmware update for the K-5, and the only two improvements one might hope for* with the K-5 are as follows:

1) Adding the option of 12-bit raw output bit depth, which would make raw buffer depth a more competitive about 12 frames at 7 frames per second (fps) with about a 1.4 fps buffer full rate and about 10 seconds buffer clear time. Competing cameras may have a deeper raw buffer, but they also have a corresponding longer buffer clear time so the above might be considered optimal. Some Nikon D7000 raw modes will have faster buffer full fps and buffer clearing times for the size of buffer due to the files being about two thirds the size of those of the Pentax compressed formats ("virtually lossless" Nikon NEF format).

2) A possible rethinking/redesign of whatever cause for the halving of raw buffer depth for high ISO's of 6400 and up if redoing this does not interfere with the current contentment with the high ISO image quality.

Regards, GordonBGood

Pete Dee Veteran Member • Posts: 3,759
Re: K5: Testing RAW Continuous Shooting

Dale108 wrote:

Hi Johan:

I can only report my experience which does not support the DPreview specs; I wish they were right.

4-5 per second for pef is still quite usable at 7.2, what is it like at 3.3 fps?

What about for jpeg?

 Pete Dee's gear list:Pete Dee's gear list
Pentax Optio W80 Pentax K-3 Pentax smc DA 16-45mm F4 ED AL Tamron SP AF 70-200mm F/2.8 Di LD (IF) MACRO Tamron SP AF 90mm F/2.8 Di Macro +4 more
Kerusker
Kerusker Senior Member • Posts: 1,228
2 x 128mb 'ping-pong' buffer ?

GordonBGood wrote:

As has already been discussed with Mike, the reason that the K-5 has this amount of memory, whatever it should be, is that the K-7 had it, but also with twice the memory there would be twice the buffer clearing time of about 20 seconds, which might well be unacceptable.

Why can't PENTAX use two 128MB buffers to read/write in parallel (often called 'ping-pong' buffer). While ping-buffer is filled by CPU, pong-buffer is read out and written to card. Then ping and pong buffers are switched.

Then burst would not be limited by write speeds (using sufficiently fast buffers and cards).
Heating up the sensor with long bursts might be the problem then.

-- hide signature --

][.Kerusker
we don't see that we don't see (eye's blind spot)

my albums: http://picasaweb.google.de/Kerusker

 Kerusker's gear list:Kerusker's gear list
Olympus Stylus 1 Sony RX10 IV Pentax K-5 II Pentax K-1 Pentax smc DA 15mm F4 ED AL Limited +1 more
Pete Dee Veteran Member • Posts: 3,759
Re: K5 vs Kr: if only...

GordonBGood wrote:

Dale108 wrote:

Hi Gordon:

Lets hope that Pentax can improve the K-5 buffer write times with a firmware update as it is surprising that the "flagship' camera doesn't have the best burst in the range.

Hi Dale, just to clarify, I don't think that Pentax can actually improve "write times" with a firmware update for the K-5, and the only two improvements one might hope for* with the K-5 are as follows:

1) Adding the option of 12-bit raw output bit depth, which would make raw buffer depth a more competitive about 12 frames at 7 frames per second (fps) with about a 1.4 fps buffer full rate and about 10 seconds buffer clear time. Competing cameras may have a deeper raw buffer, but they also have a corresponding longer buffer clear time so the above might be considered optimal. Some Nikon D7000 raw modes will have faster buffer full fps and buffer clearing times for the size of buffer due to the files being about two thirds the size of those of the Pentax compressed formats ("virtually lossless" Nikon NEF format).

2) A possible rethinking/redesign of whatever cause for the halving of raw buffer depth for high ISO's of 6400 and up if redoing this does not interfere with the current contentment with the high ISO image quality.

Seems there is always a trade unless there is a large increase in cost, personally I think it will do me very well and I look forward to getting one at some stage.

Pete

 Pete Dee's gear list:Pete Dee's gear list
Pentax Optio W80 Pentax K-3 Pentax smc DA 16-45mm F4 ED AL Tamron SP AF 70-200mm F/2.8 Di LD (IF) MACRO Tamron SP AF 90mm F/2.8 Di Macro +4 more
GordonBGood Veteran Member • Posts: 6,308
Re: 2 x 128mb 'ping-pong' buffer ?

Kerusker wrote:

GordonBGood wrote:

As has already been discussed with Mike, the reason that the K-5 has this amount of memory, whatever it should be, is that the K-7 had it, but also with twice the memory there would be twice the buffer clearing time of about 20 seconds, which might well be unacceptable.

Why can't PENTAX use two 128MB buffers to read/write in parallel (often called 'ping-pong' buffer). While ping-buffer is filled by CPU, pong-buffer is read out and written to card. Then ping and pong buffers are switched.

Then burst would not be limited by write speeds (using sufficiently fast buffers and cards).
Heating up the sensor with long bursts might be the problem then.

I don't think that dividing say the hypothetical 256 MByte raw buffer into two alternating buffers would actually help that much, even for a new model camera as it obviously isn't possible for the current design of the K-5 which mostly uses the K-7 basic design with a new sensor. Let's analyse this as follows:

1) The high speed shooting speed of 7 full resolution frames per second (fps) is already competitive with what is out there and faster shooting speeds would only serve to fill the buffer of whatever size faster. Thus the real problem is the speed of writing the data to memory.

2) The flash memory interface is capable of writing up to about 22.5 MBytes per second for flash memory that is fast enough to support this.

3) The current K-5 raw file size is something close to 20 MBytes per image at low ISO's, which means that the current buffer limit of about eight images adds up to about 160 MBytes or something over eight seconds to clear if one could take full advantage of the write speed.

4) As long as the flash write speed is only about 22.5 MBytes per second, your alternating page scheme isn't going to make any difference to the buffer depth of four images per side, nor the buffer full rate of about 1.1 fps, nor the time to clear the buffer (both) of something just over eight seconds (longer for larger raw files of higher detail or noise). In fact, the camera will be waiting to switch memory "sides" for the files to be written out from the last side as the output rate is slower than the acquisition rate.

Conclusion: the real solution to deeper raw buffer depth in images is always more memory, whether paged or not, and the real solution to increasing buffer full shooting rates and decreasing full buffer clearing time is always just faster flash memory write speed for a given size of raw output file . My temporary or alternate solution proposed above may be practical for the current design of the K-5, where neither can the amount of memory nor the flash memory write speed be increased , thus the proposal is to reduce the average size of the files that need to be written by having an optional 12-bit mode.

For future models that will certainly have more memory, flash memory write speed is not likely to be a problem as they will also take advantage of the new SDXC specifications that allow for memory write speeds of 60 MBytes per second or more. Ultimately, there may be flash memory write speeds that are capable of accepting raw image files as fast as the sensor can generate them, in which case there will be a need for only a very little buffer and one would be able to shoot continuously at the maximum rate.

Regards, GordonBGood

Jeff Charles Veteran Member • Posts: 7,514
Re: 2 x 128mb 'ping-pong' buffer ?

GordonBGood wrote:

...My temporary or alternate solution proposed above may be practical for the current design of the K-5, where neither can the amount of memory nor the flash memory write speed be increased , thus the proposal is to reduce the average size of the files that need to be written by having an optional 12-bit mode.

Makes sense. Why there isn't a 12-bit mode already is unclear. The advantages that 14 bit brings may not be important in many cases, and I think I read that there is little benefit at high ISOs. So, the "price" of 14 bit has to be paid with every raw, whether you need it or not.

Also, one issue with K-5 buffer clearing seems to be that you can't review any images until the buffer is fully cleared. A couple of other DSLRs I own (Olympus and Nikon) both allow review will data is being written to the card. Could Pentax make a firmware change to support this?
--
Jeff

'Yeah, well, you know, that's just, like, your opinion, man.' The Dude

 Jeff Charles's gear list:Jeff Charles's gear list
Sony RX100 III Fujifilm X100T Olympus E-M1 Fujifilm X-E2 Olympus M.Zuiko Digital ED 40-150mm 1:4-5.6 R +5 more
Dale108
OP Dale108 Veteran Member • Posts: 7,358
Re: 2 x 128mb 'ping-pong' buffer ?

Hi Jeff:

For me this is not that big of an issue as when you are shooting an active scene like BIF, you are focused on your next shot and not as much on what hapened; at least for me.

Dale
--
http://www.pbase.com/abundant108

http://www.pentaxphotogallery.com/home#section=ARTIST&subSection=272176&subSubSection=1787360&language=EN

 Dale108's gear list:Dale108's gear list
Sony RX10 III Olympus TG-5 Nikon Coolpix P1000 Olympus E-M5 II Pentax K-1 +1 more
Jeff Charles Veteran Member • Posts: 7,514
Re: 2 x 128mb 'ping-pong' buffer ?

Dale108 wrote:

...

For me this is not that big of an issue as when you are shooting an active scene like BIF, you are focused on your next shot and not as much on what hapened; at least for me.

What you say makes sense. Getting the next shot is always more important than checking the last one. In any case, usability issues can be hard to evaluate until you actually have used a camera for a while.

johnbee did point out above that in live view, you don't get it back until writes are completed.

-- hide signature --

Jeff

'Yeah, well, you know, that's just, like, your opinion, man.' The Dude

 Jeff Charles's gear list:Jeff Charles's gear list
Sony RX100 III Fujifilm X100T Olympus E-M1 Fujifilm X-E2 Olympus M.Zuiko Digital ED 40-150mm 1:4-5.6 R +5 more
Keyboard shortcuts:
FForum MMy threads