Silly RAW conversion question

Kiran P

Senior Member
Messages
2,198
Reaction score
0
Location
Vancouver, CA
OK, this may seem silly to some, but I've been plauged by this so excuse my ignorace. I keep seeing posts over and over again about how long it takes to convert a RAW image on a computer, something like 30 seconds or more, and this is with dual processors and the whole bit.

What I wonder is how does the camera do it so quickly? I'm assuming that the brains inside the D60 or D100 are nowhere as quick as that of a dual processor system. So how can your jpeg file be saved in several seconds, and have it take multiple times longer to do on the computer. I'm assuming of course that the camera captures RAW information, which I guess it has to, but then it has to process it to jpeg given your White Balance, compression, saturation... etc. settings. So how does the camera go from the RAW information from the sensor to a jpeg file on your card in seconds in the camera while it takes well over 30 seconds for the computer to come up with the same jpeg? Thank-you.

Kiran
 
Part of my answer is conjecture so take it for what it's worth. To begin well written software isn't that bad. The original veriso of Minolta Divu is dog slow but the new version is pretty snappy.

That doesn't explain it all so I'd guess it's mostly due to a special purpose processor versus a general purpose one. The hardware in your camera that processes the images does nothing but that so it can be very, very speedy. The CPU in your computer is a CISC processor made to do everything. That combined with the overead in application software gives you a big gulf in processing time.

Also compressing straight to JPG isn't so bad. If the computer applicaiton only did that it would be a lot faster. Saving to TIFF which also requires conversion is much slower in the camera.
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
 
IMO its a great quesiton.

I can attempt part. First and foremost the camera is operating on a single application and at the machine level. Like running an old DOS application . More than likely its is DOS or some other machine level code. Its also a very straight forward series of image tags as the image is converted from raw to jpg, white bal, sharpness, etc.

Believe it or not, the 1.8GHz machines running that XP or W2K have a huge amount of overhead and something as simple as this little DOS type app won't run that well. Also many of the conversion applicaitons are pulling up the exif info, showing the image at 100% pixel view, (a big drain).

Even the code written by the Camera Compnay, i.e. Nikon Capture 3, can't perform as well as the camera which is only designed only to do one thing.

Paul Caldwell
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
 
In the camera, processing the image into JPEG is done by special hardware. While in the computer, the processing is done by software.

Take an analogy which may not be very appropriate. It is like using a photocpying machine to copy a picture against copying (re-drawing) the picture by hand. The photocopying machine is a special purpose hardware. We, human is a general purpose entity, capable of doing different things, not just drawing (copying) pictures.

Regards,

K. Tse
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
 
... I still don't see how the camera could have the computing power that is anywhere as close to the computer as it would have to be to do the job that is at least an order of 10 times in magnitude faster. (ie. 3 second vs. 30 seconds). I'm all for the hardware being specialized in doing just one thing, converting the RAW information and saving as JPEG, but with a computer, do you not just tell it to do one thing, and if you tell it how to do it well, it will? Basically, if you tell me that all the RAW conversion software is too complicated and goes through too many steps and this is what slows it down, then fine, I believe it, but this seems illogical as I would expect that the programming should be somewhat simple, or at least not needing 10 times as many steps as necessary. Think about it. You've got a CPU that is lets say 1.5 GHz. It will do 1.5 billion cycles a second, which equates to a lot of pixels being analyzed. Now if the processing power of your camera is even at 150MHz, which I doubt, and it does it 10 times faster, then you've really got a 100 fold difference. (ie. 10 times faster processing with 10 times slower processer) Granted, the full power of the processor may not be utilized, but when I see my CPU sitting idle, its only at a couple of percent, and when its busy, it can shoot right up in the high 90's, so its fair to say that a good 90% of computing power can be attributed to conversion process.

So why is it soooo slow? I don't know if I can accept that the camera only does one thing and hence, is really good at it because the computer can be programmed to do just one thing too. Why can you not take the software from the camera, rewrite it to the proper code for a PC, and have something that is in the same ballpark in conversion time? We're not just talking 50 or 100% slower, we're talking 1000% slower and beyond, at best!

Kiran
 
is the raw file information quite as raw as we think it is.. in other words i dont think the camera is doing the same as what the computer has to do.. something is being byepassed "somehow".. ??.. converting a raw file to a jpeg must be different than simply creating the jpeg without the raw file format in the first place.. i never knew it took thirty seconds.. it does seem a long time..

what i am trying to say is.. the data the camera uses to create its jpeg is not the same as the (raw file) data the computer has to work with perhaps..

trog100
  1. #
... I still don't see how the camera could have the computing
power that is anywhere as close to the computer as it would have to
be to do the job that is at least an order of 10 times in magnitude
faster. (ie. 3 second vs. 30 seconds). I'm all for the hardware
being specialized in doing just one thing, converting the RAW
information and saving as JPEG, but with a computer, do you not
just tell it to do one thing, and if you tell it how to do it well,
it will? Basically, if you tell me that all the RAW conversion
software is too complicated and goes through too many steps and
this is what slows it down, then fine, I believe it, but this seems
illogical as I would expect that the programming should be somewhat
simple, or at least not needing 10 times as many steps as
necessary. Think about it. You've got a CPU that is lets say 1.5
GHz. It will do 1.5 billion cycles a second, which equates to a
lot of pixels being analyzed. Now if the processing power of your
camera is even at 150MHz, which I doubt, and it does it 10 times
faster, then you've really got a 100 fold difference. (ie. 10
times faster processing with 10 times slower processer) Granted,
the full power of the processor may not be utilized, but when I see
my CPU sitting idle, its only at a couple of percent, and when its
busy, it can shoot right up in the high 90's, so its fair to say
that a good 90% of computing power can be attributed to conversion
process.

So why is it soooo slow? I don't know if I can accept that the
camera only does one thing and hence, is really good at it because
the computer can be programmed to do just one thing too. Why can
you not take the software from the camera, rewrite it to the proper
code for a PC, and have something that is in the same ballpark in
conversion time? We're not just talking 50 or 100% slower, we're
talking 1000% slower and beyond, at best!

Kiran
 
30 seconds seems loke a long time for a raw to jpg. The only conversion that seems to take nearly that long would be raw to 16 bit tiff.

Just my 2 cents worth.
David
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
 
There's always some wiseguy who throws out a question that no one ever thought about and bolluxes up the work. To add insult to injury, the prolog was "silly question."

I for one am tired of people revealing my ignorance about questions I never thought about. Ban then from the board would be my first response.

I can however come up with some snappy answers that start off sounding good until the last part...

1. How do you know that the camera is converting RAW to JPEG? Maybe it's translating data directly to JPEG?

2. It takes longer to deal with TIF because it takes longer to write TIF.

3. So, how come the damn machine can capture NEF in a shorter time then TIF, when it takes so long to do it in a computer? Beats me and that's why the topic starter should be banned!!!! :)

Dave
Just my 2 cents worth.
David
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
 
At first I though you were seriuos and I was getting quite peeved but then I realized that you're kidding.... right? You must be kidding!
There's always some wiseguy who throws out a question that no one
ever thought about and bolluxes up the work. To add insult to
injury, the prolog was "silly question."
Well... I used silly question just incase this was something that everyone but me knew. I don't have a camera that captures RAW so I don't have any hands on experince but I do love to read about the DSLR's just incase a pot of gold falls one day from the sky on my lawn!
I for one am tired of people revealing my ignorance about questions
I never thought about. Ban then from the board would be my first
response.
Here's where I had to read twice to see if you were flamming me or just joking. he he
1. How do you know that the camera is converting RAW to JPEG? Maybe
it's translating data directly to JPEG?
Hmmmm... translating seems awfully similar to converting. And if the camera is just translating.... taking the easy way out... why can't the computer translate instead of the dreaded converting! :)
2. It takes longer to deal with TIF because it takes longer to
write TIF.
Someone mentioned this too, and I understand that saving to Tiff on a card takes a while because of write speeds, but a coputer should be able to write a 20mb tiff in only 3 or 4 seconds, no? (I'm guessing the tiff would be about 20mb but I don't remember... maybe more)
3. So, how come the damn machine can capture NEF in a shorter time
then TIF, when it takes so long to do it in a computer? Beats me
and that's why the topic starter should be banned!!!! :)
I think its because a) it doesn't do anything to the NEF unlike the TIFF which is processed and b) the NEF is a smaller file to write than the TIFF, and this is easily several seconds consider the write speeds of cards or microdrives. Oh wait... i don't think you were asking... we're you? Oh, and thanks for the smile at the end.... I was getting worried! ;)
 
but basically u have ask a question no one seems to know the real answer to.. he he.. at least not when u stop and think about it.. anyway i recon my answer is the nearest to being correct so far.. the camera is not having to convert a raw data file.. the computer is.. he he he

trog100
  1. #
There's always some wiseguy who throws out a question that no one
ever thought about and bolluxes up the work. To add insult to
injury, the prolog was "silly question."
Well... I used silly question just incase this was something that
everyone but me knew. I don't have a camera that captures RAW so I
don't have any hands on experince but I do love to read about the
DSLR's just incase a pot of gold falls one day from the sky on my
lawn!
I for one am tired of people revealing my ignorance about questions
I never thought about. Ban then from the board would be my first
response.
Here's where I had to read twice to see if you were flamming me or
just joking. he he
1. How do you know that the camera is converting RAW to JPEG? Maybe
it's translating data directly to JPEG?
Hmmmm... translating seems awfully similar to converting. And if
the camera is just translating.... taking the easy way out... why
can't the computer translate instead of the dreaded converting! :)
2. It takes longer to deal with TIF because it takes longer to
write TIF.
Someone mentioned this too, and I understand that saving to Tiff on
a card takes a while because of write speeds, but a coputer should
be able to write a 20mb tiff in only 3 or 4 seconds, no? (I'm
guessing the tiff would be about 20mb but I don't remember... maybe
more)
3. So, how come the damn machine can capture NEF in a shorter time
then TIF, when it takes so long to do it in a computer? Beats me
and that's why the topic starter should be banned!!!! :)
I think its because a) it doesn't do anything to the NEF unlike the
TIFF which is processed and b) the NEF is a smaller file to write
than the TIFF, and this is easily several seconds consider the
write speeds of cards or microdrives. Oh wait... i don't think you
were asking... we're you? Oh, and thanks for the smile at the
end.... I was getting worried! ;)
 
what i am trying to say is.. the data the camera uses to create its
jpeg is not the same as the (raw file) data the computer has to
work with perhaps..
I don't know, but I don't think this can be it. The whole idea about RAW data from my understanding is that it is information as it is pulled off the sensor before anything is done to it. Now I think that we can both agree that no matter what format you choose to save the picture in, you start with the same thing that the sensor got as it pulls it off and into its little computer. Now, depending on what you want to do with it is where the line splits. If you want RAW, I think is just takes all that info, dumps it in a file, and saves it. If you want TIFF, it has to apply your sharpening setting... tone... saturation... WB..... and then gets a picture. If you want a JPEG, it further has to compress it. Of course the Tiff takes longer cause its a big file to write compared to the jpeg, but both are processed. So I do think that the data the camera has off the sensor is the same data used either for saving as RAW, or saving as JPEG/TIFF. In fact, i do remember phil mentioning in the D60 or D100 review that its odd that applying sharpening in camera was different than the same amount of sharpeing when converting a RAW to TIFF on the computer, with the idea being that Nikon or Canon should have given the RAW conversion software the same properties as the camera had, but somehow... the camera didn't do enough sharpening in camera for the normal mode and that's why people complain about their pictures being soft... oh yes.. this was the Nikon D100. but he did go on to say that the normal sharpening in the software on the computer was just about right... or someting like that.

Kiran
 
Hi Kiran

Yes I also hope that I was kidding. You raised this question - It never occured to me and if I lived to a thousand years it wouldn't have occured to me!

One of my annoyances with these formats is the time it takes to translate a NEF. So, from time to time, when the shots were important, I would use NEF, making little growling sounds and cursing.

Your question is indeed a very good one...and I am clueless. It's not enough to say, well, the program is in ROM and that's why the camera writes so quickly.

I will say one thing, when a NEF is taken, the camera parameters are embedded in the file. When you load it into an editor you can CHANGE the parameters but they are already embedded. IOW, what you set your camera for is the default. Perhaps this is a clue?

Thanks for the mindbender.

One thing you might try is posting this on the pro forum. Some of those people seem to know the tech aspects of these kinds of questions.
At first I though you were seriuos and I was getting quite peeved
but then I realized that you're kidding.... right? You must be
kidding!
Well if it was a flame I suspect my credibility on this board, already low, would drop to zero.

Dave
There's always some wiseguy who throws out a question that no one
ever thought about and bolluxes up the work. To add insult to
injury, the prolog was "silly question."
Well... I used silly question just incase this was something that
everyone but me knew. I don't have a camera that captures RAW so I
don't have any hands on experince but I do love to read about the
DSLR's just incase a pot of gold falls one day from the sky on my
lawn!
I for one am tired of people revealing my ignorance about questions
I never thought about. Ban then from the board would be my first
response.
Here's where I had to read twice to see if you were flamming me or
just joking. he he
1. How do you know that the camera is converting RAW to JPEG? Maybe
it's translating data directly to JPEG?
Hmmmm... translating seems awfully similar to converting. And if
the camera is just translating.... taking the easy way out... why
can't the computer translate instead of the dreaded converting! :)
2. It takes longer to deal with TIF because it takes longer to
write TIF.
Someone mentioned this too, and I understand that saving to Tiff on
a card takes a while because of write speeds, but a coputer should
be able to write a 20mb tiff in only 3 or 4 seconds, no? (I'm
guessing the tiff would be about 20mb but I don't remember... maybe
more)
3. So, how come the damn machine can capture NEF in a shorter time
then TIF, when it takes so long to do it in a computer? Beats me
and that's why the topic starter should be banned!!!! :)
I think its because a) it doesn't do anything to the NEF unlike the
TIFF which is processed and b) the NEF is a smaller file to write
than the TIFF, and this is easily several seconds consider the
write speeds of cards or microdrives. Oh wait... i don't think you
were asking... we're you? Oh, and thanks for the smile at the
end.... I was getting worried! ;)
 
What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.
There are many things going on:
  • On a dual processor machine, RAW converters only use one processor.
  • Cameras have digital signal processors and/or special purpose chips designed to speed up precisely the operations used for dealing with and saving images.
  • The camera buffers things so that you can keep using the camera even as it processes the previous shot.
  • The code for your PC is probably written in C and compiled with little or no optimization, while the code in your camera is probably written direclty in the native machine code of whatever processor is doing the hard work.
  • The camera doesn't need to uncompress the RAW file.
--
Ron Parr
FAQ: http://www.cs.duke.edu/~parr/photography/faq.html
Gallery: http://www.pbase.com/parr/
 
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
The reason the camera is so much faster is because the camera has chips that were designed to only process the RAW information. These special chips have software that was optimized to run efficiently and quickly on these special chips. The conversion software is far more generalized since the software engineer can not so finely tune it to run on the more generalized consumer chips.

It is much the same as in the old days when processors did not have floating point processors built in. They were much slower than processors with a FP unit on board. Or to steal an example from our video friends. It is the difference between a software MPEG codec vs. a hardware codec. (I would be willing to consider buying a hardware RAW converter if Canon ever wanted to sell one.)

So the answer really boils down to the fact that a chip custom built and running custom software is going to be faster and more efficient than a more generalized chip running less optimized software.

--
Valliesto

Want to see my gear list? Check my profile.
 
Hi Valliesto

Your explanation sounds good but I don't buy it. Todays computers are overwhelmingly faster then any specialised chip. Indeed even saying specialised chip your are really saying "ROM." Essentially your arguement is that, all things be equal, a program in ROM will always beat a program in RAM - You would be absolutely right. But "all things are NOT equal." The RAM in this case is much faster then the ROM and the computer is being beat by from 5 to 40 times faster. (The slowest computer being a G3 or Pentium 3).

I want want a better explanation or my money back.

Dave
OK, this may seem silly to some, but I've been plauged by this so
excuse my ignorace. I keep seeing posts over and over again about
how long it takes to convert a RAW image on a computer, something
like 30 seconds or more, and this is with dual processors and the
whole bit.

What I wonder is how does the camera do it so quickly? I'm
assuming that the brains inside the D60 or D100 are nowhere as
quick as that of a dual processor system. So how can your jpeg
file be saved in several seconds, and have it take multiple times
longer to do on the computer. I'm assuming of course that the
camera captures RAW information, which I guess it has to, but then
it has to process it to jpeg given your White Balance, compression,
saturation... etc. settings. So how does the camera go from the
RAW information from the sensor to a jpeg file on your card in
seconds in the camera while it takes well over 30 seconds for the
computer to come up with the same jpeg? Thank-you.

Kiran
The reason the camera is so much faster is because the camera has
chips that were designed to only process the RAW information. These
special chips have software that was optimized to run efficiently
and quickly on these special chips. The conversion software is far
more generalized since the software engineer can not so finely tune
it to run on the more generalized consumer chips.

It is much the same as in the old days when processors did not have
floating point processors built in. They were much slower than
processors with a FP unit on board. Or to steal an example from our
video friends. It is the difference between a software MPEG codec
vs. a hardware codec. (I would be willing to consider buying a
hardware RAW converter if Canon ever wanted to sell one.)

So the answer really boils down to the fact that a chip custom
built and running custom software is going to be faster and more
efficient than a more generalized chip running less optimized
software.

--
Valliesto

Want to see my gear list? Check my profile.
 
Hi Valliesto

Your explanation sounds good but I don't buy it. Todays computers
are overwhelmingly faster then any specialised chip.
This is just false - and obviously so.

Why do you think you have dedicated 3D graphics chip in your computer?
Indeed even
saying specialised chip your are really saying "ROM." Essentially
your arguement is that, all things be equal, a program in ROM will
always beat a program in RAM
No. That's not his argument at all.

It seems that the problem is that you need to learn more about computer hardware.

--
Ron Parr
FAQ: http://www.cs.duke.edu/~parr/photography/faq.html
Gallery: http://www.pbase.com/parr/
 
Hi Kalvin
It is a special digital circuitry designed to perform JPEG
processing in a fast way.
The image data leaves the CCD sensor and is them written to memory (yes, yes, I'm leaving out a few steps). Ok, it writes JPEG, or TIF, or NEF

My computer reads the JPEG and TIF at about twice the speed the camera uses to write the file. My computer reads the NEF data about twenty times slower. (450 Mhz, G4)

Some users with much faster machines read the NEF data about 5 or 6 times slower.

Are the import NEF drivers that bad? Certainly it would appear that Windows machines are faster then Mac and my machine is relatively slow. But even so...

I think there's more to it, Ron Parr not withstanding.

Dave
 

Keyboard shortcuts

Back
Top