Silly RAW conversion question

David Barkin said:

You say I haven’t understood your posts? Well let’s see:

Click to expand...

A pretty longwinded excursion with the single point being that I don't know the details between the G3 and the G4. Well, that's true - I haven't examined the detailed spec sheets, and I haven't done any assembly level programming on either.

However, I know what has been added. Yes, the vector engine has some capacilities that are DSP like. This is nothing new, or groundbreaking, or even major. All modern CPU's have a mishmash of CISC and RISC architectural advantages built in, with several DSP inspired parts; there is less difference between the various CPU types than ever. Adding a new pipe with extra calculation functionality is the standard incremental improvement that all chip makers do now.

This does not mean the CPU will suddenly perform like a DSP; it takes more than a pipe engineered the same way to do it (comparison: if you add some nifty suspension details to a family car that were originally developed for a formula one car, will that increase the family cars performance? Yes, but only marginally).

In all honesty, I thought the G4 had more improvements than it does over the G3. From what it seems it won't even gain as much general purpose speed as I was led to believe by the initial spec overview.

The end result is, for some tasks there will be a rather major speed increase with minor changes (basically a recompile or relink - static or dynamic - with a new library). For other software even careful hand engineering will yield very limited results.

You consistently claim that simply recompiling for the G4 will magically improve things. Your basis for this is shady, mostly putting words in other people's mouth. In practice, maybe a 20-30% increase may be possible by adapting the same algorithms to use the improved G4 engine; probably less unless the algorithm can inherently take advantage of the architecture.

What really would make a difference is changing the algorithms and optimizing internal data flow in the software to accomodate the available capabilities. This is what has been done for the in-camera software, to a rather extreme degree.

David Barkin said:

For that matter when the old PPC chip came out it needed software
written especially for it. There was only a minimal speed increase
until this occured.

Click to expand...

This was due to the fact that the old software was for the 68k and was being run in a 68k emulator. A completely different problem both in scope and scale.

David Barkin said:

Well actually it proves that you don't know much about the
difference between the G3 and the G4 nor for that matter about what
is being incorporated in them.

Click to expand...

You still don't understand what you're talking about.

--
Jesper

Jesper

I quoted you saying that at most there would be a speed-up of a "few percentage points." Now you are saying 20 - 30 percent. I do believe there is a big difference. Now I quoted people and when I quoted them I said they mostly agreed with you. (Dare I suggest you re-read my post?)

But a 20 - 30 percent increase, plus software that takes advantage of the chip would make shooting in NEF practical for me without a new computer.

Since you're incapable of admiting any error at any time, I'll keep that in mind.

Dave

TheSwede said:

David Barkin said:

You say I haven’t understood your posts? Well let’s see:

Click to expand...

A pretty longwinded excursion with the single point being that I
don't know the details between the G3 and the G4. Well, that's true

• I haven't examined the detailed spec sheets, and I haven't done

any assembly level programming on either.

However, I know what has been added. Yes, the vector engine has
some capacilities that are DSP like. This is nothing new, or
groundbreaking, or even major. All modern CPU's have a mishmash of
CISC and RISC architectural advantages built in, with several DSP
inspired parts; there is less difference between the various CPU
types than ever. Adding a new pipe with extra calculation
functionality is the standard incremental improvement that all chip
makers do now.

This does not mean the CPU will suddenly perform like a DSP; it
takes more than a pipe engineered the same way to do it
(comparison: if you add some nifty suspension details to a family
car that were originally developed for a formula one car, will that
increase the family cars performance? Yes, but only marginally).

In all honesty, I thought the G4 had more improvements than it does
over the G3. From what it seems it won't even gain as much general
purpose speed as I was led to believe by the initial spec overview.

The end result is, for some tasks there will be a rather major
speed increase with minor changes (basically a recompile or relink

• static or dynamic - with a new library). For other software even

careful hand engineering will yield very limited results.

You consistently claim that simply recompiling for the G4 will
magically improve things. Your basis for this is shady, mostly
putting words in other people's mouth. In practice, maybe a 20-30%
increase may be possible by adapting the same algorithms to use the
improved G4 engine; probably less unless the algorithm can
inherently take advantage of the architecture.

What really would make a difference is changing the algorithms and
optimizing internal data flow in the software to accomodate the
available capabilities. This is what has been done for the
in-camera software, to a rather extreme degree.

theswede #100093 said:

For that matter when the old PPC chip came out it needed software
written especially for it. There was only a minimal speed increase
until this occured.

Click to expand...

This was due to the fact that the old software was for the 68k and
was being run in a 68k emulator. A completely different problem
both in scope and scale.

theswede #100093 said:

Well actually it proves that you don't know much about the
difference between the G3 and the G4 nor for that matter about what
is being incorporated in them.

Click to expand...

You still don't understand what you're talking about.

--
Jesper

Click to expand...

Hi Mike

This was an informative post on transfering data. Most of it was carefully written for the layperson. I've been following the byplay between Ed and Ron and couldn't follow a thing (no criticism intended).

However the original question has nothing to do with transfering data from the memory chip (CF or MD) but rather why it takes so long for the RAW format to be loaded into an image processor.

For example, my machine can load a TIF or JPEG in a few seconds, that same machine takes almost a minute loading in a RAW file.

So we notice that the camera writes a RAW file as fast or faster then JPEG yet our image processing software takes much longer to load the file.

This is the original question.

Ron and Ed seem to have come to some agreement on this issue.

Although I wish they would give their answer in lay terms even if the discussion must be done in technalise.

Dave

David Barkin said:

Jesper

I quoted you saying that at most there would be a speed-up of a
"few percentage points."

Click to expand...

Yes, that is the likely increase.

David Barkin said:

Now you are saying 20 - 30 percent.

Click to expand...

Yes, that is the likely increase if the algorithm is also adapted; a simple recompile to make use of the new chip will not yield that much increase.

David Barkin said:

I do believe there is a big difference.

Click to expand...

The two different scenarios have different outcomes, yes.

David Barkin said:

Now I quoted people and when I
quoted them I said they mostly agreed with you. (Dare I suggest you
re-read my post?)

Click to expand...

I read it, and I saw, but I don't see the "mostly". None of them claimed miraculous effects from incremental design improvements in general purpose CPU's; nor did I. Nor do I discount the possibility that the improvements will help, probably significantly, if the software is redesigned to take advantage of it (not just retargeted for the new CPU, that will seldom yield much improvement if the algorithm is shoddy - which it provably is if a gigahertz CPU will take 30 seconds to convert one file).

David Barkin said:

But a 20 - 30 percent increase, plus software that takes advantage
of the chip would make shooting in NEF practical for me without a
new computer.

Click to expand...

The 20-30% increase would be from software taking advantage of the chip - providing it does so reasonably intelligently. It would not be in addition - it is the increase. And you won't even get that increase without improvements in addition to retargeting.

I suspect that we may see a big improvement in Bibble once they retarget it, because they actively redevelop and they are likely to adapt algorithms to make use of the new engine. This will yield a good improvement, but the manufacturer provided software is unlikely to ever yield more than small, incremental improvements. However, I'd be positively surprised if it was more than a 30% improvement even with that. In all honesty, I'd be positively surprised by 20% too.

David Barkin said:

Since you're incapable of admiting any error at any time, I'll keep
that in mind.

Click to expand...

It's one of my major faults, to not back down when I have a clue what I'm talking about.

--
Jesper

Hi,

I´m using NC3 to open nefs, jpegs and tiffs and the time to open corresponds to the file-size, i.e. jpegs open faster then nefs which open faster than tiffs which in return are larger than nefs...

Sorry, couldn´t find in your (many) posts which programm you´re using, but I do not experience your problem.

Anyway, I think with a new digital camera we should get used to getting a new computer as well...

Alvaro

David Barkin said:

For example, my machine can load a TIF or JPEG in a few seconds,
that same machine takes almost a minute loading in a RAW file.

So we notice that the camera writes a RAW file as fast or faster
then JPEG yet our image processing software takes much longer to
load the file.

This is the original question.

Dave

Click to expand...

Hi Alvaro

As far as I know you are the only person to post such results. Even the people who can open up NEF's quite quickly state that it takes longer then the other files.

However, maybe they haven't posted yet?

Dave

alvarodigit said:

I´m using NC3 to open nefs, jpegs and tiffs and the time to open
corresponds to the file-size, i.e. jpegs open faster then nefs
which open faster than tiffs which in return are larger than nefs...
Sorry, couldn´t find in your (many) posts which programm you´re
using, but I do not experience your problem.
Anyway, I think with a new digital camera we should get used to
getting a new computer as well...

Alvaro

David Barkin said:

For example, my machine can load a TIF or JPEG in a few seconds,
that same machine takes almost a minute loading in a RAW file.

So we notice that the camera writes a RAW file as fast or faster
then JPEG yet our image processing software takes much longer to
load the file.

This is the original question.

Dave

Click to expand...

Click to expand...

Hi Dave,

which program are you using?

Alvaro

David Barkin said:

As far as I know you are the only person to post such results. Even
the people who can open up NEF's quite quickly state that it takes
longer then the other files.

However, maybe they haven't posted yet?

Dave

alvarodigit said:

I´m using NC3 to open nefs, jpegs and tiffs and the time to open
corresponds to the file-size, i.e. jpegs open faster then nefs
which open faster than tiffs which in return are larger than nefs...
Sorry, couldn´t find in your (many) posts which programm you´re
using, but I do not experience your problem.
Anyway, I think with a new digital camera we should get used to
getting a new computer as well...

Alvaro

David Barkin said:

For example, my machine can load a TIF or JPEG in a few seconds,
that same machine takes almost a minute loading in a RAW file.

So we notice that the camera writes a RAW file as fast or faster
then JPEG yet our image processing software takes much longer to
load the file.

This is the original question.

Dave

Click to expand...

Click to expand...

Click to expand...

Hi Alvaro

I'm using Photoshop with the Bibble import driver. Sometimes I use Bibble directly. Can't see it makes much difference in speed either way.

My machine is a 450 Mhz Mac G4 running System 9.1. It will load Jpeg's and Tiff's in a few seconds.

NEF takes a bit short of one minute. If I use Nikon View it takes two minutes. The import driver for Nikon View and Capture are the same.

I would be curious to know if anyone else gets the same results as you. Windows machines, in general do better then Mac's, I presume because of better software. For that matter people with the same machine as I, using OS 10, are able to load a NEF in 35 seconds. However, at least so far, everyone else has said that loading NEF takes longer, no matter how fast the NEF files actually load.

Dave

alvarodigit said:

which program are you using?

Alvaro

David Barkin said:

Hi Alvaro

As far as I know you are the only person to post such results. Even
the people who can open up NEF's quite quickly state that it takes
longer then the other files.

However, maybe they haven't posted yet?

Dave

Click to expand...

Click to expand...

Hi Jesper

TheSwede said:

David Barkin said:

Jesper

I quoted you saying that at most there would be a speed-up of a
"few percentage points."

Click to expand...

Yes, that is the likely increase.

theswede #100093 said:

Now you are saying 20 - 30 percent.

Click to expand...

Yes, that is the likely increase if the algorithm is also adapted;
a simple recompile to make use of the new chip will not yield that
much increase.

Click to expand...

My goodness my apolgies. I didn't see the above explanation in your posts. I also would like to go back at this point and add some additional information to mine.

theswede #100093 said:

theswede #100093 said:

Now I quoted people and when I
quoted them I said they mostly agreed with you. (Dare I suggest you
re-read my post?)

Click to expand...

I read it, and I saw, but I don't see the "mostly". None of them
claimed miraculous effects from incremental design improvements in
general purpose CPU's; nor did I. Nor do I discount the possibility
that the improvements will help, probably significantly, if the
software is redesigned to take advantage of it (not just retargeted
for the new CPU, that will seldom yield much improvement if the
algorithm is shoddy - which it provably is if a gigahertz CPU will
take 30 seconds to convert one file).

Click to expand...

Once again, you are so right. I actually said "more or less" not mostly. Once again my apologies - More or less clearly has a different meaning then Mostly although what that difference is escapes me.

Ed postulated that I might live to see the ability to load NEF at about 12 seconds with my existing set-up. By no means was he making a definite claim, he was just speculating. Still if this speculation is true then that would indeed be a "miraculous effect."

I want to be careful not to rope Ed into argueing with you. This speculation was apparently off the top of his head. Still apparently you didn't read it.

And I completely missed your comments about the potential speed increase. I was under the impression that you said there would be little speed increase with any scenario.

theswede #100093 said:

I suspect that we may see a big improvement in Bibble once they
retarget it, because they actively redevelop and they are likely to
adapt algorithms to make use of the new engine. This will yield a
good improvement, but the manufacturer provided software is
unlikely to ever yield more than small, incremental improvements.
However, I'd be positively surprised if it was more than a 30%
improvement even with that. In all honesty, I'd be positively
surprised by 20% too.

theswede #100093 said:

Since you're incapable of admiting any error at any time, I'll keep
that in mind.

Click to expand...

It's one of my major faults, to not back down when I have a clue
what I'm talking about.

Click to expand...

Yes indeed, invisible ink is a godsend. Especially so on a forum. I'll post again after I make some additional changes in my earlier posts.

Dave

Hi David,

I was only referring to how long it took to open files with NC3.

Alvaro

David Barkin said:

I'm using Photoshop with the Bibble import driver. Sometimes I use
Bibble directly. Can't see it makes much difference in speed either
way.

My machine is a 450 Mhz Mac G4 running System 9.1. It will load
Jpeg's and Tiff's in a few seconds.

NEF takes a bit short of one minute. If I use Nikon View it takes
two minutes. The import driver for Nikon View and Capture are the
same.

I would be curious to know if anyone else gets the same results as
you. Windows machines, in general do better then Mac's, I presume
because of better software. For that matter people with the same
machine as I, using OS 10, are able to load a NEF in 35 seconds.
However, at least so far, everyone else has said that loading NEF
takes longer, no matter how fast the NEF files actually load.

Dave

alvarodigit said:

which program are you using?

Alvaro

David Barkin said:

Hi Alvaro

As far as I know you are the only person to post such results. Even
the people who can open up NEF's quite quickly state that it takes
longer then the other files.

However, maybe they haven't posted yet?

Dave

Click to expand...

Click to expand...

Click to expand...

You didn't see the explanations in depth because each and every thing was taken in its own context. Sure, you can pull a lot of my statements out of their context, and compare them with other statements made in different contexts and build some pretty interesting stories.

This has been, as far as I can tell, an evolving discussion (disagreement?) where I've been pulling stuff off the top of my head. It's the nature of these mediums. If you want to make a dissertation about my inconsistency, go ahead; I'm not claiming I did the best job trying to explain where I'm coming from, especially considering I popped these messages in during breaks from work.

If you want to know why things are the way they are instead of insisting that correct explanations must be wrong because you don't buy it we might get somewhere. To me it doesn't matter; you seem to be all about proving that all I'm after is being right no matter the cost.

Fine. I want to be right no matter the cost. Happy? If you want to find out how something works, ask someone else.

--
Jesper

Hi Alvaro

2 minutes. I've never used the import driver to open up anything but NEFS. If you're saying it takes even longer to open up TIF or JPEG in that program, rest assured I'll take your recomendation and never use it.

Dave

alvarodigit said:

Hi David,

I was only referring to how long it took to open files with NC3.

Alvaro

Click to expand...

Rest assured Jesper that as hard as you beg, I will never teach you anything about mushroom hunting, heating installation or the fine art of training a Kuvasz.

Go ahead beg, I wont tell you a thing.

And you're incorrect I NEVER said "I don't buy it." Read my mind I meant to say something completely different, I'll add it in later. Indeed, I was enjoying this thread until you PROVED me wrong.

Dave

TheSwede said:

You didn't see the explanations in depth because each and every
thing was taken in its own context. Sure, you can pull a lot of my
statements out of their context, and compare them with other
statements made in different contexts and build some pretty
interesting stories.

This has been, as far as I can tell, an evolving discussion
(disagreement?) where I've been pulling stuff off the top of my
head. It's the nature of these mediums. If you want to make a
dissertation about my inconsistency, go ahead; I'm not claiming I
did the best job trying to explain where I'm coming from,
especially considering I popped these messages in during breaks
from work.

If you want to know why things are the way they are instead of
insisting that correct explanations must be wrong because you don't
buy it we might get somewhere. To me it doesn't matter; you seem to
be all about proving that all I'm after is being right no matter
the cost.

Fine. I want to be right no matter the cost. Happy? If you want to
find out how something works, ask someone else.

--
Jesper

Click to expand...

Hi David,

please read what I wrote! In NC3 the smaller jpeg opens faster than the bigger Nef. The even bigger tif is taking even longer.
All 3 fast, way under 1 minute.

Try it, it´s 30 days free on Nikon USA site, then you will know for your machine.

Alvaro

David Barkin said:

2 minutes. I've never used the import driver to open up anything
but NEFS. If you're saying it takes even longer to open up TIF or
JPEG in that program, rest assured I'll take your recomendation and
never use it.

Dave

alvarodigit said:

Hi David,

I was only referring to how long it took to open files with NC3.

Alvaro

Click to expand...

Click to expand...

Hi Alvaro

I appreciate the sentiments of your post, but I think you missed the point of my comment. Nikon Capture has an import driver for NEF, this is the same driver as is used in Nikon View. I did indeed use Nikon's driver, latest version. My results are based on actually using the driver.

If what you say is true the I should expect LONGER times for loading TIF then for loading NEF's.

I assume you have a very fast machine. Let me then ask you if your other image processing software loads TIF faster then NC3 loads TIF?

Dave

alvarodigit said:

please read what I wrote! In NC3 the smaller jpeg opens faster than
the bigger Nef. The even bigger tif is taking even longer.
All 3 fast, way under 1 minute.
Try it, it´s 30 days free on Nikon USA site, then you will know for
your machine.

Alvaro

David Barkin said:

2 minutes. I've never used the import driver to open up anything
but NEFS. If you're saying it takes even longer to open up TIF or
JPEG in that program, rest assured I'll take your recomendation and
never use it.

Dave

alvarodigit said:

Hi David,

I was only referring to how long it took to open files with NC3.

Alvaro

Click to expand...

Click to expand...

Click to expand...

it doesn´t load it (nef) at all (yet)...

While saving for the next expense I can think if it is a good lense or photoshop.

I have enough to learn about curves, histogramms etc. so for that NC3 is good for me so far.

If you open a Nef in Photoshop, doés it stay nef or is the file converted? Conversion to tif or photoshopformat might take the extra-time, while jpeg or tif need no conversion.

Alvaro

David Barkin said:

I appreciate the sentiments of your post, but I think you missed
the point of my comment. Nikon Capture has an import driver for
NEF, this is the same driver as is used in Nikon View. I did indeed
use Nikon's driver, latest version. My results are based on
actually using the driver.

If what you say is true the I should expect LONGER times for
loading TIF then for loading NEF's.

I assume you have a very fast machine. Let me then ask you if your
other image processing software loads TIF faster then NC3 loads TIF?

Dave

alvarodigit said:

please read what I wrote! In NC3 the smaller jpeg opens faster than
the bigger Nef. The even bigger tif is taking even longer.
All 3 fast, way under 1 minute.
Try it, it´s 30 days free on Nikon USA site, then you will know for
your machine.

Alvaro

David Barkin said:

2 minutes. I've never used the import driver to open up anything
but NEFS. If you're saying it takes even longer to open up TIF or
JPEG in that program, rest assured I'll take your recomendation and
never use it.

Dave

alvarodigit said:

Hi David,

I was only referring to how long it took to open files with NC3.

Alvaro

Click to expand...

Click to expand...

Click to expand...

Click to expand...

Hi Alvaro

No image processing program will load NEF without outside help. I realise that Nikon Capture does, and that program has some powerful features, but as far as I know it's not considered a full sized Image processing program.

That's just fine. 95 percent of what Image processing programs do, can no doubt be performed in NC3.

Other programs are dependant on import drivers. Aside from Nikon there are a handful of other program makers who make such import drivers for NEF. Adobe, the makers of Photoshop provide no such driver.

When Photoshop loads in a file it becomes simply an image. It doesn't really have a format unless you count the internal Photoshop display as a format. It only becomes a format when you save the image. I know of no program that can save as NEF. Perhaps Nikon Capture can save as NEF?

So ANY file loaded into Photoshop is done via an external driver. Once loaded you can save it in the same format or another, once again with the exception of NEF (RAW).

So to summarise this post, since I use Photoshop (and some other programs) NEF is the only format that takes a long time to load on my machine. I would indeed consider another computer if other aspects of my work were slow. Such is not the case. Even complex modeling/rendering software works fairly quickly on my machine, certainly as fast as I work on my machine.

So this thread has been useful in understanding where the NEF import drivers are and what is the potential for their development.

Dave

alvarodigit said:

it doesn´t load it (nef) at all (yet)...
While saving for the next expense I can think if it is a good lense
or photoshop.
I have enough to learn about curves, histogramms etc. so for that
NC3 is good for me so far.
If you open a Nef in Photoshop, doés it stay nef or is the file
converted? Conversion to tif or photoshopformat might take the
extra-time, while jpeg or tif need no conversion.

Alvaro

Click to expand...

Hi Jesper

You are apparently clueless so after reflection I will make one last post.

In my first post on this topic I suggested that the originator of the topic be banned from the board because she came up with a question that never occured to me.

I then, in that same post, made some facious remarks on what was happening. At that point Valliesto Bailey gave me an explanation.

My reply to him started with the words "I don't buy it."

But what was the last sentence to that same post?

"I want a better explanation or my money back."

Mr. Bailey doers indeed have a sense of humor, so he came right back with a better explanation. If you had read my first post which included such professional computer technical talk such as "beats me." you just might have paused.

Now since neither you or Ron have anything resembling a sense of humor you both leaped in to strike a blow against the ignorant. The difference between you and Ron was that, he at least, was perturbed about why I was so angry. You continue and I might add continue poorly.

So your posts are full of little digs to show how ignorant I am, such as your dumb remark about me using the expression "velocity engine." You just couldn't help yourself - You had to show that I knew nothing and was merely repeating a "hype" when I should be using Altavec Engine. I say dumb becuase all this stuff is hype. When you communicate and ask for an aspirin you don't ask for acetysalicsilic acid.

So you write below about context. Do my remarks also have a context?

Lord help me, I doubt if you'll ever get it.

Last of all, this post is not a flame but an explanation.

Dave

TheSwede said:

You didn't see the explanations in depth because each and every
thing was taken in its own context. Sure, you can pull a lot of my
statements out of their context, and compare them with other
statements made in different contexts and build some pretty
interesting stories.

This has been, as far as I can tell, an evolving discussion
(disagreement?) where I've been pulling stuff off the top of my
head. It's the nature of these mediums. If you want to make a
dissertation about my inconsistency, go ahead; I'm not claiming I
did the best job trying to explain where I'm coming from,
especially considering I popped these messages in during breaks
from work.

If you want to know why things are the way they are instead of
insisting that correct explanations must be wrong because you don't
buy it we might get somewhere. To me it doesn't matter; you seem to
be all about proving that all I'm after is being right no matter
the cost.

Fine. I want to be right no matter the cost. Happy? If you want to
find out how something works, ask someone else.

--
Jesper

Click to expand...

I can't say that you got it either; I was pointing out incorrectness, trying to explain why and how, and found myself in the position of being questioned on elementary technical issues.

As for what you perceived as talking down, I was only using what phrases you were using. I have no idea what the buzzwords used for the new integer engine in the G4 is, and I don't care. All I know is it's an integer engine. I know what it does without memorizing the latest names. It was in no way a dig, and I was ignorant in not understanding you would see it as one.

As for angry, I have to admit I didn't notice you were. I guess my skin is too thick after many years of online debates (and yes, they're usually debates, even if heated).

In summary, I did not intend to attempt to belittle, talk down to or denigrate anyone; I just wanted to point out some rather off the wall factual problems, and things got out of hand.

--
Jesper