Lightroom 4 corrupting RAW files / 'Unexpected end-of-file' error'

[PaultheGCnut]

When the bit flips in RAM it auto copies due to the sudden change...at least thats what I have been taught in school

 

[PaultheGCnut]

I have had the same error. Windows 7, 8 GB then 16GB RAM (tested with multiple system stress test, RAM copy tests, power supply test. Multiple re-importing testing, refresh cache to stress the error on file out. One test was interesting...taking a corrupt file and finding a copy from a CD (not at all writable) and then doing 2 rounds of import and refresh...the cd file did not re-corrupt and has not as I have been using the system. I now import to 2 disks and one does not touch LR...Corrupt file gets replaced from external drive to box and then imported into LR from internal disk.

 

[PaultheGCnut]

I know from a tester that his opinion is that Adobe has gotten sloppy writing code and memory issues are popping up all over newer versions. Too many teams where an orange in one group is not the same object as an orange in another code group.

 

[PaultheGCnut]

An interesting point...sometimes when I've been testing why I get the same corruption type of pattern as in this thread...I would get a current time stamp in the file rename I use to make each file unique (YYYYMMDD_HHMMSS_Filename). So LR has been changing the time on its own while doing a large import (not every time but 15 out of 100 or more)

I too am on the edge of quiting LR and hope to learn more on this post!!!!

 

[PaultheGCnut]

Alex, but what if you were running as the admin on Win7 as your normal login...I know in Linux you'd never do this (so same should apply for Win)...I'm having this error too. I rebuilt the machine after RAM issues and the mosiac patterns are gone. Only NEF and CR2 files corrupt and only in LR...However I had the strangest thing happen the other day. In Photoshop I was showing edits and what you could recover from if you didn't do snapshots or layers. I saved an image as a copy and new name. then closed the file without saving. PS when opening them both had the changes done in PS so the orginal file was also changed by PS not just the copy.

 

[alexsid]

PaultheGCnut wrote:

I know from a tester that his opinion is that Adobe has gotten sloppy writing code and memory issues are popping up all over newer versions. Too many teams where an orange in one group is not the same object as an orange in another code group.

Yes, sloppy coding can create bugs easily. But assuming that RAW files are opened as read-only (if Adobe opens them R/W, this would be unbelievably sloppy!), a file cannot be corrupted by the application - no matter how buggy it is. The corruption can be created only by OS bugs or hardware failure for files opened R/O.

Alex

 

[alexsid]

PaultheGCnut wrote:

Alex, but what if you were running as the admin on Win7 as your normal login...I know in Linux you'd never do this (so same should apply for Win)...I'm having this error too. I rebuilt the machine after RAM issues and the mosiac patterns are gone. Only NEF and CR2 files corrupt and only in LR...However I had the strangest thing happen the other day. In Photoshop I was showing edits and what you could recover from if you didn't do snapshots or layers. I saved an image as a copy and new name. then closed the file without saving. PS when opening them both had the changes done in PS so the orginal file was also changed by PS not just the copy.

If you open a file with RD_ONLY flag, even a privileged user ('root' on Linux or Administrator on Windows) cannot write to it - this is a requirement of POSIX. True, a privileged user can bypass this using some special tricks - but there is no way even for a sloppy programmer to do this accidentally.

If you have rebuilt the machine (reinstalling Windows?), then the problem can be in a buggy device driver (not necessarily a disk-driver - as most drivers run in kernel mode, they can corrupt kernel structures). Unfortunately, it is extremely difficult to troubleshoot problems like that.

Alex

 

[PaultheGCnut]

Thanks Alex,


From adobe they mentioned Nvidea Driver issues in LR3 I believe. I have one of those and that's the only thing not changed. I suspect that conflict #1, Bad CPU #2, Heat/Brown-Outs in Power from Condo wiring/power company.

I had re-installed Win7 twice, had Norton's Software suite and it would show minor/mod registry errors every time it was restarted...thought it was unrelated but after reading these posts, it is something not to overlook.

Sloppy programming would have to be so bad that it would act like a worm/trojan and over ride at the assembly language level by overwriting into the protection bit fields. Pretty unlikely to happen accidentally, not getting hit by lightning unlikely, but wining back to back lottos.


When I get settled into our new place, I plan on using a APC power bat, re-install OS, remove many cards. What would you recommend for testing the re-build like stress tests, etc. I wonder if its a bad CPU (sent in to Intel, seemed returned w.out replacement).

Tried Read-Only on Disk permission, can't tell if that would work because the corruptions pop up in LR each time I let the system open up all the images for 1 year (10K-78K). Corruptions could be from long before.

I have hundreds of these files, some are from the original sdhc card but not most.

 

[alexsid]

PaultheGCnut wrote:

What would you recommend for testing the re-build like stress tests, etc. I wonder if its a bad CPU (sent in to Intel, seemed returned w.out replacement).

From my experience, the problems of this type are usually due to a bad motherboard. For example, this can be a bad contact or a unstable capacitor. In cases like that, it is extremely difficult to find what is wrong. I remember having intermittent problems with disk errors occurring once in 2-3 months. Finally I have replaced the motherboard and they have disappeared completely.

When system is loaded heavily, the temperature of CPU and at motherboard components goes up. Then the load goes down and temperature goes down too. This leads to expansion/contraction and as a result bad contacts can manifest themselves.

I am not aware of any reliable test for such cases. I suggest the following:

• start a test writing/reading to disk with low/moderate load
• while disk test is running, start/stop another test several times, this test should load CPU/memory heavily (IntelBurnTest should work well)

The idea is to make temperature/load change several times. But unfortunately this still does not guarantee a reliable problem detection.

Finally, check whether there are BIOS updates for your motherboard - sometimes they really help.

Regards,

Alex

 

[PaultheGCnut]

I don't kn ow if this piece of info leads more light or not.

After initial RAM corruption and switching out all but CPU, I used WIN 7 OS to copy paste and I would have new ones from only import w/ out copy. Some were very new and imported w/out error message from imports. I finally figured out that Adobe considered those corrupted to uselessness... What I don't get is why there is no error correction. I've lost partition tables and formatted drive (by accident) and still recovered RAW files in a recovery program that was at the bit level.

I'm going to run diagnostics, remove many things including switching Video Card to different brand. In mean time I'm copying to 2 disks...can run chksum to see if they match, which they should but an interment bit flip would be in different places in file, right?

 

[lilhatch]

I am a new member to the forum but have been reading and using it for a while now. I was having the same issues with my late 2011 Macbook Pro with LR4 end-of-file-errors and corrupting NEF files from a D7000.

It started right after my upgrade from Lion to Mountain Lion. My local apple dealer said the install most likely corupted the RAM. My mac was also constantly shutting down, bogging down and crashing apps at random.

Everything added up to bad RAM...I replaced the 8GB of RAM which was fine for 1 1/2 years. I turned the macbook back on, confirmed the ram install....checked LR and the same files that were corrupted before are no longer corrupted.

This may not be a silver bullet for everyone but it seems to have worked for me!

 

[mgco]

Google brought me to this thread, sadly due to the fact that I experienced this today, with at least nine (9) RAW files. I copied the files from card to external drive "Watched" folder along with other JPEGs of the shoot I recently had. The error "Unexpected end-of-file" displayed after the auto-import was done, 2 of the 11 NEFs were ingested without issues.

Also tried converting the NEFs with DNG Converter (post auto-import), but I guess LR had already damaged or tainted the NEFs. Photoshop couldn't open them as well (same error message).

• SW: LR5.0 CR8.1 Win7_32b HW: Core2Duo 2GB RAM, N_D7k

Current solution implemented is to save the NEF file as a TIFF using irfanview, where the TIFF is then re-ingested into LR. Anyone have an explanation to this mystery or perhaps a better workaround?

Never had this problem using LR3 or LR4

 

[Simon Garrett]

mgco wrote:

I copied the files from card to external drive "Watched" folder along with other JPEGs of the shoot I recently had. The error "Unexpected end-of-file" displayed after the auto-import was done, 2 of the 11 NEFs were ingested without issues.

How did you copy files from the card to the 'external drive "Watched" folder'?

 

[Oscar90]

I have had this exact experience, and was not so lucky as to still have my files on the card. I actually first had this issue using Capture One Pro 7 and stopped using it as a result. Lightroom has worked well for several months using the d800, until about 2 weeks ago when almost every shoot I upload has corrupted files.

I'll be trying to find solutions/ answers for this issue on dpreview but if anyone can help by pointing me to the right explanation, that would be much appreciated.

 

[Marcin 3M]

I'd suggest You to check:

- USB infrastructure (cables, hubs, card readers and devices like HDD),

- antivirus software (You may exclude Your RAW files from being scanned).

I do experience some problems with both downloading the files and processing them (C1Pro7 fails with huge batches).

 

[Gabe GDL]

Hey guys, I have the same problem but its not with LR4. Im using LR5 and the same problem with "An unexpected end-of-file occurred." I don't think it is the software but as one person posted it may be with the hardware. And I think it may be the SD card that is breaking down and I do have a good SD card Extreme 32 300x at 45mb/s. But I have been using this card for quite a while shooting raw images way before LR came out. For some reason new high end cameras go to SD cards just for saving space. Problem is these SD cards are not so durable or reliable in the long run as my old compact flash I use in my D300. And still today my D300 still kicks with not problem with transfer. I have used SD card to transfer files. That was a no no. Images got corrupted for some reason so I use external drives. With today's cameras that have extremely high resolution and transferring a raw image into a SD card, there is where your bottle neck is, at a fast rate will cause a problem if you are shooting fast. Moving files at a fast rate causes heat and heat breaks down components and components eventually fail. You can get the fastest SD card on the planet but you still have to get the image on to the card at a faster rate. That's why buffers are in place to handle the image but the SD card must handle the transfer and storage without being corrupted. I have never had these problems before until I moved into higher resolution cameras. Its too bad I lost are really good image and I'm not happy about that. So my only resolve is to get a brand new SD cards or get a few. Another option is to set you camera for mirror backup slot 2 if you have two SD slots or a CF slots. But remember when you are shooting raw and saving to both cards the buffer and writing to card processing is slow and you are using up a lot of your battery. Sometime reinventing the wheel will get you a flat tire. Hope this helps.

 

[SpaceDoc]

interesting. but: if I open the same files with another Mac they should be non-corrupted as well, shouldn't they?

unfortunately in my case they were still destroyed.

but interesting input. if I ever find time enough I might change my old MacBook (late 2008) back to old RAM, and check if the problem persists.

SpaceDoc
.

 

[SandyKay]

Since updating to LR 5.4 I have the exact same trouble. Unexpected end-of-file error. Files are fine after 2nd download. DON'T clear your card until you verify that your photos are safe!! This is alarming.

I am also getting the color stripes intermittently moving from Library to Develope and back. Sometimes they correct themselves sometimes not. Stripes also appear when saving from am add-on program.

Any help would be greatly appreciated.

 

[SpaceDoc]

sorry for late reply! I only saw this today!

even worse: when I had that problem the files were corrupted even later. and even multiple times.

therefore: keep a 2nd copy of the files which you *never* open to edit. in case of file corruption you have to copy them over the corrupted file.

SpaceDoc

.

Since updating to LR 5.4 I have the exact same trouble. Unexpected end-of-file error. Files are fine after 2nd download. DON'T clear your card until you verify that your photos are safe!! This is alarming.

I am also getting the color stripes intermittently moving from Library to Develope and back. Sometimes they correct themselves sometimes not. Stripes also appear when saving from am add-on program.

Any help would be greatly appreciated.

 

[rondhamalam]

Your description told me that you have hardware problem. You Definitely Need to Replace your Hard Drive. Suggestion to do it now before it damages all the contents.

Other possibility of course computer virus that screwed up the low level computer codes.