InTheMist: Kind of a major screw-up, Apple!
@InTheMist :Things happen. We do try to avoid crashes, we insert additional data sanity checks (those affect speed in a negative way), we try to address the problems in the matter of hours after they are discovered (see, for example, http://www.dpreview.com/forums/post/55797944 ), we employ methods like this: http://en.wikipedia.org/wiki/Fuzz_testing . Still, all possible situations can't be covered. Here is an example http://feedback.photoshop.com/photoshop_family/topics/lightroom-cc-images-got-corrupted-after-doing-meta-data-updateThat is why as a photographer I ask for files that conform to the standards and specifications, including those out of camera files. I think it is in our best interest to have error- and trouble-free files form our cameras, and fully documented, including bugs, too.
@Dr_Jon :Does the problem affect previous Monochrom? Why after I repaired a file by hand it stopped causing problems? OK, maybe I was lucky, and suppose the issue is in something else. The question remains: why Leica deviates from EXIF/DNG standard, recording a tag with zero content length but having 80 in the length field? If one cares about data integrity and preservation, how good as that?
@Mark Alan Thomas :That is not how you interpreted it when you typed "It’s up to the third party to fully support the latest, openly documented, DNG spec." And quite frankly you were right, nothing in Leica statement hints to Leica problem. On the contrary, they are saying "It is anticipated that Apple will have this resolved in the next update for 'Photos'." The issue is this: particular software can be patched, but the malformed files will stay malformed and may continue to cause troubles. I do not see Leica offering a free utility to repair their files, while any developer taking that Leica DNG conform to the "openly documented, DNG spec." may run into the same problem, and who knows what havoc it can create in our image archives. I tried my best to explain what I see as an honest, fair, transparent, and productive approach to the problems like this. Whoever has ears, let them hear.
falconeyes: The real message here is that a crash of the Photos app can corrupt the library.
That should never happen with a well designed app (hint: transactional safety).
I think that's a real advantage with LR: not only did I never see a corrupted library after a LR crash (I've seen quite a few after fresh major releases). But because it is an SQLite database file, it could be repaired using SQLite tools. And then, LR really cares to keep backups of that file (of course, Time Machine does the same).
To understand how serious this is: Just imagine an application crash would leave your file system emptied ...
It is hard to accept Apple released such a poorly designed piece of software. Apple seems to have changed a lot recently.
@Horshack : well, are we saying Leica should continue recording exifs that cause problems, and not openly publishing how to parse their files that contain errors? I do not think so.
joe6pack: Now hackers all over the world is going to exploit this. I suspect this issue is not limited to DNG file. It could be .ARW, .CRW or even .JPG as long as the metadata in question is malformed. Watch out!
Been done https://isc.sans.edu/forums/diary/JPEG+exploit+toolkit+JPEG+Hacktool+GDIScan+Tool+In+search+of+the+Botnet+Lessons+learned/318/
@falconeyes : Actually, I see things like http://lcamtuf.coredump.cx/afl/ more helpful. A file may be intentionally malformed, or malformed as a result of broken transfer, and for other reasons, too. What we expect from manufacturers I guess is not to add to chaos.
@Horshack : yes, very much diplomatically, "It is anticipated that Apple will have this resolved in the next update for 'Photos'.". Not a word Leica themselves will start writing exifs that do not trigger
Tag 0x927c Warning = [minor] Possibly incorrect maker notes offsets (fix by -126?)
Tag 0x0302 Warning: Attempted dump outside data (80 bytes specified, but only 0 available)
@Jono SlackI do not doubt that Apple need to address the problem on their side. All I say I want Leica to act, and not to put all the blame on Apple, like Leica themselves are not doing anything wrong. Leica DNG are showing problems in exif/metadata for quite some time already. In 10 years from now such exifs may cause serious problems. Online sevrice problems and exploits are possible with such malformed exifs right now.
@Horshack : I think I said that couple of times. What I do not understand is why Leica are not fixing their metadata, do not admit their part of the problem, and do not openly publish how they deviate from exif/dng specs.
I tried too yesterday, library was never destroyed. The reason Alex and I are interested in testing such things is because we do not want corrupt metadata to cause LibRaw/FastRawViewer/RawDigger crash. If you run Leica DNG files through "exiftool -v3" you will see 3-4 badly formed tags per file, wrong offsets, wrong lengths, possible reads past file end.
User experience varies with other raw converters too. Here is an example http://feedback.photoshop.com/photoshop_family/topics/lightroom-cc-images-got-corrupted-after-doing-meta-data-update
Said that, I do not like Photos at all.
@Sch64 : I wonder why Leica are not openly recognizing their share of the problem. BTW, errors in metadata format can be used to trigger different problems with online services. Those errors are not as harmless as one might hope.
"NG files are currently incompatible with Apple's 'Photos' App in Mac OS X Yosemite causing the library to crash and potentially lose all existing image files in the Apple Photos library." - says Leica about an Apple product. Maybe Apple statement is in order, and I hope they will not turn it is mud throwing match.
@Mark Alan Thomas : Looks like you are ignoring the fact that Leica is putting blame on Apple and plugging Adobe software; while not recognizing their own format errors. That is not transparency; and that is sounding like Adobe apologists. For the record, I consider Photos to be rather limited raw converter.
@Mark Alan Thomas : You have a confirmed case, or you are relying on Leica's "potentially" in their PR cover-up statement?
@Mark Alan Thomas : Right. What Leica should have done is to be more transparent, to acknowledge their formatting is causing the issue, and make an effort to solve the issue on their end, too.
@Mark Alan Thomas : Data is not the issue. Format is. One can crash Excel, for example, with a poorly formatted xls file.
@BJN : Yes, tests for garbage input data are necessary. But I never saw actual loss of "all existing image files in the Apple Photos library" while testing this.
@InTheMist : incorrect lengths and offsets are somewhat different from recording arbitrary information. Like in tag 0x0302 they state data length is 80, but there is 0 actual data bytes.
@Mark Alan Thomas : so, incorrect maker notes offset Leica are recording in tag 0x927c is in DNG standard?
Leica are recording invalid metadata tags. Of course Apple may have used something like http://lcamtuf.coredump.cx/afl/ to avoid such crashes, but why Leica are doing that on the first place?