XT3 image cut when TimeLapse

Mendimundi said:

Hi, Herbert.

Thanks for all your tips.

I first set all the exposure parameters manually as you, but realized that in the day-night transitions had issues. from night to day the light was overexposed soon and the TL went from good exposure to overexposed before I wanted. I normally only set aperture to automatic and that gives me a smoothest transitions from night to day. I usually use angular lenses for TL so the depth of field changes that the diferentes apertures give are not noticeable. When I want movement objets to change their movement time during the TL I set the Shutter Speed to automatic too. So when there is still light you can still and recognize people but later the Shutter Speed changes to bigger lapses and you see moving objects. I like that. As you say flicker is a price you have to pay.

I Use a 128 gb Sandisk Extreme PRO SDXC II 300mb/s (3) (10) card but will follow you advice "keep my interval time at least around 1 sec longer than my exposure time"

About the intervalometer... Which one do you use?

In those specific shots the info is the same

2" between photos

ISO 160

Shutter Speed 1/8s

Aperture f4

Thanks

M.

Click to expand...

Mendimundi said:

boogisha said:

Mendimundi said:

The photo, while when it is done and you see it in the camera it is seen wrongly. When you pass the photo to a folder in the computer and you open it in preview it is seen wrongly too. If you use this photo to do a Time Lapse, in Final Cut for example, it is seen wrongly too. But, if you open it in Capture One or Photoshop you can see it correctly, just opening it.

Click to expand...

But which exact file are you eventually correctly opening in Capture One or Photoshop - the same JPEG file that is wrong when previewed in-camera and on computer...?

If it`s RAW file instead, then it could really be its embedded JPEG preview is corrupted (as probably original JPEG is, if you saved that one alongside RAW as well), so any application showing you RAW file embedded JPEG preview only will have you see a corrupted (JPEG) image. But once you open RAW inside Capture One or Photoshop (Camera Raw, I would assume), those show you real RAW data (ignoring embedded JPEG preview once RAW is fully loaded), and that one might still be fine.

Click to expand...

Yes, the same exact file that in the camera screen and in the preview once downloaded is wrong opens ok in C1 and Ph. In Time Lapses I only take jpg, so no Raw available to check.

Click to expand...

crowley213 said:

Mendimundi said:

Hi, Herbert.

Thanks for all your tips.

I first set all the exposure parameters manually as you, but realized that in the day-night transitions had issues. from night to day the light was overexposed soon and the TL went from good exposure to overexposed before I wanted. I normally only set aperture to automatic and that gives me a smoothest transitions from night to day. I usually use angular lenses for TL so the depth of field changes that the diferentes apertures give are not noticeable. When I want movement objets to change their movement time during the TL I set the Shutter Speed to automatic too. So when there is still light you can still and recognize people but later the Shutter Speed changes to bigger lapses and you see moving objects. I like that. As you say flicker is a price you have to pay.

I Use a 128 gb Sandisk Extreme PRO SDXC II 300mb/s (3) (10) card but will follow you advice "keep my interval time at least around 1 sec longer than my exposure time"

About the intervalometer... Which one do you use?

In those specific shots the info is the same

2" between photos

ISO 160

Shutter Speed 1/8s

Aperture f4

Thanks

M.

Click to expand...

Hi M.

I use the same cards as you do, just the 64GB versions. They are fast enough to handle really short interval times, nevertheless will also need some time to store the image on the card. Watching carefully you can observe (light indicator below the AF-L button) how long the camera is busy with that process. In my observations I figured out that the stated 1 second should be sufficient to ensure that the camera has enough time to store the data on the card.

The exposure parameters you used, when observing this weird behavior of cutting the image, should not be the reason for this problem, they look pretty normal.

Be aware that flicker can be introduced as well during shooting as in post processing. But with careful handling of the involved processes the effects can at least be minimized.

Shooting day to night time transitions (or vice versa) I normally use a Variable ND filter. Beside shutter speed, aperture and ISO it gives me around 5 additional stops to work with. With sufficient time between the shots, approx. 1.5 secs or more, and the camera mounted on a stable tripod setup, I can change the exposure value of the filter easily without a negative impact (shake, misalignment) on the camera. I normally first work with the filter, means changing the exposure value, before working with the other parameters. It also helps to keep ISO values down.

The intervalometer I use is the LRT Pro Timer 3. Yes, it is definitely an expensive gadget, but it offers quite some helpful features and possibilities.

- Most important, in between the shots all the essential information like histogram and exposure values are displayed and all camera functions are fully accessible.

- Connected to the remote release port (2.5mm) it doesn't occupy the USB C port, means I still can charge my camera with am external power supply during really long shootings.

- You can set intervals in 0.1 sec steps, something that is quite useful in different situations.

- You can do interval ramping / fairing, means change the shooting interval smoothly during the time lapse shooting without interruption.

- It offers shortest possible "camera blocked" times, means you can use really short intervals. You can adjust necessary parameters based on the needs of your specific camera.

- It offers also bulb mode and other options and can finally also act as a simple shutter release for single images.

Finally, I use LRTimelapse for my time lapse editing, for me simply the best solution I found for fast and effective time lapse image editing.

Herbert

Click to expand...

Mendimundi said:

Her you have. It uploads correctly...

Click to expand...

boogisha said:

Mendimundi said:

Her you have. It uploads correctly...

Click to expand...

Thanks, and indeed, I downloaded the photo and tried opening in a few applications, and they all show it correctly (as seen here).

Curious, if you now download this same file and open it again, is it still corrupted? By the way, where do you see the corruption, which application(s) exactly (besides in-camera, from SD card)?

For what that is worth, I even tried saving it to SD card and opening in-camera myself (X-T3 as well, Ver.4.00), and it still looks fine, so I`m wondering if this file is (now?) really "corrected" (or at least not corrupted), or there`s still something with it where certain applications may struggle showing it correctly...?

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

robert1955 said:

boogisha said:

Mendimundi said:

Her you have. It uploads correctly...

Click to expand...

Thanks, and indeed, I downloaded the photo and tried opening in a few applications, and they all show it correctly (as seen here).

Curious, if you now download this same file and open it again, is it still corrupted? By the way, where do you see the corruption, which application(s) exactly (besides in-camera, from SD card)?

For what that is worth, I even tried saving it to SD card and opening in-camera myself (X-T3 as well, Ver.4.00), and it still looks fine, so I`m wondering if this file is (now?) really "corrected" (or at least not corrupted), or there`s still something with it where certain applications may struggle showing it correctly...?

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

That is 16:9 aspect ratio. Could that be a cause of incorrectly writing jpg's?

Click to expand...

boogisha said:

Mendimundi said:

Her you have. It uploads correctly...

Click to expand...

Thanks, and indeed, I downloaded the photo and tried opening in a few applications, and they all show it correctly (as seen here).

Curious, if you now download this same file and open it again, is it still corrupted? By the way, where do you see the corruption, which application(s) exactly (besides in-camera, from SD card)?

For what that is worth, I even tried saving it to SD card and opening in-camera myself (X-T3 as well, Ver.4.00), and it still looks fine, so I`m wondering if this file is (now?) really "corrected" (or at least not corrupted), or there`s still something with it where certain applications may struggle showing it correctly...?

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

boogisha said:

Mendimundi said:

Her you have. It uploads correctly...

Click to expand...

Thanks, and indeed, I downloaded the photo and tried opening in a few applications, and they all show it correctly (as seen here).

Curious, if you now download this same file and open it again, is it still corrupted? By the way, where do you see the corruption, which application(s) exactly (besides in-camera, from SD card)?

For what that is worth, I even tried saving it to SD card and opening in-camera myself (X-T3 as well, Ver.4.00), and it still looks fine, so I`m wondering if this file is (now?) really "corrected" (or at least not corrupted), or there`s still something with it where certain applications may struggle showing it correctly...?

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

Mendimundi said:

boogisha said:

Thanks, and indeed, I downloaded the photo and tried opening in a few applications, and they all show it correctly (as seen here).

Curious, if you now download this same file and open it again, is it still corrupted? By the way, where do you see the corruption, which application(s) exactly (besides in-camera, from SD card)?

For what that is worth, I even tried saving it to SD card and opening in-camera myself (X-T3 as well, Ver.4.00), and it still looks fine, so I`m wondering if this file is (now?) really "corrected" (or at least not corrupted), or there`s still something with it where certain applications may struggle showing it correctly...?

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

Hi

If I download it from here I can see it correctly. The corruption is seen in the camera, also if I download it and open it with the space bar and if I open it with the preview app. Also If I open it with Final Cut Pro. If I upload it to twitter it I seen wrong too.

https://twitter.com/x/status/1358437265316397057

Click to expand...

Mendimundi said:

The size in 16:9 is the resolution I took the photo

Click to expand...

Mendimundi said:

robert1955 said:

boogisha said:

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

That is 16:9 aspect ratio. Could that be a cause of incorrectly writing jpg's?

Click to expand...

Could be but I always do Time Lapses with this aspect ration and only has happened two times. So, I don't know.

Click to expand...

boogisha said:

Mendimundi said:

boogisha said:

Thanks, and indeed, I downloaded the photo and tried opening in a few applications, and they all show it correctly (as seen here).

Curious, if you now download this same file and open it again, is it still corrupted? By the way, where do you see the corruption, which application(s) exactly (besides in-camera, from SD card)?

For what that is worth, I even tried saving it to SD card and opening in-camera myself (X-T3 as well, Ver.4.00), and it still looks fine, so I`m wondering if this file is (now?) really "corrected" (or at least not corrupted), or there`s still something with it where certain applications may struggle showing it correctly...?

By the way, I see image size reported as 6240x3512 - when you see it corrupted, what resolution/size does the offending application report?

Click to expand...

Hi

If I download it from here I can see it correctly. The corruption is seen in the camera, also if I download it and open it with the space bar and if I open it with the preview app. Also If I open it with Final Cut Pro. If I upload it to twitter it I seen wrong too.

https://twitter.com/x/status/1358437265316397057

Click to expand...

Ok, just to confirm - all these places where the file is still corrupt, they work fine with the photo downloaded from here (but just not from original photo downloaded from SD card)?

If the uploaded file here is indeed "fixed" for some reason, then it might be good to try to pack the original photo (showing the issue) into ZIP, RAR or similar, and upload that somewhere - then first try to download it yourself and see if the issue is still there once unpacked. If so, we can check that one, where the issue is still present.

By the way, how do you transfer images from your camera to your computer? Do you take the SD card out of camera and insert it into a card reader, or you connect an USB cable to camera to read from it directly, or maybe using some (phone or computer) app...?

boogisha said:

The size in 16:9 is the resolution I took the photo

Click to expand...

That`s fine, but what I`m interested in is _exact_ resolution being reported inside applications where the (original) file is corrupted - is it exactly 6240x3512 there as well, or maybe it differs in a few pixels, like 6245x3512, or something?

Click to expand...

Mendimundi said:

Hi

I use software 4.0.0 . Twice has happened that after a 1000 photos TL some of them are cut vertically as seen in the attached images (look at the very right of the second. The one that is correct is just the previous one). It corrects itself later, the first time after 300 photos and the second after 100. any ideas? where should I report this bug?

Thanks!

M.

Click to expand...

Joep van Steen said:

Mendimundi said:

Hi

I use software 4.0.0 . Twice has happened that after a 1000 photos TL some of them are cut vertically as seen in the attached images (look at the very right of the second. The one that is correct is just the previous one). It corrects itself later, the first time after 300 photos and the second after 100. any ideas? where should I report this bug?

Thanks!

M.

Click to expand...

I see corruption in the top MCU line. Data is missing, this causes the image to shift to the left, all MCU lines that follow will shift as well as JPEG is coded/decoded per MCU, starting top left working it's way down line by line. Visually it looks like the image was cut but this is a matter of a few missing bytes at the start of the JPEG data.

Maybe the card can't keep up?

Click to expand...

Mendimundi said:

Joep van Steen said:

Mendimundi said:

Hi

I use software 4.0.0 . Twice has happened that after a 1000 photos TL some of them are cut vertically as seen in the attached images (look at the very right of the second. The one that is correct is just the previous one). It corrects itself later, the first time after 300 photos and the second after 100. any ideas? where should I report this bug?

Thanks!

M.

Click to expand...

I see corruption in the top MCU line. Data is missing, this causes the image to shift to the left, all MCU lines that follow will shift as well as JPEG is coded/decoded per MCU, starting top left working it's way down line by line. Visually it looks like the image was cut but this is a matter of a few missing bytes at the start of the JPEG data.

Maybe the card can't keep up?

Click to expand...

Hi, Joep

Thanks for the info. Sorry, I don't know what MCU is. Where do you see it and how do you know there is information missing?

The card is a Sandisk Extreme Pro 128gb 300mb/s SDXC II (3) (10) (almost 2 years of use now). I think it should keep up but I don't know. The problem appears only here, not using burst for example, and only have happened 2 times in the last 20 Time Lapses, not a single time before.

I have checked that this problem happened in 29 continued photos in this TL, later it fixed

Thanks

M.

Click to expand...

Joep van Steen said:

An MCU is a block of pixels, 8 x 8 pixels for example and it's the smallest encodable unit in JPEG data. So JPEG encoder takes these pixels and encoded and compresses them as one unit. When MCU's drop, image data in JPEG tends to shift to left.

This is a totally unrelated problem I purposely removed (corrupt) data, it shows image shift and how it responds to adding data (in your image we see opposite effect): (Open in new TAB, embedded player does not honor time stamp)

So for whatever reason your file is missing some data. We can see it's in first line of MCUs because it's slightly discolored.

So, some data appears missing. Question remains of course, why? This isn't a sensor problem. Maybe some buffer for the encoder not keeping up or something. Or the card buffer. If the card is 2 years old I can't hurt to replace it.

Click to expand...

boogisha said:

Joep van Steen said:

An MCU is a block of pixels, 8 x 8 pixels for example and it's the smallest encodable unit in JPEG data. So JPEG encoder takes these pixels and encoded and compresses them as one unit. When MCU's drop, image data in JPEG tends to shift to left.

This is a totally unrelated problem I purposely removed (corrupt) data, it shows image shift and how it responds to adding data (in your image we see opposite effect): (Open in new TAB, embedded player does not honor time stamp)

So for whatever reason your file is missing some data. We can see it's in first line of MCUs because it's slightly discolored.

So, some data appears missing. Question remains of course, why? This isn't a sensor problem. Maybe some buffer for the encoder not keeping up or something. Or the card buffer. If the card is 2 years old I can't hurt to replace it.

Click to expand...

But isn`t it even more curious how come missing data "reappears"...?

What OP reports is having a corrupted JPEG - both in-camera, and after saving from camera and previewing in some applications. But, some other applications seem to open JPEG just fine - and even uploading the corrupted file here to DPReview seems to "fix" it to a point where downloaded file now works fine where the original was previously "corrupted".

So it looks like data is not really missing, but might be "just" corrupted (misaligned, so to say), where certain applications (DPReview as well?) are able to both fix the corruption for preview, but possibly fix the original file, too.

p.s. I`ve just skimmed through that linked video, and it looks like the guy is "correcting" a photo there as well, where it seemed as a large part of it was missing - but it is not? Or that case is just similar, but not the same as this one here (as some data is really missing there, a gray line being reconstructed in the end, where here everything is fine, eventually)?

Click to expand...

Joep van Steen said:

Ah, I missed that, the JPEG 'magically fixing itself'. Is it possible to see one 'corrupt' and one magically fixed so I can examine them?

Click to expand...

boogisha said:

Mendimundi said:

If I download it from here I can see it correctly. The corruption is seen in the camera, also if I download it and open it with the space bar and if I open it with the preview app. Also If I open it with Final Cut Pro. If I upload it to twitter it I seen wrong too.

https://twitter.com/x/status/1358437265316397057

Click to expand...

Ok, just to confirm - all these places where the file is still corrupt, they work fine with the photo downloaded from here (but just not from original photo downloaded from SD card)?

If the uploaded file here is indeed "fixed" for some reason, then it might be good to try to pack the original photo (showing the issue) into ZIP, RAR or similar, and upload that somewhere - then first try to download it yourself and see if the issue is still there once unpacked. If so, we can check that one, where the issue is still present.

By the way, how do you transfer images from your camera to your computer? Do you take the SD card out of camera and insert it into a card reader, or you connect an USB cable to camera to read from it directly, or maybe using some (phone or computer) app...?

Click to expand...

Joep van Steen said:

Ah, I missed that, the JPEG 'magically fixing itself'. Is it possible to see one 'corrupt' and one magically fixed so I can examine them?

Click to expand...

Mendimundi said:

Her you have. It uploads correctly...

Click to expand...

Joep van Steen said:

boogisha said:

Joep van Steen said:

An MCU is a block of pixels, 8 x 8 pixels for example and it's the smallest encodable unit in JPEG data. So JPEG encoder takes these pixels and encoded and compresses them as one unit. When MCU's drop, image data in JPEG tends to shift to left.

This is a totally unrelated problem I purposely removed (corrupt) data, it shows image shift and how it responds to adding data (in your image we see opposite effect): (Open in new TAB, embedded player does not honor time stamp)

So for whatever reason your file is missing some data. We can see it's in first line of MCUs because it's slightly discolored.

So, some data appears missing. Question remains of course, why? This isn't a sensor problem. Maybe some buffer for the encoder not keeping up or something. Or the card buffer. If the card is 2 years old I can't hurt to replace it.

Click to expand...

But isn`t it even more curious how come missing data "reappears"...?

What OP reports is having a corrupted JPEG - both in-camera, and after saving from camera and previewing in some applications. But, some other applications seem to open JPEG just fine - and even uploading the corrupted file here to DPReview seems to "fix" it to a point where downloaded file now works fine where the original was previously "corrupted".

So it looks like data is not really missing, but might be "just" corrupted (misaligned, so to say), where certain applications (DPReview as well?) are able to both fix the corruption for preview, but possibly fix the original file, too.

p.s. I`ve just skimmed through that linked video, and it looks like the guy is "correcting" a photo there as well, where it seemed as a large part of it was missing - but it is not? Or that case is just similar, but not the same as this one here (as some data is really missing there, a gray line being reconstructed in the end, where here everything is fine, eventually)?

Click to expand...

Ah, I missed that, the JPEG 'magically fixing itself'. Is it possible to see one 'corrupt' and one magically fixed so I can examine them? Perhaps best to ZIP them so we can be sure no in between decoding and encoding is done. What I mean is some website software when you upload a file decodes and encodes the file to save storage space mainly.

No in the video part of the data is corrupt and this corrupt portion is removed from the raw file data. The guy in the video is me using a tool I made for that purpose.

Click to expand...