user controlled D800 hot pixel remapping: details

Started Aug 16, 2013 | Discussions
Bernard Delley Senior Member • Posts: 1,769
user controlled D800 hot pixel remapping: details
17

Good news: I just learned that hot pixels can be removed by a user controlled remapping procedure: just run sensor clean manually twice in immediate succession.

I have not seen a mention of this in the camera manual.

This hint is due to AllOtherNamesTaken http://www.dpreview.com/forums/post/51985241

The procedure remaps hot pixels which are stronger than ~1000 raw camera units at ISO 1600 and 1/5s .

All bright pixels (hot are bright which should not be bright) are automatically, and not under user control, treated by Nikons HPS algorithm: http://www.dpreview.com/forums/post/50874784

Hot pixels are produced at rates of 0.x to tens per year, and especially on intercontinental flights (worth about 100days at sea level) by the ubiquitous highly energetic cosmic rays. See also http://www.dpreview.com/forums/post/37150608

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
Astrophotographer 10 Forum Pro • Posts: 13,907
Re: user controlled D800 hot pixel remapping: details
1

This sort of post is gold.

Thank you for the info.

Greg.

 Astrophotographer 10's gear list:Astrophotographer 10's gear list
Sony a7R III Canon EOS Ra +1 more
Glen78 Senior Member • Posts: 1,411
Re: user controlled D800 hot pixel remapping: details

I can confirm that it works on the D600 also, although apparently not on the D7000. Maybe it is only for the latest generation of cameras.

 Glen78's gear list:Glen78's gear list
Nikon D810 Nikon D5500 Nikon D850 Nikon AF-S DX Nikkor 16-85mm f/3.5-5.6G ED VR Nikon AF-S Nikkor 14-24mm f/2.8G ED +10 more
soloryb Senior Member • Posts: 2,405
Re: user controlled D800 hot pixel remapping: details

Bernard Delley wrote:

Hot pixels are produced at rates of 0.x to tens per year, and especially on intercontinental flights (worth about 100days at sea level) by the ubiquitous highly energetic cosmic rays. See also http://www.dpreview.com/forums/post/37150608

This thing about cosmic rays at high altitude (in air flights) came up a few years ago and I believe it was debunked. That is, it doesn't happen to camera sensors at typical commercial air travel altitudes.

http://www.consumertraveler.com/columns/does-airplane-travel-kill-digital-camera-sensors/

 soloryb's gear list:soloryb's gear list
Fujifilm FinePix X100 Nikon D800E Nikon AF-S Nikkor 24-70mm f/2.8G ED Nikon AF-S Nikkor 70-200mm f/2.8G ED VR II Nikon AF Fisheye-Nikkor 16mm f/2.8D +9 more
rubank Senior Member • Posts: 1,249
Re: user controlled D800 hot pixel remapping: details

Glen78 wrote:

I can confirm that it works on the D600 also, although apparently not on the D7000. Maybe it is only for the latest generation of cameras.

This is an old "trick" that works also on D300. Reported here many years ago.

Actually, the first camera I had that this worked on was a Canon (40D if I recall correctly), so it´s not a Nikon only thing. I think the Canon forum was the origin of this procedure.

j_photo Veteran Member • Posts: 5,641
Re: user controlled D800 hot pixel remapping: details

Awesome tip. Thank you.

 j_photo's gear list:j_photo's gear list
Nikon Df Nikon Z7 Nikon Z6 II
OP Bernard Delley Senior Member • Posts: 1,769
Re: user controlled D800 hot pixel remapping: details

This thing about cosmic rays at high altitude (in air flights) came up a few years ago and I believe it was debunked. That is, it doesn't happen to camera sensors at typical commercial air travel altitudes.

http://www.consumertraveler.com/columns/does-airplane-travel-kill-digital-camera-sensors/

Just take a black frame before an intercontinental flight and after and have a careful "look", as I did before making my introductory statement. (needs raw loss-less  image format). If I dig them out, I can post  my findings for a recent flight forth and back to Japan.

Of course these  effects from radiation are not dramatic, its a few pixels that become sick out of millions. Also for humans its not dramatic, air personnel does not have a noticeably shorter life than the rest of us.

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
NikonMan09
NikonMan09 Contributing Member • Posts: 557
Which manual sensor clean?

Very interesting information. . .

By running sensor clean manually, do you mean running manually the sensor clean that can be set to run when you turn the camera on and off, or the manual sensor clean that lifts the mirror so you can visually examine and blow the sensor?  There are two kinds of sensor cleaning methods that can be run manually, and I would like to clarify which one.

Good work digging this up.

Fred

-- hide signature --

Formerly known as MinoltaMan78. Switched to Nikon in 2009.

 NikonMan09's gear list:NikonMan09's gear list
Nikon D7200 Nikon D7500 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR Nikon AF-S DX Nikkor 16-85mm f/3.5-5.6G ED VR Nikon AF-S DX Nikkor 35mm F1.8G +11 more
OP Bernard Delley Senior Member • Posts: 1,769
Re: Which manual sensor clean?

NikonMan09 wrote:

By running sensor clean manually, do you mean running manually the sensor clean that can be set to run when you turn the camera on and off, or the manual sensor clean that lifts the mirror so you can visually examine and blow the sensor? There are two kinds of sensor cleaning methods that can be run manually, and I would like to clarify which one.

thanks for the question!

The vibration type, built in, dust shake off procedure is meant.  There is a menu option to clean 'now'. This is the menu option to use twice in a row. It takes about 1s to go through the first cleaning step, before the follow up can be done.

Technically the remapping has nothing to do with shaking dust off. From a user standpoint remapping may well be considered as some sort of sensor cleaning.

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
NikonMan09
NikonMan09 Contributing Member • Posts: 557
Re: Which manual sensor clean?

Bernard Delley wrote:

thanks for the question!

The vibration type, built in, dust shake off procedure is meant. There is a menu option to clean 'now'. This is the menu option to use twice in a row. It takes about 1s to go through the first cleaning step, before the follow up can be done.

Technically the remapping has nothing to do with shaking dust off. From a user standpoint remapping may well be considered as some sort of sensor cleaning.

Bernard,

Thanks for your reply!  I thought it would be as you described above, but, for my sake and that of other readers of this tread, I thought the exact method should be better clarified.

I presently have a D90, D7000, and D7100.  I have never had an issue with hot pixels and never tried to look for them, but it was good to know about this method of dealing with them.  I came to Nikon from Minolta/Sony, and their digital SLR's had a user-accessible method of mapping the hot pixels.

Best wishes,

Fred

-- hide signature --

Formerly known as MinoltaMan78. Switched to Nikon in 2009.

 NikonMan09's gear list:NikonMan09's gear list
Nikon D7200 Nikon D7500 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR Nikon AF-S DX Nikkor 16-85mm f/3.5-5.6G ED VR Nikon AF-S DX Nikkor 35mm F1.8G +11 more
thomo
thomo Senior Member • Posts: 1,330
Re: Which manual sensor clean?

It's really nice to know Nikon have finally introduced a 'fix' for the hot/dead/lazy pixel problem - even if they haven't admitted it or published the information on how to do it!

I was one of the early posts on this topic back in 2010 that drew some great responses and research on the problem http://www.dpreview.com/forums/thread/2867277 together with these others http://www.dpreview.com/forums/thread/2872406 and http://www.dpreview.com/forums/thread/2919341

The most intersting reply to a series of posts on this topic was one I got from someone who worked for the company that made the Kodak sensors for some movie cameras and MFD cameras. Unfortunatel the link from that old post no longer works, but the essence of the response to my post back then was that his company would never ship sensors by air (from the country of manufacture) to their destination. They only sent them by sea because of the likelihood of cosmic radiation damage.

It certainly wasn't just a CCD problem beacause I had to have two D80's (CCD) remapped and two D90's (CMOS) done. At the time I had three other Nikon DSLRs that had no dead/lazy/hot pixel problems - two D700's and a D7000. None of these three bodies had ever flown in an aircrfaft!

The other four that had problems had flown regularly. I have a fifth camera, a D90 that my employer purchased for me to use for work purposes that has a one hot red pixel. It shows up in every shot but because the images are never printed large and generally only viewed on computer screens, I've never bothered to have it remapped.

But that experience has made me cautious. Now when I travel on aircraft, I put the bodies (usually three) in lead-lined film bags. I get some strange enquirees from the airport customs people but have never had any problems with them. and that is based on doing two or three long international flights each year, going through up to eight or nine airport X-ray machines and flight durations up to 22 hours each way.

I've subsequently bought another D7000, an X100, a D800E and a D7100. I never fly with the D800E but all the other three bodies have flown regularly - always in a lead-lined bag. And I've never had a hot/dead/lazy pixel problem since !!!!

I raised the problem with my local authorised Nikon service centre as well as Nikon (Australia), pointing out (at the time) that Canon and Olympus had the facility for the end user to to fix pixel problems without have to send it to the manufacture/importer to rectify the problem at a cost of between $50 and $85.

I'm guessing that Nikon do have people monitoring these forums and have finally acted on it by incorporating the remapping feature in firmwear in the latest cameras. My suspicion is though that it was only to make it easier for the service technicians rather than the end user. But some how the word got out!

 thomo's gear list:thomo's gear list
Canon PowerShot G15 Nikon D80 Nikon D90 Nikon D700 Nikon D7000 +43 more
OP Bernard Delley Senior Member • Posts: 1,769
a lot of history!

Ted Kahn gave links to scientific articles and tech notes on the issue dating back to 1996.

I actually replied to your third thread with "cosmic rays degrade the sensor"

http://www.dpreview.com/forums/post/37150608

There I also mentioned that lead lined bags do not help much, as you are missing a shield of 700 grams/cm^2 at 10000m altitude as compared to sea level. This makes a really heavy wrap to fully restore sea level, hundreds of kilograms to wrap the DSLR !

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
thomo
thomo Senior Member • Posts: 1,330
Re: a lot of history!

Bernard Delley wrote:

Ted Kahn gave links to scientific articles and tech notes on the issue dating back to 1996.

I actually replied to your third thread with "cosmic rays degrade the sensor"

http://www.dpreview.com/forums/post/37150608

There I also mentioned that lead lined bags do not help much, as you are missing a shield of 700 grams/cm^2 at 10000m altitude as compared to sea level. This makes a really heavy wrap to fully restore sea level, hundreds of kilograms to wrap the DSLR !

Thanks Bernard. I remember your reply and you're probably correct. In a previous life (job) I was a Nuclear Radiation Safety officer for the company - we used to have a lot of nucleonic density guages on our process plants.

Regardless of the type of radiation, it is too much to be coincidence that my five cameras that have done a lot of air travel (without using the lead lined bags) have all had damaged sensor pixels. The others that I've since put in the film bags and those that have never 'flown' are trouble free.

That's 5 against 7. One thing I do remember from my radiation courses was the statement by one of my lecturers, "The only safe level of exposure to radiation is no exposure".

If I've had 100% failure with 5 cameras that haven't been protected and 100% success with 7 cameras that have either been protected or never flown, I think the odds are that I'll continue with the lead-lined bags regardless. ( call me paranoid)

I'm just greatful that Nikon have now given us a tool to fix the damaged pixels - even if the have kept it "top-secret"

 thomo's gear list:thomo's gear list
Canon PowerShot G15 Nikon D80 Nikon D90 Nikon D700 Nikon D7000 +43 more
SilentWave Junior Member • Posts: 31
Re: user controlled D800 hot pixel remapping: details

Do you know if there is a procedure which will work with a D3x, which does not have sensor cleaning? If so, how is it done?  Thanks.

labalaba Contributing Member • Posts: 939
Re: user controlled D800 hot pixel remapping: details

rubank wrote:

Glen78 wrote:

I can confirm that it works on the D600 also, although apparently not on the D7000. Maybe it is only for the latest generation of cameras.

This is an old "trick" that works also on D300. Reported here many years ago.

Actually, the first camera I had that this worked on was a Canon (40D if I recall correctly), so it´s not a Nikon only thing. I think the Canon forum was the origin of this procedure.

I tried this last night with my D700 and its hot pixels were not remapped.

OP Bernard Delley Senior Member • Posts: 1,769
D4 is probably the oldest model having this feature

D4 is not confirmed yet. We know that D4 uses the same HPS algorithm as D800 http://www.dpreview.com/forums/post/50874784

newer D600 has the user controlled remapping too.

D7000 with latest firmware L06 does not have the user controlled remapping.

Older models can be remapped at the service center. Under user control are equivalent methods in PP.  Automated remapping methods for the PP work flow have been conceived, but appear not to be available.

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
OP Bernard Delley Senior Member • Posts: 1,769
will double check D300 when I can test it

labalaba wrote:

rubank wrote:

This is an old "trick" that works also on D300. Reported here many years ago.

I tried this last night with my D700 and its hot pixels were not remapped.

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
labalaba Contributing Member • Posts: 939
Re: will double check D300 when I can test it

thanks, I'd like to hear

soloryb Senior Member • Posts: 2,405
Re: user controlled D800 hot pixel remapping: details

OK post your shots. I'm not afraid to learn something new.

Bernard Delley wrote:

This thing about cosmic rays at high altitude (in air flights) came up a few years ago and I believe it was debunked. That is, it doesn't happen to camera sensors at typical commercial air travel altitudes.

http://www.consumertraveler.com/columns/does-airplane-travel-kill-digital-camera-sensors/

Just take a black frame before an intercontinental flight and after and have a careful "look", as I did before making my introductory statement. (needs raw loss-less image format). If I dig them out, I can post my findings for a recent flight forth and back to Japan.

Of course these effects from radiation are not dramatic, its a few pixels that become sick out of millions. Also for humans its not dramatic, air personnel does not have a noticeably shorter life than the rest of us.

 soloryb's gear list:soloryb's gear list
Fujifilm FinePix X100 Nikon D800E Nikon AF-S Nikkor 24-70mm f/2.8G ED Nikon AF-S Nikkor 70-200mm f/2.8G ED VR II Nikon AF Fisheye-Nikkor 16mm f/2.8D +9 more
OP Bernard Delley Senior Member • Posts: 1,769
Re: user controlled D800 hot pixel remapping: details

soloryb wrote:

OK post your shots. I'm not afraid to learn something new.

Bernard Delley wrote:

If I dig them out, I can post my findings for a recent flight forth and back to Japan.

Of course these effects from radiation are not dramatic, its a few pixels that become sick out of millions. Also for humans its not dramatic, air personnel does not have a noticeably shorter life than the rest of us.

looks like the only copy of the data from that test are on my recently died computer...

You should not expect pictures though. (pictures of black frames have been shown before, with proper contrast they look like a starry sky except you do not find constellations and the milky way)

Anyway, this image shows a very strong hot pixel showing up in jpg after a transatlantic flight. Was not there the evening before the flight. This false red light was always in the same location with respect to the sensor, not the image. It could not be seen in low ISO images when this spot was not too dark. Remapping at the service center made it finally disappear.

This post by somebody else shows many hot pixels  aka stars

http://www.dpreview.com/forums/post/52000388

Why? There are false stars shining through the dark side of the moon! The folse stars are clearly sharper than the slightly blurred moon - optically imaged stars should have the same blur!  Real stars could not be recorded with the mentioned exposure!

For technical purposes it is better to "look" through loss-less raw files using a computer program to spot the few to hundreds of excessively bright pixels in the millions of well behaved pixels. Of course a bright pixel may hit any of the Bayer RGB colors.

 Bernard Delley's gear list:Bernard Delley's gear list
Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Micro-Nikkor 60mm F2.8G ED +10 more
Keyboard shortcuts:
FForum MMy threads