Saturday, April 14, 2007

Better "Dark Frame" Subtraction in Photoshop

Night photography brings with it a number of unique challenges (not the least of which is tripping over your gear in the dark), but one such nuisance are so-called "hot pixels." Hot pixels are the bane of any long exposure digital photography, including night photography and astrophotography, and typically they appear as star-like bright specs in the image. From what I understand, they arise from "thermal noise" within the image sensor. Technological specifics aside, you'll generally find that the longer the exposure, the more likely you are to see these spots (they look a little like stars, but they never move).

The most common way to deal with them, though, involves shooting what's called a "dark frame" which can then be digitally subtracted from the original image to eliminate the hot pixels. A dark frame is made by shooting an image of equal duration (shutter speed) to the original shot, but with the lens cap on so that the frame is completely black ("dark") except for the sensor noise and hot pixels. Some cameras, in fact, do this automatically for exposures beyond a certain length. [Note: you don't put the lens cap on...the camera simply closes the shutter and exposes the equal exposure time dark frame which is then subtracted in-camera].

But there's a big problem with this simple approach: the subtraction turns the hot pixels into black holes or black spots (something minus the same something equals zero which equals black). Now, for astrophotography, this is not so much of an issue since a black spot against a black sky is basically invisible. But for those of us shooting night photography this can be a real pain when odd black specs appear on illuminated subjects. (see image below)

What we really need is a way to fill in the hot pixels with a blend of the surrounding pixels to better conceal their removal. But in a quick search of the web, I couldn't find a Mac OS X compatible program or Photoshop action that would do what I want. Sure, there are several apps out there for Mac (e.g., the very cool and uncrippled shareware, Keith's Image Stacker, for OS X) and for Windows, but all just do simple subtraction. The exception to this is a freeware app called BlackFrame NR but it only runs on Windows [grrrr...].

What follows is my own little method I devised. I've made it into a freeware Photoshop action which can be easily tweaked to your camera's specific behavior and should work under Windows and Mac OS X, but I've only tried it under Photoshop CS for Mac. You can download my Hot Pixel Remover here, but I'll explain it below since I don't provide any documentation (hey, it's free, so don't complain).

First, in Photoshop CS open the original image, hot pixels and all. It will occupy a layer called "Background". Next, open the dark frame image, select all (Cmd-A) of it, copy it (Cmd-C) and paste it into a new layer over the original image. The dark frame, now called "Layer 1" should thus be in perfect registration with the Background layer. Now simply run my Hot Pixel Remover action, wait a few seconds, and voila, no more hot pixels and no more black holes.

Here's what it's doing:

1) It starts by making a duplicate of the original layer and applying a median blur of about 5 pixels (Filter-->Noise-->Median...) which should average out the hot pixels, which is good. But it also averages out lots of image detail, which is bad. But here's where the magic comes in: we're going to use the dark frame as a transparency mask to only show through those areas of the blurred image that correspond to a hot pixel.

2) The dark frame layer is curves-adjusted (Image->Adjustments->Curves...) to clip the very darkest 15 levels so that we can enhance only the hot pixels.

3) The dark frame is then desaturated to grayscale (Image->Adjustments->Desaturate) and then an Auto Contrast (Image->Adjustments->Auto Contrast) is performed to punch up the hot pixels.

4) The hot pixels are then selected (Select->Color Range...) in such a way as to select based on closeness to black.

5) This selection is then applied as a transparency mask to the median-blurred layer by clicking on the "Add layer mask" button in the layers palette.

6) The dark frame is then turned off and the result is a nicely cleaned up photo.

Below is the final result (a closeup crop) which shows neither hot pixels nor black holes. Nice!

Remember, this is just a version 1.0 attempt at this, so there may be some bugs under certain circumstances. Please let me know if there are. Maybe this will inspire someone to come up with a better way. (But best of all for me is that it's quick, free, and works on my Mac!) Again, you can download "Faceman's Hot Pixel Remover.atn" here.

Labels: , , ,


Blogger Andy Frazer said...

Very interesting. I've posted this to my night photography blog.

I'm hoping we can get some feedback from some other night photographers.

Andy Frazer

April 17, 2007 2:24 PM  
Blogger M. Covington said...

Interesting approach. I think the root of the problem is gamma correction. When an image has been gamma-corrected (see, subtraction is no longer linear; dark frame subtraction works much better on raw frames or linear conversions.

See also:
and scroll down to dark frame subtraction.

June 9, 2007 11:10 AM  
Blogger an eye in the world said...

This post has been removed by the author.

July 28, 2007 4:44 PM  
Blogger an eye in the world said...

Thanks so much for posting this. Also being a Mac user, I was looking for a Mac program equivalent to Blackframe NR and came across this posting.

Depending upon what camera you are using, it may be necessary to adjust the curves clipping point in step two. This is easily done by double clicking on this step in the actions palette once you import the action. Drag the bottom curve point to change it, restore the original image, and rerun the action to see the effect.

For my Pentax DS body with a 30 second exposure, a black clipping point of about 35 seems to work pretty well (instead of the default 15). Otherwise, too much of the blurred image shows through and takes away most of the detail in the image.

Now, to setup a script that runs this automatically on a set of images!

July 28, 2007 4:55 PM  
Blogger system operator said...

This seems interesting and working aproach. I will try this soon:)

I have been thinking that those what you refer as hot pixels are actually dead pixels in you cameras sensor. They are not able to make sensors without some dead areas and when you shoot long apertures you can see them because the cameras software is no longer able to correct it.

July 13, 2008 10:58 AM  

Post a Comment

<< Home