Recommended Strategy for Processing Digital Images

Depending upon the illumination conditions, specimen integrity, and preparation methods, digital images captured in the optical microscope may require a considerable amount of rehabilitation to achieve a balance between scientific accuracy, cosmetic equilibrium, and aesthetic composition. When first acquired by a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) image sensor, digital images from the microscope often suffer from poor signal-to-noise, uneven illumination, focused or defocused dirt and debris, glare, color shifts, and a host of other ailments that degrade overall image quality.

After an image has been acquired from the microscope, it is commonly referred to as a raw image and should be catalogued and saved before commencement of image processing or analysis operations. This step will ensure that an original copy of the image is available in case irreversible mistakes are made during image processing or the image is lost or otherwise rendered unrecoverable. In addition to the raw image, a background image or flat-field frame should also be recorded with the specimen removed from the optical path to generate an image of the background without the specimen. The best method is to relocate the viewfield to an area of the slide that contains mounting medium and the coverslip, but does not have a significant level of debris, in order to simulate the background illumination profile. An alternative method is to defocus the microscope with the specimen in place, and then record a background image. Although the latter technique is not preferred, it is sometimes unavoidable, particularly if the specimen consumes a majority of the area underneath the coverslip. In situations where quantitative information must be recorded from the specimen, several background images can be averaged together to reduce the level of noise.

In addition to a background image, it is often wise to also record a dark frame to establish the dark (electronic and thermal) noise level of the digital camera system. The dark frame should use the same exposure as the original raw image, but is obtained without opening the camera shutter. In some cases, it may be necessary to detach the camera from the microscope and cover the mounting adapter with a lens cap or a section of black cardboard. As with the flat-field frame or background image, multiple dark frames can be collected and averaged together. After all of the necessary images have been gathered (raw, background, and dark), image processing operations can proceed.

A typical image captured in oblique illumination with the Olympus MIC-D digital microscope is presented in Figure 1(a). The specimen is a fixed and stained young starfish (just after metamorphosis) prepared as a whole mount imbedded in media and sandwiched between a glass slide and coverslip. Because of illumination problems frequently encountered with off-axis illumination scenarios, this specimen was chosen to illustrate problems due to the large visible background region. Oblique illumination from the microscope illuminator produces a somewhat uneven background (often appearing as a gradient of intensity), as is evidenced by brightness variations in Figure 1(a). There is also a greenish cast to the entire image that is caused by incorrect white balance adjustment of the CMOS image sensor. In addition, the image was captured at a relatively low light level, which increases the amount of random noise present in the image (note the grainy appearance of the background and specimen).

A gray-level image histogram (Figure 1) of the original raw image reveals that only a small portion of the CMOS image sensor's dynamic range was utilized in producing the image. A vast majority of the gray levels in the image are clustered in a region between values of 60 and 210 with very few pixels containing levels that are either higher or lower. This restriction of the histogram gray-level values to the center of the scale produces poor image contrast (see Figure 1(a)) that must be corrected during image processing. The large spike in gray levels centered around 140 corresponds to the background color. As the background becomes more uniform (reducing the number of gray levels) during processing, the width of this distribution will narrow, but the pixel count will increase.

Image processing of the optical micrograph illustrated in Figure 1 can be accomplished with a number of commercially available software packages, including the MIC-D interface software itself. Recommended low-cost aftermarket image-editing software includes Adobe Photoshop, Corel Photo-Paint, Macromedia Fireworks, and Paint Shop Pro. In addition, higher-end software programs, designed specifically for digital images from the microscope, can be utilized to perform the necessary image-editing functions. Many of these programs contain algorithms that simplify commonly utilized steps in micrograph image processing, such as background subtraction, flat-field correction, histogram manipulation, and gamma correction.

The first step in image processing is to remove brightness fluctuations (due to uneven background illumination) and noise introduced by specimen or camera system. For a majority of digital images, simple background subtraction algorithms are sufficient and will produce corrected images that have even brightness values across the image. However, with images destined for quantitative analysis that requires photometric accuracy, flat-field correction is the preferred technique.

The raw starfish image illustrated in Figure 1(a) was adjusted with a background subtraction algorithm (Figure 2(a)) to produce the corrected image presented in Figure 2(b). The algorithm employs user-selectable control points that can be strategically relocated around the image to specify brightness areas within the background. After placement of the control points, their representative brightness values are utilized to fit a surface function to create an artificial background. The calculated background is then subtracted from the specimen image to obtain a least squares fit of the surface function that approximates how the background should appear. In practice, the control points should be chosen so that they are evenly distributed across the image (as they are in Figure 2(a)), and the brightness level at each control point should be representative of the overall background intensity.

After background subtraction or flat-field correction adjustments have been applied to the image, the next step is to restore brightness and contrast levels to match how the specimen appears in the microscope (either through the eyepieces or on the live video feed to the camera capture software). Histogram stretching and sliding operations typically appear as brightness and contrast adjustment sliders in the common image-editing software programs. More sophisticated manipulation of the histogram profile is usually possible in high-end software packages, but the basic tools available in all image editors can often be utilized to perform satisfactory adjustments to the image.

The starfish specimen image, after background subtraction, contrast, and brightness alterations is illustrated in Figure 3(a). Note how the stained appendages have become more clearly defined and the overall image contrast has improved. In addition, the background has been shifted from a greenish fluctuating gradient to a smooth (when overlooking the grainy noise) and even gray tone. The modified histogram is presented in Figure 3(b), and can be compared to the original raw image histogram in Figure 1(b). After processing, the histogram is stretched to cover a wider range of gray levels (10 to 220 versus 60 to 210) with more pixels shifted to darker values. As a result, shadow tones in the corrected image appear deeper and richer than in the original and the contrast level is much higher.

The next step in processing is adjustment of gamma to enable exponential scaling for the purpose of simultaneously displaying both bright and dark features of the image on a computer monitor. Presented in Figure 4(a) is the starfish specimen image after gamma correction. When adjusting a computer monitor that will be utilized for image processing, the gamma correction depends, in part, on the brightness and contrast control settings on the monitor. This feature enables the microscopist or digital artist to alter gamma correction to suit individual tastes and compensate for variability between presentation formats, such as individual web browsers on a variety of computer operating system platforms.

In order to reduce or eliminate random noise from the image, a specialized convolution kernel, known as a smoothing filter is often applied to the image. In most image-editing software programs, this type of operation is referred to as a noise (often specifically designated for dust and scratches), blur, or Gaussian blur filter. Some of the algorithms have user-configurable controls, while others simply apply a series of fixed settings to the image. Proper application of smoothing filters can effectively remove noise, scratches, and other high-spatial-frequency artifacts from digital images. Noise in the starfish specimen image (readily apparent in the background) has been removed by filtration in Figure 4(b) to produce a smooth background, but noticeably softer (more blurred) image features.

Once the noise and other imperfections have been removed from the image, a sharpening algorithm can be applied to remove low-frequency spatial information and enhance the definition of fine edge detail. The MIC-D software interface Processing Window contains a sharpness slider in the Image settings panel that can be used to increase detail definition. In addition, many of the popular image-editing programs contain an unsharp mask algorithm that is ideal for this purpose. One of the primary advantages of the unsharp mask filter over other sharpening filters is the flexibility of control, because a majority of the other filters do not provide any user-adjustable parameters. Illustrated in Figure 5(a) is the starfish specimen after an unsharp mask filter has been applied to increase edge detail. When compared to Figure 4(b), which is the same image after application of a Gaussian blur filter, the dramatic enhancement of fine detail is obvious. Care should be taken not to over-apply sharpening filters, which can re-introduce noise and similar artifacts into the image and, when taken to an extreme, produce severe pixelation at the edges.

The final step in processing is to correct the color balance and adjust saturation of the image to remove unwanted color casts and return image colors to those observed in the microscope. The MIC-D software interface Processing Window has a Color settings panel that enables the operator to adjust hue, saturation, and color balance of images captured with the microscope. These algorithms are applied through the use of sliders to either increase or decrease saturation, or transition through various hues and color balance ratios. All of the popular image-editing software programs also contain similar algorithms at varying levels of sophistication.

After the image processing steps have been completed, the adjusted image can be compared to the original raw image (see Figures 6(a) and 6(b)) to determine how much improvement has been achieved. It is evident, by examining Figure 6, that overall image contrast, brightness, and saturation is remarkably better in the processed image. In addition, the uneven background has been replaced by an almost uniformly gray matte substitute that also eliminates the green cast. Noise has also been reduced, and the fine image details are noticeably sharper.

As discussed above, the processing steps can be accomplished with aftermarket image-editing software or through the MIC-D digital microscope interface software Processing Window. Although the microscope software contains basic image processing algorithms that perform adequately, many of the more complex software programs contain additional features that can be utilized to create special effects or conduct measurements at any stage of the processing scheme. For example, Adobe Photoshop contains a wide range of filter algorithms, which enable the operator to apply stylizing features, de-speckle or distort the image, and add special effects such as brush textures and lens flare. In fact, this program is so comprehensive that images captured with the microscope can be rendered into completely different manifestations, which bear little resemblance to the original raw image. If the purpose of image processing is to rehabilitate and restore the image for scientific measurement or presentation, then use of auxiliary image-editing algorithms should be kept to a minimum and applied only when necessary. However, if the image is destined to be modified for artistic purposes, then the wide spectrum of special effects available in modern software programs can be applied without regard to scientific accuracy.