Advanced Image Processing Methods

Python libraries like scikit-image offer some advanced image processing techniques used by scientists. We'll touch on a few below. If you're interested in learning more, a good resource is http://scikit-image.org/docs/dev/auto_examples. 

Thresholding

We briefly touched upon the idea of thresholding in the previous section when we picked out all the pixels that were above a certain value in order to identify which parts of the image corresponded to the white flowers. In general, thresholding is useful for picking out certain objects if these objects have a similar color (or intensity) to one another and are different in intensity from other objects in the picture. The threshold we set in the last section, 90%, was a good guess that let us identify the flowers, but how do we know whether there are better values we could have picked?

One popular technique is Otsu's method, which analyzes the values of all the pixels in a specific image to choose the best threshold value to separate different objects. The scikit-image library has a function called “threshold_otsu” that uses Otsu’s method to calculate a threshold. Because this is a very famous thresholding technique, Otsu’s method is included in a lot of image processing software.

We find that the threshold value of the flowers picture is 0.54 (recall that the maximum value is 1.0). We create a thresholded image with the command "thresholded_image = imagefile > threshold", which returns a 2D boolean array to determine which parts of the image are above or below the threshold. A boolean can be either True or False. If the value of a pixel is True, the pixel intensity is above the threshold (lighter than the threshold). If the value of the pixel is False, the pixel intensity is below the threshold (darker than the threshold).

In computer science, True and False are usually set equivalent to 1 and 0, respectively. Recall that in our image, 1 corresponded to white and 0 corresponded to black. So “thresholded_image” is a NumPy array that represents a binary image, where every pixel can only take on one of two possible values. Again, we can plot this array/image using matplotlib. Note that this method enables us to pick out the regions of the image with the flower petals, but you can’t really see the leaves anymore. This is useful if you want to simplify the image so that you can focus on the flowers.

Edge detection

In the previous section, we picked out objects of interest (i.e., the flowers) by focusing on the brightest regions, since we know that the flowers are lighter than the other parts of the image. Another way you can pick out an object of interest from image is through edge detection. The basic idea behind edge detection methods is that they check how quickly intensity values change across an image. This assumes that within a single object, the intensity does not change too much, but the intensities change a lot between objects. For example, within a single flower petal, there are small variations in color, but the pixels are mostly similar in intensity to one another. But when you cross over to a leaf, the pixels suddenly become much darker.

One widely-used edge detection method is called the Roberts algorithm, which is also included as a function in the scikit-image library. We apply this algorithm to the same image of flowers we analyzed in the previous section, and we see that the processed image mostly consists of the edges of the flowers, with some contribution from the wooden frame on the side and the leaves on the bottom of the image.

Template matching

Sometimes, you'll want to figure out where or how many times a certain kind of object appears in an image. (Imagine, for instance, that you're dealing with a huge number of images that would be difficult for a single person to look through quickly). This is where template matching comes in. For example, if you're curious about how many times the white flower shows up in the image, you can start with a template, and then make a map of where the image of interest best matches the template. For our template image, we are using one of the flowers from the image and assuming that it is similar enough to the other flowers in the image that computer software will identify the other flowers as a good match to the template image, without confusing other objects as flowers.

We do not necessarily have to use part of the original image as a template. You could just as easily select a flower from a different image. Sometimes people will simulate an image to use as a template based on their knowledge of what the object should look like.

Underneath the hood, the function is essentially figuring out where the array of numbers represented by the flower template best matches a sub-section of the array of numbers representing the image you're processing. This function returns a 2D array of numbers (which we call “match_intensity” in the code below) indicating how well a location in the image matches to the template. Higher numbers mean the match is better.

We can make a plot of this 2D array. We can see that the brightest spot in the map is near the center, which corresponds to the location of the flower that was originally used to make the template. But notice that we also get four other bright spots, corresponding to the other four similar flowers that are also in the image. We can locate these other four flowers using our old friends np.max() and np.where() to find the local maxima in the image (i.e., the pixels in the image that are brighter than anything nearby).

If we run the code

y_bestmatch, x_bestmatch = np.where(np.max(match_intensity)==match_intensity) 

then we’ll only get the location of the brightest spot at the center. To get the location of a different local maximum, we need to select only part of the array using array slicing:

y_match, x_match = np.where(np.max(match_intensity[:400, 1000:])==match_intensity[:400,1000:])

This line of code will help you find the location of the flower in the upper right corner of the intensity map. However, the value that you get for x_match will be 176, because the code only looks at the part of the array where the x-value is larger than 1000. So, to get back the value of x that corresponds to the original match_intensity plot, you need to add 1000 so that you get 1176.

Now it's your turn!

Click on the Edit in Repl.it button in the upper right corner of the terminal below. This will take you to the repl.it website, where you can create a free account (recommended, if you haven't already). Then, you can write, run, and save your own code to answer the questions below (also provided as comments on the repl.it site):

If you have any trouble navigating the repl.it site, please see our Student Guide to Repl.it.