TDM 19000: Systematic Approaches to Masks

We have looked at the mask for just one object in this image. The YOLO model is predicting 18 items - 17 people, 1 plant. If we tried taking our lines from earlier and looping through each item of 'masks':

# display the image
plt.imshow(original_img)

# loop through the masks
for mask in masks:
    if mask.ndim == 3:
        mask = mask[0]
    plt.imshow(mask, alpha=0.4)

# clean up the display
plt.axis("off")
plt.show()

….the image is not very clear. The image from the shopping mall that we processed with SAM showed the mask as yellow for the object, purple for everything else. When we layer 18 slightly different versions of this together, the background color is going to build up and block almost everything.

There are a few ways to avoid this.

Method 1

Display a copy of the image for each mask. This could be as simple as changing how the for loop is defined:

(old) for mask in masks: ….
(new) for i, mask in masks: ….

This does work, but creates a very long output. The code below helps format these image to make them more viewable at once.

fig, axes = plt.subplots(6, 3, figsize=(10, 18))
axes = axes.flatten()

for i, mask in enumerate(masks):
    if mask.ndim == 3:
        mask = mask[0]

    axes[i].imshow(original_img)
    axes[i].imshow(mask, alpha=0.4)

plt.show()

Method 2

Display just the masks for each of the objects, no background coverage.

Because we’re going to be coming back to the original image later, it is important to make a copy here. At the same time, we’ll also make sure it is a NumPy array, and convert it to be of type float to allow for blending to happen.

You could avoid making a copy of the image by just re-reading it in later. Either way, this copy is going to be "drawn" on (will have color shapes added).

import numpy as np

image = np.array(original_img).copy()
overlay = image.astype(np.float32)

Now we can bring back our loop from the start of these notes, and add in some lines.

for mask in masks:
# fill in with the mask loop lines
    if mask.ndim == 3:
        mask = mask[0]

    # random color for each object
    color = np.random.rand(3)

    # simple coloring
    overlay[mask > 0] = color * 255

overlay = overlay.astype(np.uint8)
plt.imshow(overlay)
plt.axis("off")
plt.show()