Skip to content

Latest commit

 

History

History
155 lines (99 loc) · 4.49 KB

image_ade20k.md

File metadata and controls

155 lines (99 loc) · 4.49 KB
Raw

Semantic segmentation of images with PixelLib using Ade20k model

PixelLib is implemented with Deeplabv3+ framework to perform semantic segmentation. Xception model trained on ade20k dataset is used for semantic segmentation.

Download the xception model from here

Code to implement semantic segmentation:

  import pixellib
  from pixellib.semantic import semantic_segmentation

  segment_image = semantic_segmentation()
  segment_image.load_ade20k_model("deeplabv3_xception65_ade20k.h5")
  segment_image.segmentAsAde20k("path_to_image", output_image_name= "path_to_output_image")

We shall take a look into each line of code.

  import pixellib
  from pixellib.semantic import semantic_segmentation

  #created an instance of semantic segmentation class
  segment_image = semantic_segmentation()

The class for performing semantic segmentation is imported from pixellib and we created an instance of the class.

  
  segment_image.load_ade20k_model("deeplabv3_xception65_ade20k.h5")

We called the function to load the xception model trained on ade20k dataset.

  segment_image.segmentAsAde20k("path_to_image", output_image_name= "path_to_output_image")

This is the line of code that performs segmentation on an image and the segmentation is done in the pascalvoc's color format. This function takes in two parameters:

path_to_image: the path to the image to be segemented.

path_to_output_image: the path to save the output image. The image will be saved in your current working directory.

Sample1.jpg

alt_test4

  import pixellib
  from pixellib.semantic import semantic_segmentation

  segment_image = semantic_segmentation()
  segment_image.load_ade20k_model("deeplabv3_xception65_ade20k.h5")
  segment_image.segmentAsAde20k("sample1.jpg", output_image_name="image_new.jpg")

alt_test5

Your saved image with all the objects present segmented.

You can obtain an image with segmentation overlay on the objects with a modified code below.

  segment_image.segmentAsAde20k("sample1.jpg", output_image_name = "image_new.jpg", overlay = True)

We added an extra parameter overlay and set it to true, we produced an image with segmentation overlay.

alt_test6

Sample2.jpg

alt_test7

  
  segment_video.segmentAsAde20k("sample2.jpg", output_image_name="image_new2.jpg")

alt

Specialised uses of PixelLib may require you to return the array of the segmentation's output.

  • Obtain the array of the segmentation's output by using this code,
  segvalues, output = segment_image.segmentAsAde20k()
  • You can test the code for obtaining arrays and print out the shape of the output by modifying the semantic segmentation code below.
  
  import pixellib
  from pixellib.semantic import semantic_segmentation
  import cv2

  segment_image = semantic_segmentation()
  segment_image.load_ade20k_model("deeplabv3_xception65_ade20k.h5")
  segvalues, output = segment_image.segmentAsAde20k("sample2.jpg")
  cv2.imwrite("img.jpg", output)
  print(output.shape)

Note: Access the masks of the objects segmented using segvalues["masks"] and their class ids using segvalues["class_ids"].

  • Obtain both the output and the segmentation overlay's arrays by using this code,
  segvalues, segoverlay = segment_image.segmentAsAde20k(overlay = True)
  
  import pixellib
  from pixellib.semantic import semantic_segmentation
  import cv2

  segment_image = semantic_segmentation()
  segment_image.load_ade20k_model("deeplabv3_xception65_ade20k.h5")
  segvalues, segoverlay = segment_image.segmentAsAde20k("sample2.jpg", overlay= True)
  cv2.imwrite("img.jpg", segoverlay)
  print(segoverlay.shape)

This xception model is trained on ade20k dataset, a dataset with 150 object categories.

Process opencv's frames

  import pixellib
  from pixellib.semantic import semantic_segmentation
  import cv2

  segment_frame = semantic_segmentation()
  segment_frame.load_ade20k_model("deeplabv3_xception65_ade20k.h5")
  capture = cv2.VideoCapture(0)
  while True:
    ret, frame = capture.read()
    segmap, output = segment_frame.segmentFrameAsPascalvoc(frame)
    cv2.imshow("frame", output)
    if  cv2.waitKey(25) & 0xff == ord('q'):
        break