Skip to content

Latest commit

 

History

History
328 lines (287 loc) · 17.8 KB

v0.2.9.summary.md

File metadata and controls

328 lines (287 loc) · 17.8 KB
Raw

0.2.9

This update mainly covers the following topics:

  • Moved classes/methods related to augmentable data to their own modules.
  • Added polygon augmentation methods.
  • Added line strings and line string augmentation methods.
  • Added easier augmentation interface.

New 'augmentables' Modules

For the Polygon and Line String augmentation, new classes and methods had to be added. The previous file for that was imgaug/imgaug.py, which however was already fairly large. Therefore, all classes and methods related to augmentable data were split off and moved to imgaug/augmentables/<type>.py. The new modules and their main contents are:

  • imgaug.augmentables.batches: Contains Batch, UnnormalizedBatch.
  • imgaug.augmentables.utils: Contains utility functions.
  • imgaug.augmentables.bbs: Contains BoundingBox, BoundingBoxesOnImage.
  • imgaug.augmentables.kps: Contains Keypoint, KeypointsOnImage.
  • imgaug.augmentables.polys: Contains Polygon, PolygonsOnImage.
  • imgaug.augmentables.lines: Contains LineString, LineStringsOnImage.
  • imgaug.augmentables.heatmaps: Contains HeatmapsOnImage.
  • imgaug.augmentables.segmaps: Contains SegmentationMapOnImage.

Currently, all augmentable classes can still be created via imgaug.<type>, e.g. imgaug.BoundingBox still works.

Changes related to the new modules:

  • Moved Keypoint, KeypointsOnImage and imgaug.imgaug.compute_geometric_median to augmentables/kps.py.
  • Moved BoundingBox, BoundingBoxesOnImage to augmentables/bbs.py.
  • Moved Polygon, PolygonsOnImage and related classes/functions to augmentables/polys.py.
  • Moved HeatmapsOnImage to augmentables/heatmaps.py.
  • Moved SegmentationMapOnImage to augmentables/segmaps.py.
  • Moved Batch to augmentables/batches.py.
  • Added module imgaug.augmentables.utils.
    • Added function normalize_shape().
    • Added function project_coords().
  • Moved line interpolation functions _interpolate_points(), _interpolate_point_pair() and _interpolate_points_by_max_distance() to imgaug.augmentables.utils and made them public functions.
  • Refactored __init__() of PolygonsOnImage, BoundingBoxesOnImage, KeypointsOnImage to make use of imgaug.augmentables.utils.normalize_shape().
  • Refactored KeypointsOnImage.on() to use imgaug.augmentables.utils.normalize_shape().
  • Refactored Keypoint.project() to use imgaug.augmentables.utils.project_coords().

Polygon Augmentation

Polygons were already part of imgaug for quite a while, but couldn't be augmented yet. This version adds methods to perform such augmentations. It also makes some changes to the Polygon class, see the list of changes below.

Example for polygon augmentation:

import imgaug as ia
import imgaug.augmenters as iaa
from imgaug.augmentables.polys import Polygon, PolygonsOnImage

image = ia.quokka(size=0.2)
psoi = PolygonsOnImage([
    Polygon([(0, 0), (20, 0), (20, 20)])
], shape=image.shape)

image_aug, psoi_aug = iaa.Affine(rotate=45).augment(
    images=[image],
    polygons=[psoi]
)

See imgaug-doc/notebooks for a jupyter notebook with many more examples.

Changes related to polygon augmentation:

  • Added _ConcavePolygonRecoverer to imgaug.augmentables.polys.
  • Added PolygonsOnImage to imgaug.augmentables.polys.
  • Added polygon augmentation methods:
    • Added augment_polygons() to Augmenter.
    • Added _augment_polygons() to Augmenter.
    • Added _augment_polygons_as_keypoints() to Augmenter.
    • Added argument polygons to imgaug.augmentables.batches.Batch.
    • Added attributes polygons_aug and polygons_unaug to imgaug.augmentables.batches.Batch.
    • Added polygon handling to Augmenter.augment_batches().
  • Added property Polygon.height.
  • Added property Polygon.width.
  • Added method Polygon.to_keypoints().
  • Added optional drawing of corner points to Polygon.draw_on_image() and PolygonsOnImage.draw_on_image().
  • Added argument raise_if_too_far_away=True to Polygon.change_first_point_by_coords().
  • Added imgaug.quokka_polygons() function to generate example polygon data.
  • [rarely breaking] Polygon.draw_on_image(), PolygonsOnImage.draw_on_image()
    • Refactored to make partial use LineString methods.
    • Added arguments size and size_perimeter to control polygon line thickness.
    • Renamed arguments alpha_perimeter to alpha_line, color_perimeter to color_line to align with LineStrings.
    • Renamed arguments alpha_fill to alpha_face and color_fill to color_face.
  • [rarely breaking] Changed the output of Polygon.clip_out_of_image() from MultiPolygon to list of Polygon. This breaks for anybody who has already used Polygon.clip_out_of_image().
  • Changed Polygon.exterior_almost_equals() to accept lists of tuples as argument other_polygon.
  • Changed arguments color and alpha in Polygon.draw_on_image() and PolygonsOnImage.draw_on_image() to represent the general color and alpha of the polygon. The colors/alphas of the inner area, perimeter and points are derived from color and alpha (unless color_inner, color_perimeter or color_points are set (analogous for alpha)).
  • Refactored Polygon.project() to use LineString.project().
  • Refactored Polygon.shift() to use LineString.shift().
  • [rarely breaking] Polygon.exterior_almost_equals(), Polygon.almost_equals()
    • Refactored to make use of LineString.coords_almost_equals().
    • Renamed argument interpolate to points_per_edge.
    • Renamed argument other_polygon to other.
  • Renamed color_line to color_lines, alpha_line to alpha_lines in Polygon.draw_on_image() and PolygonsOnImage.draw_on_image().
  • Fixed Polygon.clip_out_of_image(image) not handling image being a tuple.
  • Fixed Polygon.is_out_of_image() falsely only checking the corner points of the polygon.

LineString Augmentation

This version adds Line String augmentation. Line Strings are simply lines made up of consecutive corner points that are connected by straight lines. Line strings have similarity with polygons, but do not have a filled inner area and are not closed (i.e. first and last coordinate differ).

Similar to other augmentables, line string are represented with the classes LineString(<iterable of xy-coords>) and LineStringsOnImage(<iterable of LineString>, <shape of image>). They are augmented e.g. via Augmenter.augment_line_strings(<iterable of LineStringsOnImage>) or Augmenter.augment(images=..., line_strings=...).

Example:

import imgaug as ia
import imgaug.augmenters as iaa
from imgaug.augmentables.lines import LineString, LineStringsOnImage

image = ia.quokka(size=0.2)
lsoi = LineStringsOnImage([
    LineString([(0, 0), (20, 0), (20, 20)])
], shape=image.shape)

image_aug, lsoi_aug = iaa.Affine(rotate=45).augment(
    images=[image],
    line_strings=[lsoi]
)

See imgaug-doc/notebooks for a jupyter notebook with many more examples.

Simplified Augmentation Interface

Augmentation of different data corresponding to the same image(s) has been a bit convoluted in the past, as each data type had to be augmented on its own. E.g. to augment an image and its bounding boxes, one had to first switch the augmenters to deterministic mode, then augment the images, then the bounding boxes. This version adds methods that perform these steps in one call. Specifically, Augmenter.augment(...) is used for that, which has the alias Augmenter.__call__(...). One argument can be used for each augmentable, e.g. bounding_boxes=<bounding box data>. Example:

import imgaug as ia
import imgaug.augmenters as iaa
from imgaug.augmentables.kps import Keypoint, KeypointsOnImage

image = ia.quokka(size=0.2)
kpsoi = KeypointsOnImage([Keypoint(x=0, y=10), Keypoint(x=10, y=5)],
                         shape=image.shape)

image_aug, kpsoi_aug = iaa.Affine(rotate=(-45, 45)).augment(
    image=image,
    keypoints=kpsoi
)

This will automatically make sure that image and keypoints are rotated by corresponding amounts.

Normalization methods have been added to that class, which allow it to process many more different inputs than just variations of *OnImage. Example:

import imgaug as ia
import imgaug.augmenters as iaa

image = ia.quokka(size=0.2)
kps = [(0, 10), (10, 5)]

image_aug, kps_aug = iaa.Affine(rotate=(-45, 45)).augment(
    image=image, keypoints=kps)

Examples for other inputs that are automatically handled by augment():

  • Integer arrays as segmentation maps.
  • Float arrays for heatmaps.
  • list([N,4] ndarray) for bounding boxes. (One list for images, then N bounding boxes in (x1,y1,x2,y2) form.)
  • list(list(list(tuple))) for line strings. (One list for images, one list for line strings on the image, one list for coordinates within the line string. Each tuple must contain two values for xy-coordinates.)
  • list(list(imgaug.augmentables.polys.Polygon)) for polygons. Note that this "skips" imgaug.augmentables.polys.PolygonsOnImage.

In python <3.6, augment() is limited to a maximum of two inputs/outputs and if two inputs/outputs are used, then one of them must be image data and such (augmented) image data will always be returned first, independent of the argument's order. E.g. augment(line_strings=<data>, polygons=<data>) would be invalid due to not containing image data. augment(polygons=<data>, images=<data>) would still return the images first, even though they are the second argument.

In python >=3.6, augment() may be called with more than two arguments and will respect their order. Example:

import numpy as np
import imgaug as ia
import imgaug.augmenters as iaa

image = ia.quokka(size=0.2)
kps = [(0, 10), (10, 5)]
heatmap = np.zeros((image.shape[0], image.shape[1]), dtype=np.float32)
rotate = iaa.Affine(rotate=(-45, 45))

heatmaps_aug, images_aug, kps_aug = rotate(
    heatmaps=[heatmap],
    images=[image],
    keypoints=[kps]
)

To use more than two inputs/outputs in python <3.6, add the argument return_batch=True, which will return an instance of imgaug.augmentables.batches.UnnormalizedBatch.

Changes related to the augmentation interface:

  • Added Augmenter.augment() method.
  • Added Augmenter.augment_batch() method.
    • This method is now called by Augmenter.augment_batches() and multicore routines.
  • Added imgaug.augmentables.batches.UnnormalizedBatch.
  • Added module imgaug.augmentables.normalization for data normalization routines.
  • Changed augment_batches():
    • Accepts now UnnormalizedBatch as input. It is automatically normalized before augmentation and unnormalized afterwards. This allows to use Batch instances with non-standard datatypes.
    • Accepts now single instances of Batch (and UnnormalizedBatch).
    • The input may now also be a generator.
    • The input may now be any iterable instead of just list (arrays or strings are not allowed).
  • Marked support for non-Batch (and non-UnnormalizedBatch) inputs to augment_batches() as deprecated.
  • Refactored Batch.deepcopy()
    • Does no longer verify attribute datatypes.
    • Allows now to directly change attributes of created copies, e.g. via batch.deepcopy(images_aug=...).

Other Additions, Changes and Refactorings

Keypoint augmentation

  • Added method Keypoint.draw_on_image().
  • [mildly breaking] Added an alpha argument to KeypointsOnImage.draw_on_image(). This can break code that relied on the order of arguments of the method (though will usually only have visual consequences).
  • KeypointsOnImage and Keypoint copying:
    • Added optional arguments keypoints and shape to KeypointsOnImage.deepcopy().
    • Added optional arguments keypoints and shape to KeypointsOnImage.copy().
    • Added method Keypoint.copy().
    • Added method Keypoint.deepcopy().
      • Refactored methods in Keypoint to use deepcopy() to create copies of itself (instead of instantiating new instances via Keypoint(...)).
    • KeypointsOnImage.deepcopy() now uses Keypoint.deepcopy() to create Keypoint copies, making it more flexible.
    • Refactored KeypointsOnImage to use KeypointsOnImage.deepcopy() in as many methods as possible to create copies of itself.
    • Refactored Affine, AffineCv2, PiecewiseAffine, PerspectiveTransform, ElasticTransformation, Rot90 to use KeypointsOnImage.deepcopy() and Keypoint.deepcopy() during keypoint augmentation.
  • Changed Keypoint.draw_on_image() to draw a rectangle for the keypoint so long as any part of that rectangle is within the image plane. (Previously, the rectangle was only drawn if the integer xy-coordinate of the point was inside the image plane.)
  • Changed KeypointsOnImage.draw_on_image() to raise an error if an input image has shape (H,W).
  • Changed KeypointsOnImage.draw_on_image() to handle single-number inputs for color.
  • KeypointsOnImage.from_coords_array()
    • Marked as deprecated.
    • Renamed to from_xy_array().
    • Renamed arg coords to xy.
    • Changed the method from staticmethod to classmethod.
    • Refactored to make code simpler.
  • KeypointsOnImage.get_coords_array()
    • Marked as deprecated.
    • Renamed to to_xy_array().
  • Refactored KeypointsOnImage.draw_on_image() to use Keypoint.draw_on_image().

Heatmap augmentation

  • Changed Affine, PiecewiseAffine, ElasticTransformation to always use order=3 for heatmap augmentation.
  • Changed check in HeatmapsOnImage that validates whether the input array is within the desired value range [min_value, max_value] from a hard exception to a soft warning (with clipping). Also improved the error message a bit.

Deprecation warnings:

  • Added imgaug.imgaug.DeprecationWarning. The builtin python DeprecationWarning is silent since 2.7, which is why now a separate deprecation warning is used.
  • Added imgaug.imgaug.warn_deprecated().
    • Refactored deprecation warnings to use this function.
  • Added imgaug.imgaug.deprecated decorator.
    • Refactored deprecation warnings to use this decorator.

Bounding Boxes:

  • Added to BoundingBox.extract_from_image() the arguments pad and pad_max.
  • Changed BoundingBox.contains() to also accept Keypoint.
  • Changed BoundingBox.project(from, to) to also accept images instead of shapes.
  • Renamed argument thickness in BoundingBox.draw_on_image() to size in order to match the name used for keypoints, polygons and line strings. The argument thickness will still be accepted, but raises a deprecation warning.
  • Renamed argument thickness in BoundingBoxesOnImage.draw_on_image() to size in order to match the name used for keypoints, polygons and line strings. The argument thickness will still be accepted, but raises a deprecation warning.
  • Refactored BoundingBox to reduce code repetition.
  • Refactored BoundingBox.extract_from_image(). Improved some code fragments that looked wrong.
  • Refactored BoundingBoxesOnImage.draw_on_image() to improve efficiency by evading unnecessary array copies.

Other:

  • [rarely breaking] Added arguments cval and mode to PerspectiveTransform (PR #301). This breaks code that relied on the order of the arguments and used keep_size, name, deterministic or random_state as positional arguments.
  • Added dtypes.clip_() function.
  • Added function imgaug.imgaug.flatten() that flattens nested lists/tuples.
  • Changed PerspectiveTransform to ensure minimum height and width of output images (by default 2x2). This prevents errors in polygon augmentation (possibly also in keypoint augmentation).
  • Refactored imgaug.augmenters.blend.blend_alpha() to no longer enforce a channel axis for foreground and background image.
  • Refactored imgaug/parameters.py to reorder classes within the file.
  • Re-allowed numpy 1.16 in requirements.txt.

Fixes

  • Fixed possible crash in blend.blend_alpha() if dtype numpy.float128 does not exist.
  • Fixed a crash in ChangeColorspace when cv2.COLOR_Lab2RGB was actually called cv2.COLOR_LAB2RGB in the local OpenCV installation (analogous for BGR). (PR #263)
  • Fixed ReplaceElementwise always sampling replacement per channel.
  • Fixed an error in draw_text() due to arrays that could not be set to writeable after drawing the text via PIL.
  • Fixed errors in docstring of parameters.Subtract.
  • Fixed a division by zero bug in angle_between_vectors().
  • Augmentation of empty KeypointsOnImage instances
    • Fixed Rot90 not changing KeypointsOnImage.shape if .keypoints was empty.
    • Fixed Affine not changing KeypointsOnImage.shape if .keypoints was empty.
    • Fixed PerspectiveTransform not changing KeypointsOnImage.shape if .keypoints was empty.
    • Fixed Resize not changing KeypointsOnImage.shape if .keypoints was empty.
    • Fixed CropAndPad not changing KeypointsOnImage.shape if .keypoints was empty. (Same for Crop, Pad.)
    • Fixed PadToFixedSize not changing KeypointsOnImage.shape if .keypoints was empty.
    • Fixed CropToFixedSize not changing KeypointsOnImage.shape if .keypoints was empty.
    • Fixed KeepSizeByResize not changing KeypointsOnImage.shape if .keypoints was empty.
  • Fixed Affine heatmap augmentation producing arrays with values outside the range [0.0, 1.0] when order was set to 3.
  • Fixed PiecewiseAffine heatmap augmentation producing arrays with values outside the range [0.0, 1.0] when order was set to 3.
  • Fixed assert in SegmentationMapOnImage falsely checking if max class index is <= nb_classes instead of < nb_classes.
  • Fixed an issue in dtypes.clip_to_value_range_() and dtypes.restore_dtypes_() causing errors when clip value range exceeded array dtype's value range.
  • Fixed an issue in dtypes.clip_to_value_range_() and dtypes.restore_dtypes_() when the input array was scalar, i.e. had shape ().
  • Fixed a Permission Denied error when using JpegCompression on windows (possibly also affected other systems). #297