What are tags used for?

[DJRIZZ]

I'm considering using Bounding Box Editor to annotate a set of images, each image has a large number of objects to annotate. My objective is simple: counting objects in the images in a reproducible way. Using Bounding Box Editor has the added value that the reproducible counts could be used by some future motivated person to potentially automate the counting.

The problem I'm thinking about relates to tagging bounding boxes, which seems to be pretty time intensive when dealing with many bounding boxes, i.e., adding tags for every new bounding box manually (vs. being able to do something like copy and paste an existing bounding box with tags already added and pasting it to a new object that is the same category with the same tags).

One potential solution is to create each tag as a different category of bounding box instead of using a tag on a bounding box. For example, if the object is in in the category "car" and the relevant tag is "occluded" could I just use 2 different object categories, "car occluded" and "car not occluded"?

I have no understanding of the ML/AI process to potentially automate counting and weather using categories with tags vs. different object categories matters to that process so I appreciate any insights.

Thanks.

Dan

[mfl28]

Hi Dan,
glad that you consider using Bounding Box Editor!

• So generally tags can be used to add object instance specific data, as you say e.g. occluded. First of all, to export the tag data alongside the bounding box data you have to use an export format that supports storing tag data. Currently this would be the Pascal-VOC format (this format only considers the pre-defined tags: action: ..., difficult, truncated, occluded and pose: ....) or the JSON format (this format will store all tag values). YOLO format does not allow storing any tag data because of the way the format is defined.
• To maybe speed up the tag annotation a bit, you can draw a bounding box and then hit Ctrl/Cmd+T to add a new tag and paste "occluded". I understand however that this may still be a bit difficult if you have many different tags. I may add a copy feature in a future release.
• I think integrating the "occluded" part into the category name is a viable option (the only option if you want to export to YOLO format), but it may become more difficult to annotate if you have lots of categories and tags. For an object-detection
  task it does not really matter whether the extra data for a category is present as a tag or part of the category in the ground-truth annotations. The data must be extracted and preprocessed anyway for it to used train an object detector.
• For the actual object detection task, it depends on how you want to model the problem. It could be modeled as a multi-class classification task (target would be: category: car/car_occluded) or a multi-label problem where you have an additional binary category for occlusion (i.e. category: car [or not car], occluded: True or False).
• Depending on the object types you want to detect I also strongly suggest you to look into pre-trained models. E.g. there already very good models to detect cars in images. You can use these models with Bounding Box Editor (by setting up and connecting to a Torchserve endpoint, see here - this requires a bit of Python-programming/ML knowledge) and get initial bounding box predictions which you can then edit further for your specific task.

Hope I could help you!

[DJRIZZ]

Thanks for your reply, it is very helpful. I used cars as an example, but I'm are actually annotating aerial photos of ducks, some in flocks of thousands, thus many objects to annotate with bounding boxes. The huge benefit I've found of using Bounding Box Editor is that it does not have any lag when there are thousands of bounding boxes annotating an image, whereas other annotating software apparently holds all annotations in memory and reloads them all with each new bounding box - with thousands of annotations there can be a lag lasting many seconds, which makes them unusable for my objective. Thanks again.