[Visualization/Representation] Different values in part_boxes: datav1.py vs data_snv1.py

[GregorKobsik]

When loading an object with the provided dataloaders I get different number of values for part_boxes.

Using PartGraphShapesDataset from datav1.py the Tensor has 10 values, when using PartGraphShapesDataset from data_snv1.py the Tensor has 12 values.

It seems like the visualization script vis_utils.py expects the boxes to have 12 values.

DSG-Net/code/vis_utils.py

Lines 27 to 31 in 6c6d117

	def draw_box(ax, p, color, rot=None):
	center = p[0: 3]
	lengths = p[3: 6]
	dir_1 = p[6: 9]
	dir_2 = p[9:]

So, what do the 10 values represent and how can I convert them to match the visualization (e.g. vis_pc.ipynb) for the output?

Minimal Code Examples:

from datav1 import PartGraphShapesDataset

dataset = PartGraphShapesDataset('../data/partnetdata/Chair_dgeo', '../data/part_trees/Chair_all_no_other_less_than_10_parts-train', "cpu", 1)

data_item = next(iter(dataset))
obj = data_item[1]

part_boxes, part_geos, edges, part_ids, part_sems = obj.graph(leafs_only=True)

print(part_boxes[0].shape)
print(part_boxes[0])

from data_snv1 import PartGraphShapesDataset, Tree

data_features = ['object', 'name']
dataset = PartGraphShapesDataset('../data/partnetdata/Chair_dhier', 'test.txt', data_features, Tree)

data_item = next(iter(dataset))
obj = data_item[0]

part_boxes, part_geos, edges, part_ids, part_sems = obj.graph(leafs_only=True)

print(part_boxes[0].shape)
print(part_boxes[0])

[GregorKobsik]

Example (Shape No. 2197):

torch.Size([1, 10])
tensor([[-0.4375, -0.4406, -0.2583,  0.0892,  0.0740,  0.0314,  0.6235, -0.3335,
         -0.6235,  0.3335]])

torch.Size([12])
tensor([-4.3752e-01, -4.4511e-01, -2.5151e-01,  3.1384e-02,  6.2768e-02,
         6.6692e-02, -1.0000e+00,  1.8370e-16,  0.0000e+00,  1.8370e-16,
         1.0000e+00,  0.0000e+00])

[GregorKobsik]

It looks like the computet/provided dataset does differ when it comes to rotation and size:
After inspecting the dataloaders, it seems like the first representation uses a quaternion and the second a rotation matrix.

Here a minimal example to load Chair_dgeo data and convert quaternion representation to rotation matrix:

import os
import torch
import numpy as np
from kornia.geometry.conversions import quaternion_to_rotation_matrix

geo_fn = os.path.join('../data/partnetdata/Chair_dgeo', "2197.npz")
geo_data = np.load(geo_fn)

print(geo_data["box_quat"][16])
print(geo_data["box"][16])

p = torch.from_numpy(geo_data["box_quat"][16])
box_quat = p.squeeze()
center = box_quat[:3]
size = box_quat[3:6]
rotmat = quaternion_to_rotation_matrix(box_quat[[7, 8, 9, 6]])
box = torch.cat([center, size, rotmat[:, 0].view(-1), rotmat[:, 1].view(-1)])
print(box.view(-1).numpy())

[-0.43752199 -0.44057664 -0.25829744  0.08923017  0.07400044  0.03138399
  0.62351859 -0.33350348 -0.62351859  0.33350348]
[-0.43752199 -0.44057664 -0.25829744  0.08923017  0.07400044  0.03138399
 -0.          0.8317825   0.55510168 -0.          0.55510168 -0.8317825 ]
[-0.43752199 -0.44057664 -0.25829744  0.08923017  0.07400044  0.03138399
  0.          0.83178248  0.5551017   0.          0.5551017  -0.83178248]

The output seems to confirm my theory. Unfortunately, the provided data in both dataloaders does not match for the boxes. Please, compare the output from data_snv1.py with the "box" values provided by datav1.py.

Is my assumption correct, that the datasets are computed differently and thus have different box sizes and rotations? Thus, cannot be directly compared with each other, nor we can use the visualization script properly?

[GregorKobsik]

Although, the boxes seem to fit the point clouds: