I am puzzled about your comment ”Bias is definitely not crucial to GAT“

[AnotherYx]

In pytorch-GAT-main/models/definitions/GAT.py, lines 97 to 100, we initialize the tensor for bias. Then, in line 123, we further initialize the value of bias using self.init_params(layer_type). However, the function we use is torch.nn.init.zeros_(), which sets all elements in bias to 0. Could this be the reason for the comment, "Bias is definitely not crucial to GAT"? Perhaps we should consider using torch.nn.init.normal_() or another initialization function instead?