Are the items in KL divergence disordered?

[franciszchen]

Hi, In trainer.py, Line 201-Line208,
for i in range(self.model_num): ce_loss = self.loss_ce(outputs[i], labels) kl_loss = 0 for j in range(self.model_num): if i!=j: kl_loss += self.loss_kl(F.log_softmax(outputs[i], dim = 1), F.softmax(Variable(outputs[j]), dim=1)) loss = ce_loss + kl_loss / (self.model_num - 1)
Is it the same as the Equation 4 in the paper,
L_Θ1 = L_C_1 +D_KL(p2||p1). Thx~

[chxy95]

Yes. You can do experiments to validate it or observe the implementation of the KL loss by Pytorch if you want.