[QUESTION] Why conflict with sequence parallel?

[jiurizz]

Hi, I have 2 questions about confict with sequence parallel and looking forward to getting a reply, thank you!

1. Commit 221a Support sequence-parallel for zero bubble schedule #14 fixed the conflict, but I still want to confirm again. The reason why the code conflicts with sequence parallel is because under sequence parallel, the backward of RowParallelLinear uses the global buffer for all gather and the buffer is likely to be refreshed by other data?
2. If so, why commit 221a Support sequence-parallel for zero bubble schedule #14 also all_gather the grad_output independently? Grad_output is gathered in a tensor created by torch.empty(), not from global buffer.