Questions about Task2

[cuteyyt]

Dear Organizers:
Thank you very much for your hard work in organizing this challenge. I have three questions about Task 2:

1. What are the training data and validation data used in Task 2? Are they the same as Task 1?
2. Is there an official federate split (data partition for federated training)?
3. How can you judge whether the model we submitted being trained in a federated way? Since the paper says that "Note that training on pooled data will be allowed in this task, so that the participants can develop methods that optimally exploit the meta-information of data origin." Does this mean training the model with datasets jointed together directly is also allowed in task 2, participants are not restricted to federated learning, but only the test will be done in a federated way?
   Could you please kindly answer the questions above when you are free? Thanks a lot!

[mzenk]

Hi @cuteyyt,
Thank you for your questions and your interest in our challenge! I'll try to answer them below, but feel free to ask more questions if anything remains unclear!

1. Yes, they are the same. Regarding the partitioning files, the "natural" may be more relevant because at test-time, as we'll also rank the submissions on a per-institution basis before aggregating. But you're free to use all of the provided information.
2. Two partitionings are provided for the training data: A natural partitioning by institution and a (semi-)artificial partitioning by institution and tumor size for the largest institutions. Does that answer your question?
3. You read correctly, we will not check how the networks were trained in task 2. Methodically, task 2 is more similar to a domain generalization problem and the federation allows us to assess the performance on new "domains" (institutions).

(As a side note, there is also a "discussions" page in this repo, where you can find more questions from other participants. Sorry for the confusion!)

[cuteyyt]

Thank you for your patient answer!

[mzenk]

Glad I could help! :)