Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the result I get is far away from your paper #2

Open
Xinling-Guo opened this issue Nov 14, 2023 · 0 comments
Open

the result I get is far away from your paper #2

Xinling-Guo opened this issue Nov 14, 2023 · 0 comments

Comments

@Xinling-Guo
Copy link

Hello,
I have read your paper " Meta Learning With Graph Attention Networks for Low-Data Drug Discovery". Your work inspired me a lot. So I downloaded the source code for reproduction. But the results weren't as good as they looked in your paper.
The experimental Settings are plus or minus 10, and the experimental Settings in the source code have not been changed, and the experimental Settings is shown in below
image

And the test result is shown in below, the roc_auc results of the tests for 3 query tasks in the tox21 dataset (the average of 20 test batches across 20 different seeds). The difference between my result and the result of your paper is too great. The roc-auc score of my result is range from 0.74 to 0.77, while the roc-auc score of your paper is larger than 0.9
image
image

Is there anything wrong with my reproduction that led to the bad result? Or is there anything I should pay attention to?

I am looking forward to your reply and I would appreciate it very much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant