Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for GREAT Score [NIPS-2024] #2527

Open
wants to merge 1 commit into
base: dev_1.20.0
Choose a base branch
from

Conversation

lizaitang
Copy link

Description

This pull request adds the support of the BEYOND Detection method proposed in [1].

[1] GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models. Paper

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • [✔️] New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

  • [✔️] Unit Test

Test Configuration:

  • OS: Ubuntu 22.04
  • Python version: 3.9.18
  • ART version: 1.18.1
  • PyTorch: 2.4.0
  • cuDNN version: 90100

Checklist

  • [✔️] My code follows the style guidelines of this project
  • [✔️] I have performed a self-review of my own code
  • [✔️] I have commented my code
  • [✔️] I have made corresponding changes to the documentation
  • [✔️] My changes generate no new warnings
  • [✔️] I have added tests that prove my fix is effective or that my feature works
  • [✔️] New and existing unit tests pass locally with my changes
  • [✔️] My changes have been tested using both CPU and GPU devices

@beat-buesser beat-buesser self-requested a review December 3, 2024 16:43
@beat-buesser beat-buesser self-assigned this Dec 3, 2024
@beat-buesser beat-buesser added the enhancement New feature or request label Dec 3, 2024
Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 0% with 60 lines in your changes missing coverage. Please review.

Project coverage is 53.17%. Comparing base (970c74a) to head (ae33fe1).
Report is 97 commits behind head on dev_1.19.0.

Files with missing lines Patch % Lines
art/evaluations/great_art_test.py 0.00% 33 Missing ⚠️
art/evaluations/great_art.py 0.00% 27 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff               @@
##           dev_1.19.0    #2527      +/-   ##
==============================================
- Coverage       53.38%   53.17%   -0.21%     
==============================================
  Files             658      670      +12     
  Lines           60292    61978    +1686     
  Branches        10274    10523     +249     
==============================================
+ Hits            32186    32956     +770     
- Misses          26450    27325     +875     
- Partials         1656     1697      +41     
Files with missing lines Coverage Δ
art/evaluations/great_art.py 0.00% <0.00%> (ø)
art/evaluations/great_art_test.py 0.00% <0.00%> (ø)

... and 19 files with indirect coverage changes


import numpy as np
import torch
from torch.autograd import Variable

Check notice

Code scanning / CodeQL

Unused import Note

Import of 'Variable' is not used.
great_scores = []
for i in range(len(samples)):
if correct_predictions[i]:
predicted_label = predicted_labels[i]

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable predicted_label is not used.
# SOFTWARE.

from art.defences.evaluation.evasion import great_score
import numpy as np

Check notice

Code scanning / CodeQL

Unused import Note

Import of 'np' is not used.
@beat-buesser beat-buesser added this to the ART 1.20.0 milestone Dec 12, 2024
@beat-buesser beat-buesser changed the base branch from dev_1.19.0 to dev_1.20.0 January 16, 2025 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

2 participants