Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x-pack/filebeat/input/awss3: support for Access Point ARN #41495

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

x-pack/filebeat/input/awss3: support for Access Point ARN #41495

wants to merge 8 commits into from
+301 −44

Conversation

chemamartinez
Copy link
Contributor

@chemamartinez chemamartinez commented Oct 31, 2024 •
edited
Loading

Proposed commit message

Added a new option access_point_arn to the AWS S3 input as an alternative to the bucket ARN to access S3 buckets.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@chemamartinez chemamartinez added enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team labels Oct 31, 2024
@chemamartinez chemamartinez self-assigned this Oct 31, 2024
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 31, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Oct 31, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @chemamartinez? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit

@mergify Mergify
Copy link
Contributor

mergify bot commented Oct 31, 2024

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Oct 31, 2024
@chemamartinez chemamartinez mentioned this pull request Oct 31, 2024
Open
@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 6, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 41494-awss3-access_point upstream/41494-awss3-access_point
git merge upstream/main
git push upstream 41494-awss3-access_point

@chemamartinez chemamartinez added backport-8.15 Automated backport to the 8.15 branch with mergify backport-8.16 Automated backport with mergify labels Nov 13, 2024
@chemamartinez chemamartinez marked this pull request as ready for review November 14, 2024 09:00
@chemamartinez chemamartinez requested review from a team as code owners November 14, 2024 09:00
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@chemamartinez chemamartinez mentioned this pull request Nov 14, 2024
Open
19 tasks
@chemamartinez chemamartinez requested a review from a team November 14, 2024 13:43
belimawr
belimawr approved these changes Nov 14, 2024
View reviewed changes
Copy link
Contributor

@belimawr belimawr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an expert in the AWS S3 input, but LGTM.

ShourieG
ShourieG approved these changes Nov 18, 2024
View reviewed changes
Copy link
Contributor

@ShourieG ShourieG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@AndersonQ
Copy link
Member

LGTM, just make sure to do not merge the 8.16 backport until 8.16.1 is released.

@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 18, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 41494-awss3-access_point upstream/41494-awss3-access_point
git merge upstream/main
git push upstream 41494-awss3-access_point

kaiyan-sheng
kaiyan-sheng reviewed Nov 18, 2024
View reviewed changes
x-pack/filebeat/input/awss3/config.go
@@ -292,3 +305,11 @@ func (c config) getFileSelectors() []fileSelectorConfig {
}
return []fileSelectorConfig{{ReaderConfig: c.ReaderConfig}}
}

// Helper function to detect if an ARN is an Access Point
func isValidAccessPointARN(arn string) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some unit test for isValidAccessPointARN func?

kaiyan-sheng
kaiyan-sheng reviewed Nov 18, 2024
View reviewed changes
x-pack/filebeat/input/awss3/input.go
accessPointName := strings.Split(arnParts[5], "/")[1]

// Construct the endpoint for the Access Point
endpoint := fmt.Sprintf("%s-%s.s3-accesspoint.%s.amazonaws.com", accessPointName, accountID, region)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this is important for this feature. Could we move the endpoint calculation into a separate function and add some unit tests too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kaiyan-sheng kaiyan-sheng kaiyan-sheng left review comments

@ShourieG ShourieG ShourieG approved these changes

@belimawr belimawr belimawr approved these changes

@AndersonQ AndersonQ AndersonQ approved these changes

At least 1 approving review is required to merge this pull request.

Assignees

@chemamartinez chemamartinez

Labels
backport-8.x Automated backport to the 8.x branch with mergify backport-8.15 Automated backport to the 8.15 branch with mergify backport-8.16 Automated backport with mergify enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[aws]Add support for access point in aws integration
6 participants
@chemamartinez @elasticmachine @AndersonQ @belimawr @kaiyan-sheng @ShourieG