Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix AWS auth credentials chain and upgrade minio-go #3006

Merged
merged 11 commits into from
Oct 18, 2023

Conversation

mapno
Copy link
Member

@mapno mapno commented Oct 11, 2023

What this PR does:

Fixes AWS authentication for IRSA.

PR #2889 was suppose to fix this but it didn't. Simply switching the chain flow up a bit resolves this issue without the need of if statements makes this all work again, which is what this PR does.

Removes config option native_aws_auth_enabled, as it's no longer necessary. If you were using this option, remove it from your config.

There are images publised for amd64 and arm64 architectures.

Which issue(s) this PR fixes:
Fixes #2888

Checklist

  • Tests updated
  • Documentation removed
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@ekristen
Copy link
Contributor

Looks like this is replacing #2958 -- which is fine with me.

The non-mocked test is doable if you have an AWS account that you can setup with GitHub and setup an Identity Provider in IAM. If not, no biggie.

+1 to all this, please merge as soon as possible.

@ekristen ekristen mentioned this pull request Oct 12, 2023
5 tasks
@joe-elliott
Copy link
Member

See comment for test images: #2958 (comment)

@ekristen
Copy link
Contributor

I can confirm the docker image works as expected. AWS IRSA authentication is good.

Copy link
Member

@jdbaldry jdbaldry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs changes look trivial and good to me.

@coufalja
Copy link
Contributor

coufalja commented Oct 17, 2023

Sorry for the delay but I can confirm that mapno/tempo:aws-cred-fix-amd64 works as expected in our environment. (AWS IMDSv1)

Copy link
Contributor

@knylander-grafana knylander-grafana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for removing doc that wasn't needed any more.

CHANGELOG.md Outdated
@@ -78,6 +78,8 @@ defaults:
* [BUGFIX] Moved empty root span substitution from `querier` to `query-frontend`. [#2671](https://github.com/grafana/tempo/issues/2671) (@galalen)
* [BUGFIX] Correctly propagate ingester errors on the query path [#2935](https://github.com/grafana/tempo/issues/2935) (@joe-elliott)
* [BUGFIX] Fix cases where empty filter {} wouldn't return expected results [#2498](https://github.com/grafana/tempo/issues/2498) (@mdisibio)
* [BUGFIX] Reorder S3 credential chain and upgrade minio-go [#3006](https://github.com/grafana/tempo/pull/3006) (@ekristen, @mapno)
Removes config option `native_aws_auth_enabled`. If you were using this option, remove it from your config.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a breaking change? if yes, can we please mark it as breaking change here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing native_aws_auth_enabled if left in the config will cause a break due to how the config is parsed. I would suggest that perhaps we leave the config option in and write a log message if it's set that it's deprecated and remove it in the next major release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving it in place and saying it's deprecated and is not used wouldn't hurt anything to prevent this from being a breaking change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went with that approach, is nicer for users. Thanks!

Copy link
Member

@joe-elliott joe-elliott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to approve to unblock @mapno. I do agree with the comments that we need to mark a breaking change and it would be nice to parse the field and log a warning, but other than that this LGTM.

Thank you to @ekristen @coufalja @sberz and everyone else who helped us resolve this! Who knew that aws auth was so complicated :)

@mapno mapno enabled auto-merge (squash) October 18, 2023 08:22
@mapno mapno disabled auto-merge October 18, 2023 08:22
@mapno mapno enabled auto-merge (squash) October 18, 2023 08:22
@mapno mapno merged commit ecb05a7 into grafana:main Oct 18, 2023
15 checks passed
@mapno mapno deleted the fix-credentials-and-minio branch October 18, 2023 08:41
wrapCredentialsProvider(&credentials.Static{
Value: credentials.Value{
AccessKeyID: cfg.AccessKey,
SecretAccessKey: cfg.SecretKey.String(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joe-elliott I was able to provide the session token via code. now this isn't longer possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2.2.2 regression: S3 storage does not work with AWS IRSA authentication anymore
8 participants