Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to authenticate using node role #1866

Closed
coufalja opened this issue Aug 3, 2023 · 8 comments · Fixed by #1877
Closed

Unable to authenticate using node role #1866

coufalja opened this issue Aug 3, 2023 · 8 comments · Fixed by #1877
Labels

Comments

@coufalja
Copy link

coufalja commented Aug 3, 2023

Cause and effect:
During the investigation of grafana/tempo#2743 I was able to track down the problem to specific commits in mino-go #1626 and #1682.
Those two together causes us not being able to authenticate, and subsequent Tempo error.

If that is expected behavior (as it seems according to this Cortex commit cortexproject/cortex#4897) the issue lies in the usage of the library.

@parshnt
Copy link

parshnt commented Aug 22, 2023

@coufalja - was there any progress on the issue?

I believe that the changes introduced in #1682 are causing the issue where if IMDSv2 is disabled, an error is raised. whereas earlier the same error was ignored. this is preventing the use of IMDSv1 as well - as the request to generate role creds is made afterwards.

https://github.com/minio/minio-go/pull/1682/files#diff-8a37275f64c96a9a71f11eb6008f45f1f764384f3ae02f685e3013af1d7f79b5L292

@harshavardhana
Copy link
Member

I will check on the fix.

@coufalja
Copy link
Author

I ended up doing the fix in Tempo itself grafana/tempo#2760 as I thought this was an intended behavior.

@parshnt
Copy link

parshnt commented Aug 22, 2023

@harshavardhana - incase of IMDSv2 is disabled. we are seeing an error thrown:
Put "http://169.254.169.254/latest/api/token": context deadline exceeded

I believe #1682 might've introduced this issue

@harshavardhana
Copy link
Member

This is not clearly intended behavior, will fix it.

@parshnt
Copy link

parshnt commented Aug 22, 2023

@harshavardhana - let me know if I can help with the fix, will create a PR.

@harshavardhana
Copy link
Member

@parshnt please look at the PR #1877 and see if this fixes your problem.

@parshnt
Copy link

parshnt commented Aug 22, 2023

@harshavardhana - LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants