Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot read AWS S3 file with SSO config file #11203

Open
fromgold opened this issue Nov 4, 2024 · 3 comments · May be fixed by #11205
Open

Cannot read AWS S3 file with SSO config file #11203

fromgold opened this issue Nov 4, 2024 · 3 comments · May be fixed by #11205
Assignees

Comments

@fromgold
Copy link

fromgold commented Nov 4, 2024

What is the bug?

In my organization, we recently moved from a single account to a multi account AWS organization.

Before I had my ~/.aws/credentials file as such:

[default]
aws_access_key_id = *********************
aws_secret_access_key = *********************************

With this setup, I could run gdalinfo to a file on s3 simply running:

gdalinfo /vsis3/bucket/path/to/file.tif

Now, in place of the credentials file, I have the ~/.aws/config file as such:

[sso-session my-sso]
sso_start_url = https://mysso.awsapps.com/start/
sso_region = eu-central-1
sso_registration_scopes = sso:account:access
[default]
region = eu-central-1
sso_session = my-sso
sso_account_id = 123456789012
sso_role_name = AdministratorAccess
[profile AdministratorAccess-123456789012]
sso_session = my-sso
sso_account_id = 123456789012
sso_role_name = AdministratorAccess
region = eu-central-1

This way, a normal aws s3 ls command works and lists my buckets, but gdalinfo fails with the error:

ERROR 15: AWS_SECRET_ACCESS_KEY and AWS_NO_SIGN_REQUEST configuration options not defined, and /home/myuser/.aws/credentials not filled
gdalinfo failed - unable to open '/vsis3/bucket/path/to/file.tif

I would expect GDAL to pick up the default profile and use it to authenticate against AWS.
I opened the issue since I did not find any information on this.

Steps to reproduce the issue

To reproduce, you need to:

  • have access to an AWS Single-Sign On instance (AWS IAM Identity Center) with at least one account attached
  • have GDAL installed (i have version 3.9.3)
  • run aws configure sso and follow the steps to configure the sso authentication
  • add a [default] entry manually in the ~/.aws/config file
  • DO NOT have a ~/.aws/credentials file
  • run gdalinfo against a tiff file on a s3 bucket within the organization

Versions and provenance

Self-built GDAL running in Ubuntu-22.04 WSL under Windows 11.
Output of gdalinfo --version:
GDAL 3.9.3, released 2024/10/07 (debug build)

Additional context

No response

@rouault
Copy link
Member

rouault commented Nov 4, 2024

AWS SSO is not currently implemented

@rouault
Copy link
Member

rouault commented Nov 4, 2024

Notes/memo:

  • https://tech.freckle.com/2022/09/16/aws-sso-credentials/
  • python -c "import logging;import boto3; logging.basicConfig(level=logging.DEBUG); logging.getLogger('boto3').setLevel(logging.DEBUG); logging.getLogger('botocore').setLevel(logging.DEBUG); logging.getLogger('botocore.endpoint').setLevel(logging.DEBUG); session = boto3.Session(profile_name='AdministratorAccess-557268267719'); s3 = session.client('s3'); s3.download_file('my_bucket', 'my_file.tif', 's3_my_file.tif')"
  • DEBUG:botocore.endpoint:Sending http request: <AWSPreparedRequest stream_output=False, method=GET, url=https://portal.sso.us-east-1.amazonaws.com/federation/credentials?role_name=AdministratorAccess&account_id=557268267719, headers={'x-amz-sso_bearer_token': b'put here value from the accessToken of ~/.aws/sso/cache/foo.json', 'User-Agent': b'Boto3/1.18.37 Python/3.8.10 Linux/5.15.0-117-generic Botocore/1.27.59', 'amz-sdk-invocation-id': b'686f66a1-2ad8-4835-bdc2-38761c4d3a23', 'amz-sdk-request': b'attempt=1'}>

@rouault rouault self-assigned this Nov 4, 2024
rouault added a commit to rouault/gdal that referenced this issue Nov 4, 2024
@rouault
Copy link
Member

rouault commented Nov 4, 2024

implementation in #11205

rouault added a commit to rouault/gdal that referenced this issue Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants