Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blob.download_to_filename does not work #1751

Open
Menghetz opened this issue Sep 22, 2024 · 2 comments
Open

Blob.download_to_filename does not work #1751

Menghetz opened this issue Sep 22, 2024 · 2 comments

Comments

@Menghetz
Copy link

I am having everytime my python scripts uses the blob.download_to_filename method.
I am running the fake-gcs-server using this command

docker run --rm -p 4443:4443 fsouza/fake-gcs-server -scheme http

Here below is my Python script

from google.auth.credentials import AnonymousCredentials
from google.cloud import storage
from pathlib import Path

client = storage.Client(
    credentials=AnonymousCredentials(),
    project="test",
    client_options={"api_endpoint": "http://127.0.0.1:4443"},
)
bucket = client.bucket("test-bucket")
# Create the bucket in the fake GCS server
if not bucket.exists():
    client.create_bucket("test-bucket")

# Upload all txt files from tests/data folder to the fake GCS server
data_folder = Path('tests/data')
txt_files = list(data_folder.glob('*.txt'))

for txt_file in txt_files:
    blob = bucket.blob(f'folder/{txt_file.name}')
    with open(txt_file, 'r') as f:
        blob.upload_from_string(f.read())
    print(f"Uploaded {txt_file.name} to fake GCS bucket.")

# List the Buckets
for bucket in client.list_buckets():
    print(f"Bucket: {bucket.name}\n")
    # List the Blobs in each Bucket
    for blob in bucket.list_blobs():
        print(f"Blob: {blob.name}")

        # Print the content of the Blob
        b = bucket.get_blob(blob.name)
        s = b.download_to_filename('test.json')
        print(f"Downloaded `{blob.name}")

And these are the execution logs

Uploaded file1.txt to fake GCS bucket.
Uploaded file2.txt to fake GCS bucket.
Bucket: test-bucket

Blob: folder/file1.txt

I can correctly upload files into the server, list the buckets and get blob info. But the code is stucked at download_to_filename

Here below is the error I am getting

requests.exceptions.ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=4443): Max retries exceeded with url: /download/storage/v1/b/test-bucket/o/folder%2Ffile1.txt?alt=media (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000002B43E8FA120>: Failed to establish a new connection: [WinError 10049] The requested address is not valid in its context'))

Any idea about what the error could be?

@fsouza
Copy link
Owner

fsouza commented Sep 22, 2024

Can you share the server logs?

@Menghetz
Copy link
Author

Sure! Here they are.

C:\Users\matte>docker logs 1d70fe6796e0
time=2024-09-22T15:20:55.940Z level=INFO msg="server started at http://0.0.0.0:4443"
time=2024-09-22T15:21:05.211Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"GET /storage/v1/b/tests?fields=name&prettyPrint=false HTTP/1.1\" 404 119\n"
time=2024-09-22T15:21:05.223Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"POST /storage/v1/b?project=test&prettyPrint=false HTTP/1.1\" 200 333\n"
time=2024-09-22T15:21:05.243Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"POST /upload/storage/v1/b/tests/o?uploadType=multipart HTTP/1.1\" 200 791\n"
time=2024-09-22T15:21:05.261Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"POST /upload/storage/v1/b/tests/o?uploadType=multipart HTTP/1.1\" 200 790\n"
time=2024-09-22T15:21:05.264Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"GET /storage/v1/b?project=test&projection=noAcl&prettyPrint=false HTTP/1.1\" 200 370\n"
time=2024-09-22T15:21:05.267Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"GET /storage/v1/b/tests/o?projection=noAcl&prettyPrint=false HTTP/1.1\" 200 1618\n"
time=2024-09-22T15:21:05.270Z level=INFO msg="172.17.0.1 - - [22/Sep/2024:15:21:05 +0000] \"GET /storage/v1/b/tests/o/folder%2Ffile1.txt?projection=noAcl&prettyPrint=false HTTP/1.1\" 200 612\n"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants