Importing task decorator from zappa.asynchronous takes ~2.5s

[texonidas]

As above, simply importing the task decorator adds >2 seconds(!!!) to application start time.

Context

This is not a bug per se, but given the (ideally) snappy nature of lambda, a simple import from Zappa should not be adding this much overhead.

Expected Behavior

Quick

Actual Behavior

Slow

Possible Fix

Not a clue, however the reproduction below seems to indicate sockets are to blame.

Steps to Reproduce

1. Install zappa in fresh venv
2. Create import_zappa.py, containing from zappa.asynchronous import task
3. python -m cProfile -s tottime import_zappa.py > output.log

Truncated output:

         284926 function calls (277992 primitive calls) in 2.489 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    2.018    1.009    2.018    1.009 {method 'connect' of '_socket.socket' objects}
      306    0.109    0.000    0.109    0.000 {built-in method io.open_code}
     2059    0.066    0.000    0.066    0.000 {built-in method nt.stat}
      306    0.025    0.000    0.025    0.000 {built-in method marshal.loads}
    16/14    0.018    0.001    0.019    0.001 {built-in method _imp.create_dynamic}

Your Environment

• Zappa version used: 0.56.1
• Operating System and Python version: Windows 11, Python 2.9.13
• The output of pip freeze:

argcomplete==3.5.0
boto3==1.34.159
botocore==1.34.159
certifi==2024.7.4
cfn-flip==1.3.0
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
durationpy==0.7
hjson==3.1.0
idna==3.7
jmespath==1.0.1
kappa==0.6.0
MarkupSafe==2.1.5
placebo==0.9.0
python-dateutil==2.9.0.post0
python-slugify==8.0.4
PyYAML==6.0.2
requests==2.32.3
s3transfer==0.10.2
six==1.16.0
text-unidecode==1.3
toml==0.10.2
tqdm==4.66.5
troposphere==4.8.1
urllib3==1.26.19
Werkzeug==3.0.3
zappa==0.56.1

[ceturc]

I am able to reproduce your issue on Windows, @texonidas , if I disable my AWS security setup (no profile or explicit keys).

To see what is happening, extend your example.
You'll see attempts to get AWS access credentials that are failing, probably because you aren't setup for AWS access.

import logging
logging.basicConfig(level=logging.DEBUG)
from zappa.asynchronous import task

The async task decorator import triggers an immediate AWS API connection to the Lambda service to keep the async resources warm and available:

Zappa/zappa/asynchronous.py

Lines 108 to 112 in 055db2e

	aws_session = boto3.Session()
	LAMBDA_CLIENT = aws_session.client("lambda")
	SNS_CLIENT = aws_session.client("sns")
	STS_CLIENT = aws_session.client("sts")
	DYNAMODB_CLIENT = aws_session.client("dynamodb")

Your Windows machine is trying to get credentials to connect to AWS during the import. Source: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials

At the end of the list of credential sources, is an attempt to make an HTTP connection to the instance metadata service. This service probably won't exist on your Windows computer. The boto3 library tries, waits, and eventually times out, for your total observed time of about 2 seconds.

urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x0000023A1F522910>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')
DEBUG:botocore.utils:Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/meta-data/iam/security-credentials/: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"

I believe this is an AWS credential setup issue on your workstation, because of how the boto3 library works. I don't believe this is a Zappa-specific issue, because this will only happen on your local laptop if you don't have your AWS credentials configured. This won't happen in the Lambda service because credentials are always available.

To resolve this issue, the easiest way is to configure your AWS access per the instructions at boto3, https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials

Another great source is directly from the Zappa README.md: Before you begin, make sure ... your AWS credentials file is properly installed.