Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AwsSdkPipelineWrapper: Asynchronous task blocked on a synchronous call #2855

Closed
gjunge opened this issue Oct 26, 2024 · 5 comments · Fixed by #2857
Closed

AwsSdkPipelineWrapper: Asynchronous task blocked on a synchronous call #2855

gjunge opened this issue Oct 26, 2024 · 5 comments · Fixed by #2857
Labels
bug Something isn't working community To tag external issues and PRs

Comments

@gjunge
Copy link

gjunge commented Oct 26, 2024

Description
After upgrading to 10.31.0 from 10.26.0 we have seen on a number of occasions that the CPU spiked to almost 100% and stayed there. After windbg-ing we have seen that we have a race condition in the new AwsSdkPipelineWrapper when inserting a new value into the HashSet<string> _unsupportedRequestTypes.
In this one instance we had around 8 threads locked on this call.

image

image

Steps to Reproduce
This happened when we have a [Trace] attribute on a asynchronous method that lists objects from S3.

        [Trace]
        public async Task<IEnumerable<S3Object>> GetFilesInternalAsync(string path)
        {
             var results = new List<S3Object>();
            ListObjectsV2Response s3Response = null;
            do
            {
                s3Response = await _s3client.ListObjectsV2Async(new ListObjectsV2Request() {
                    Prefix = path,
                    BucketName = this.Bucket,
                    ContinuationToken = s3Response?.NextContinuationToken
                }).ConfigureAwait(false);

                results.AddRange(s3Response.S3Objects.Where(f => f.Key != path));

            } while (s3Response.IsTruncated);

            return results;
        }

Your Environment
.NET 4.8.2 , Agent 10.31.0

@gjunge gjunge added the bug Something isn't working label Oct 26, 2024
@workato-integration
Copy link

@tippmar-nr
Copy link
Member

@gjunge Thanks for reporting this issue and for submitting a PR to fix it.

We are reviewing your PR and will likely accept the change via a PR directly on this repo - we need to be able to run our full set of CI workflows, which unfortunately can't run on a fork.

@gjunge
Copy link
Author

gjunge commented Oct 28, 2024

Thanks for the quick response!

@wr-cdargis
Copy link

Just came on here to report this. So glad it's already addressed.

When will this fix be released? @gjunge @tippmar-nr ?

@nrcventura
Copy link
Member

It was included in the 10.33.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working community To tag external issues and PRs
Projects
None yet
4 participants