Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checksum notes rebase #2826

Merged
merged 11 commits into from
Jan 26, 2024
Merged

Checksum notes rebase #2826

merged 11 commits into from
Jan 26, 2024

Conversation

jmklix
Copy link
Member

@jmklix jmklix commented Jan 25, 2024

Issue #, if available:
#2644

Description of changes:
Fix checksum behavior when precalculated checksum is added

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

src/aws-cpp-sdk-core/include/aws/core/utils/crypto/Hash.h Outdated Show resolved Hide resolved
@@ -264,6 +265,7 @@ bool AWSAuthV4Signer::SignRequest(Aws::Http::HttpRequest& request, const char* r
{
payloadHash = STREAMING_UNSIGNED_PAYLOAD_TRAILER;
Aws::String trailerHeaderValue = Aws::String("x-amz-checksum-") + request.GetRequestHash().first;
request.DeleteHeader(trailerHeaderValue.c_str());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is trailer being deleted?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a good name for the variable re-naming to "checksumHeaderValue". We are removing the checksum header from being sent as it is part of the trailer in this branch of logic. The bug beforehand was that a user would add the checksum to the request, it would serialized as a header AND calculated as part of the request. thus my comment on the original PR

we should consider not serializing the header x-amz-checksum- to the header in the first place during codegen as we will always end up adding it one way or another if needed. this is fine for now imo opinon, but removing the serialization could perhaps be a better long term solution

so when sent there would be a header and a trailer, which is wrong.

if (checksumAlgorithmName == "crc32")
if (request.IsStreaming() && checksumValueAndAlgorithmProvided)
{
auto hash = Aws::MakeShared<Crypto::PrecalculatedHash>(AWS_CLIENT_LOG_TAG,request.GetHeaders().find(checksumType)->second);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: spacing inconsistent, do we need to run lint?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed spacing, we really should have a clang format file. We've discussed it multiple times and i think the only thing stopping us is the one time format the world event. Im very much pro a format the world event though. but we should start thinking about adding one sooner than later.

src/aws-cpp-sdk-core/include/aws/core/utils/crypto/Hash.h Outdated Show resolved Hide resolved
// Check if user has provided a checksum value for the specified algorithm
const Aws::String checksumType = "x-amz-checksum-" + checksumAlgorithmName;
const HeaderValueCollection &headers = request.GetHeaders();
bool checksumValueAndAlgorithmProvided = headers.find(checksumType) != headers.end();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure that checksumAlgorithmName always contains a lowered-case string?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes because its serialized as such from the request. which actually relates to dmitrys quesiton about removing the header when adding the trailing checksum. Serializing the checksum as part of the request as a header is one of the issues that got us here.

src/aws-cpp-sdk-core/source/client/AWSClient.cpp Outdated Show resolved Hide resolved
@jmklix jmklix linked an issue Jan 26, 2024 that may be closed by this pull request
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PutObject reads underlying stream twice (even with PayloadSigningPolicy::Never)
4 participants