Automatic inclusion of Content-MD5 header breaks S3.upload_part_copy

[ararslan]

Signing requests with signature v4 will automatically, unconditionally insert a Content-MD5 header:

AWS.jl/src/utilities/sign.jl

Line 77 in 3ae5720

"Content-MD5" => base64encode(digest(MD_MD5, request.content)),

AFAICT, AWS does not require the Content-MD5 header for anything but multi-object delete requests, and it's otherwise just noted as being recommended for verifying integrity. There appears to be at least one case where the Content-MD5 header cannot be included, which is in an UploadPartCopy request, exposed by AWS.jl as S3.upload_part_copy. Actually using that function results in the following error response:

<Error>
  <Code>InvalidArgument</Code>
  <Message>The specified header is not valid in this context</Message>
  <ArgumentName>Content-MD5</ArgumentName>
  <ArgumentValue>1B2M2Y8AsgTpgAmY7PhCfg==</ArgumentValue>
  <RequestId>...</RequestId>
  <HostId>...</HostId>
</Error>

Amending sign_aws4! to not add Content-MD5 allows S3.upload_part_copy to succeed. It's not actually surprising that the MD5 checksum of the request contents is invalid in this case, since the body of an UploadPartCopy request is empty; all required information for this kind of request is specified in x-amz-* headers.

Many years ago, before AWS.jl as we know it today came into existence, I briefly worked on moving AWS SigV4 functionality out of HTTP.jl into a separate package, the ill-fated AWSAuth.jl, the contents of which now lives here. However, one feature that was included in AWSAuth.jl but not carried over to AWS.jl is JuliaCloud/AWSAuth.jl@b467cb4, which made inserting the Content-MD5 header optional. We may want to consider something like that, though I'm not sure how exactly we'd want to track internally when to include or exclude it.