Skip to content

(PTFE-3070) Add multi part upload for artifact#214

Draft
matthiasL-scality wants to merge 2 commits intomainfrom
PTFE-3070-migrate-to-multi-part-upload
Draft

(PTFE-3070) Add multi part upload for artifact#214
matthiasL-scality wants to merge 2 commits intomainfrom
PTFE-3070-migrate-to-multi-part-upload

Conversation

@matthiasL-scality
Copy link
Contributor

This pull request introduces multipart upload support for artifacts, allowing large files to be uploaded in multiple parts to S3, and adds end-to-end tests for these new endpoints. The changes span Nginx configuration, Lua signature computation logic, access restrictions, and comprehensive test coverage.

Multipart upload feature additions:

  • Added four new Nginx locations in nginx.conf.template to handle multipart upload lifecycle: initiate, upload part, complete, and abort. Each endpoint proxies requests to S3 with proper validation and signature computation.

  • Updated the Lua S3 signature computation logic to support multipart operations, including explicit resource canonicalization for each multipart endpoint and signature mode handling. [1] [2] [3]

Access control updates:

  • Extended GitHub upload restriction paths in github_access.lua to include /upload-multipart/, ensuring proper access control for the new endpoints.

Testing improvements:

  • Added a new end-to-end test suite in test_multipart_upload.py covering multipart upload scenarios: single-part, multi-part, abort, out-of-order part uploads, and validation for non-staging builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant