Skip to content

Use redirect instead of download to use s3#212

Merged
matthiasL-scality merged 2 commits intomainfrom
PTFE-3085-improve-redirect-to-s3
Mar 18, 2026
Merged

Use redirect instead of download to use s3#212
matthiasL-scality merged 2 commits intomainfrom
PTFE-3085-improve-redirect-to-s3

Conversation

@matthiasL-scality
Copy link
Contributor

@matthiasL-scality matthiasL-scality commented Mar 13, 2026

This pull request updates the NGINX configuration to improve how file downloads from the /builds/ path are handled, providing better support for both CLI and browser clients. The main change is the introduction of logic to distinguish between CLI tools and browsers, optimizing download behavior and maintaining compatibility for directory listings and metadata requests.

Improvements to download handling:

  • Added logic to rewrite requests from CLI clients (like curl, wget, aria2c) to /redirect/, so they receive a presigned S3 URL and download files directly, while browser clients continue to be proxied through NGINX via /download/.
  • Ensured that directory listings (requests with a trailing slash) always use the /download/ path, preserving the original behavior for browsers.
  • Updated handling for HEAD requests so that these metadata checks are always proxied directly to /download/, regardless of client type.

@matthiasL-scality matthiasL-scality force-pushed the PTFE-3085-improve-redirect-to-s3 branch 2 times, most recently from 8521df8 to b8e0597 Compare March 13, 2026 09:51
@bertranddemiddelaer
Copy link
Contributor

This one sounds like a good idea, but it will not be fully transparent for some GET implementations which do not support location header by default. Curl will need a '-L' for example. This may require a wrapper or an alias to add a '-L' in a transparent way for our beloved CI customers. The other option is to let them fix all their workflows and calls inside their scripts.

@matthiasL-scality matthiasL-scality force-pushed the PTFE-3085-improve-redirect-to-s3 branch 2 times, most recently from 8025638 to a31d4a4 Compare March 16, 2026 08:18
@matthiasL-scality matthiasL-scality force-pushed the PTFE-3085-improve-redirect-to-s3 branch from a31d4a4 to 09b9f71 Compare March 16, 2026 10:45
@matthiasL-scality matthiasL-scality marked this pull request as ready for review March 17, 2026 09:11
@matthiasL-scality matthiasL-scality requested a review from a team as a code owner March 17, 2026 09:11
Copy link
Contributor

@bertranddemiddelaer bertranddemiddelaer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Beware of those changes however, they will break any CI workflow with artifacts curl calls without '-L'

@matthiasL-scality matthiasL-scality merged commit 287d357 into main Mar 18, 2026
4 checks passed
@matthiasL-scality matthiasL-scality deleted the PTFE-3085-improve-redirect-to-s3 branch March 18, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants