-
Notifications
You must be signed in to change notification settings - Fork 4.4k
feat(s3-deployment): add contentEncodingByExtension property for per-file-extension encoding #36763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
aws-cdk-automation
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pull request linter fails with the following errors:
❌ Features must contain a change to an integration test file and the resulting snapshot.
If you believe this pull request should receive an exemption, please comment and provide a justification. A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed, add Clarification Request to a comment.
|
This PR is going to have impact on existing deployment. I think we should consider #36790 before we are able to continue this PR to minimize the risk on existing deployments. |
- Add ContentEncodingByExtension property to enable file-specific Content-Encoding headers based on extension patterns - Implement s3_deploy_with_encoding_by_extension function to handle multiple sync commands with different encodings - Update bucket deployment handler to parse and pass content encoding configuration - Modify sync logic to exclude encoded files from main sync and apply per-extension encoding in separate syncs - Add comprehensive documentation and examples for the new content encoding feature - Include test coverage for content encoding by extension scenarios - Allows users to apply different Content-Encoding headers (e.g., gzip, brotli) to files matching specific patterns during deployment
…puts - Add validateContentEncodingByExtension() method to validate encoding values and patterns - Validate encoding values against standard HTTP Content-Encoding values (br, gzip, compress, deflate, identity, zstd) - Validate include patterns contain only safe glob characters and reject path traversal sequences - Add comprehensive test coverage for invalid encoding values, invalid pattern characters, and path traversal attempts - Add tests verifying valid patterns with glob characters and case-insensitive encoding values - Improve security by preventing injection attacks through pattern validation
- Switch from allowlist to blocklist for pattern validation (allows glob patterns like brackets/braces) - Add comprehensive path traversal detection with normalized path segments - Non-standard encodings now warn instead of error, vendor extensions (x-*) accepted - Add array-level and object-level token checks for better CDK Token handling - Add warning when catch-all patterns used with contentEncoding property - Add runtime validation in Python handler as defense-in-depth
- Update binary asset snapshots for bucket deployment integration tests - Reorganize asset file paths to consolidate common dependencies - Refresh CloudFormation templates and manifests across all test suites - Update tree.json and assets.json files to reflect latest deployment configurations - Consolidate Python and JavaScript handler assets across multiple test scenarios
|
|
||||||||||||||
|
|
||||||||||||||||||||||||||
Issue # (if applicable)
Closes #7090.
Reason for this change
Users serving pre-compressed static websites from S3+CloudFront need to set different
Content-Encodingheaders based on file extension. Currently, thecontentEncodingproperty applies a single encoding to ALL files in the deployment, which breaks browser decompression when serving mixed content (e.g.,.brfiles with Brotli,.gzfiles with Gzip, and uncompressed files).The current workaround requires creating multiple
BucketDeploymentconstructs withprune: falseand complexexclude/includefilters, resulting in:This feature has strong community interest with 19 👍 reactions on the issue (open since March 2020).
Description of changes
Added a new
contentEncodingByExtensionproperty toBucketDeploymentPropsthat accepts an array of file pattern to encoding mappings. The Lambda handler executes multipleaws s3 synccommands with appropriate--include/--excludefilters and--content-encodingvalues.Key changes:
ContentEncodingMappinginterface withinclude(file pattern) andencoding(Content-Encoding value) propertiescontentEncodingByExtensionoptional property toBucketDeploymentPropsUsage example:
Files modified:
packages/aws-cdk-lib/aws-s3-deployment/lib/bucket-deployment.tsContentEncodingMappinginterface andcontentEncodingByExtensionpropertypackages/@aws-cdk/custom-resource-handlers/lib/aws-s3-deployment/bucket-deployment-handler/index.pypackages/aws-cdk-lib/aws-s3-deployment/test/bucket-deployment.test.tspackages/aws-cdk-lib/aws-s3-deployment/README.mdDescribe any new or updated permissions being added
N/A - No IAM permission changes. The Lambda handler continues to use the same S3 permissions it already has for the
s3 syncoperation.Description of how you validated changes
contentEncodingByExtensionproperty is passed to custom resourcecontentEncodingByExtensionwith single mappingcontentEncodingByExtensionis not set when not specifiedcontentEncodingByExtensioncan be used with other system metadatacontentEncodingByExtensionwith directory patternChecklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license