Add Schema Migration System for ZIO Schema 2 #723
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…chema-migration The issue 519 depends on issue 517 (that this pr resolves)
- switch to DynamicMigration(Vector[MigrationAction]) core - refactor interpreter to execute ordered actions - introduce SchemaExpr-based defaults (DefaultValueExpr) - thread source/target schemas through execution - add selector-based DSL ops (rename/add/drop field) - clean up registry and id usage
- fix selector-based DSL ops - extend migration action algebra - expand interpreter support and schema threading - evaluate SchemaExpr via evalDynamic with proper inputs - add build/buildPartial split and validation hooks - add migration law test scaffolding
…aligning with zio#519 - Embed defaults as evaluatable SchemaExpr (MigrationSchemaExpr.DefaultValue) instead of DefaultValueExpr marker - Remove runtime schema plumbing (sourceSchema/targetSchema, defaultForField) from DynamicMigrationInterpreter - Simplify SchemaExpr evaluation (drop defaultProvider + DefaultValueExpr special-casing) - Make Mandate/Optionalize reversible without inventing defaults by carrying default expressions explicitly - Adjust AddField/DropField reverses to preserve default expressions for round-trip reversibility - Update nested interpreter calls to use schema-free DynamicMigrationInterpreter(m, value) - Keep schemas in typed Migration[A, B] for encode/decode only; dynamic core remains pure and introspectable - General cleanups / minor interpreter correctness fixes (e.g., RenameCase payload preservation)
zio#519 Refactor the migration subsystem to match the zio#519 design: make the core ADT, macro-based DSL, interpreter, and validator consistent, pure, and selector-driven (DynamicOptic + SchemaExpr only). - Align MigrationAction ADT with zio#519 (remove legacy actions, duplicates, and parent+field shapes) - Update MigrationDsl macros to emit real actions using field optics (Rename, AddField, DropField) - Extend DynamicMigrationInterpreter.modifyAt to support traversal optics (.each / Elements, .when[Case], mapKeys, mapValues) - Ensure DefaultValue is schema-backed and captured at macro time for reversible migrations - Add structural validation for migration actions (field paths, case selectors, empty joins/splits) This brings migrations to a pure-data, selector-first model that is serializable, reversible (where possible), and compatible with advanced selectors.
…Type; improve DSL/validation Make DefaultValue a pure marker (MigrationSchemaExpr.DefaultValue) and resolve it in the interpreter via target schema defaults (no Schema captured in migration programs) Thread sourceSchema + targetSchema through Migration.apply -> DynamicMigrationInterpreter to support default resolution and better runtime checks Implement Join, Split, and ChangeType in DynamicMigrationInterpreter and enforce primitive→primitive constraints for these operations Fix Rename.reverse to be structurally invertible by swapping field names and rebuilding the optic path Expand typed MigrationDsl with missing builder ops: transformField, mandateField, optionalizeField, changeFieldType, joinFields, splitField, renameCase, transformCase Fix dropField macro to stop referencing a non-existing helper and use DefaultValue marker for reverse Upgrade MigrationValidator to catch common invalid programs early (field-path shape checks, join/split empties, case selector requirements)
…e API with zio#519 - Capture field-level schema defaults in macros instead of resolving defaults at runtime - Replace stub migration validation with schema-based DynamicOptic path checks - Ensure build validates migrations while buildPartial skips validation - Expose DynamicMigration.apply for direct DynamicValue execution - Keep migration programs pure, reversible, and interpreter-driven
- Ensured actions continue at the renamed field path - Added .each and .when[Case] selector support - Enabled correctly scoped enum/case migrations - Makes MigrationBuilder exist on Scala 2 only to fail clearly, not silently.
… MigrationBuilderMacros of scala-3 into separated file
…e coverage, fix compilation errors
Author
|
I expected CI will pass after implementation+test successfully compiled locally, will reopen once fix CI |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello team, this closes #519
Built on the following PRs (that already considered to be merged):
Into[A, B]andAs[A, B]Type Classes with Macro Derivation #518, as it's prerequisite of Add structural schemas #517)My edit/files changes are only:
zio-blocks/schema/shared/src/main/scala/zio/blocks/schema
zio-blocks/schema/shared/src/main/scala/zio/blocks/schema/migration
zio-blocks/schema/shared/src/main/scala3/zio/blocks/schema/migration
zio-blocks/schema/shared/src/test/migration