v0.3.2 · lakehq sail · Discussion #724 · GitHub

linhr
Aug 8, 2025
Maintainer

Overview

You can find the release summary in the documentation.

The PySail Python package is available on PyPI.

What's Changed

fix: release notes download by @linhr in fix: release notes download #605
deps(npm): bump the all group with 7 updates by @dependabot[bot] in deps(npm): bump the all group with 7 updates #607
deps(npm): bump vue-tsc from 2.2.10 to 3.0.1 in the breaking group by @dependabot[bot] in deps(npm): bump vue-tsc from 2.2.10 to 3.0.1 in the breaking group #608
chore: bump datafusion by @shehabgamin in chore: bump datafusion #611
fix: date_part('seconds', col) inconsistent type and value by @SparkApplicationMaster in fix: date_part('seconds', col) inconsistent type and value #609
docs: update brand colors by @linhr in docs: update brand colors #612
support nullifzero and zeroifnull by @jamesfricker in support nullifzero and zeroifnull #613
fix: resolve options for external table read by @linhr in fix: resolve options for external table read #592
refactor: move catalog code to a separate crate by @linhr in refactor: move catalog code to a separate crate #615
fix: array_contains inconsistent null logic by @SparkApplicationMaster in fix: array_contains inconsistent null logic #619
fix: array_position inconsistent result and internal_error in some cases by @SparkApplicationMaster in fix: array_position inconsistent result and internal_error in some cases #621
fix: array_append and array_prepend inconsistent NULL behaviour by @SparkApplicationMaster in fix: array_append and array_prepend inconsistent NULL behaviour #623
chore: address clippy lints by @lonless9 in chore: address clippy lints #624
deps(cargo): bump the all group with 4 updates by @dependabot[bot] in deps(cargo): bump the all group with 4 updates #625
deps(npm): bump the all group with 7 updates by @dependabot[bot] in deps(npm): bump the all group with 7 updates #626
fix: inconsistent size, array_size and cardinality logic by @SparkApplicationMaster in fix: inconsistent size, array_size and cardinality logic #617
refactor: data source format by @lonless9 in refactor: data source format #628
feat: basic read/write operations for Delta Lake by @lonless9 in feat: basic read/write operations for Delta Lake #578
feat: add from_csv udf function by @rafafrdz in feat: add from_csv udf function #529
feat: add functions math by @davidlghellin in feat: add functions math #580
fix: codec for from_csv by @davidlghellin in fix: codec for from_csv #633
fix: array_agg null values treatment by @SparkApplicationMaster in fix: array_agg null values treatment #640
feat: bit_count, bit_get, shiftrightunsigned, unary_not bitwise functions by @SparkApplicationMaster in feat: bit_count, bit_get, shiftrightunsigned, unary_not bitwise functions #645
perf: collect_set and array_agg functions by @SparkApplicationMaster in perf: collect_set and array_agg functions #644
fix: flatten function inconsistent null behaviour by @SparkApplicationMaster in fix: flatten function inconsistent null behaviour #642
fix: arrays_overlap function behaviour on inputs containing NULL by @SparkApplicationMaster in fix: arrays_overlap function behaviour on inputs containing NULL #643
fix: case when else expr inconsistent eager evaluation by @SparkApplicationMaster in fix: case when else expr inconsistent eager evaluation #649
feat: add array_insert function by @SparkApplicationMaster in feat: add array_insert function #638
fix: concat function behaviour on some subset of input types by @SparkApplicationMaster in fix: concat function behaviour on some subset of input types #647
feat: support Azure and GCS for object store by @shehabgamin in feat: support Azure and GCS for object store #616
fix: maxRecordsPerBatch negative and zero value by @SparkApplicationMaster in fix: maxRecordsPerBatch negative and zero value #652
feat: listagg function by @SparkApplicationMaster in feat: listagg function #654
chore: update cargo deps by @shehabgamin in chore: update cargo deps #656
feat: support Spark URL functions by @anhvdq in feat: support Spark URL functions #539
chore: rewrite cardinality, size, array_size, deep_size functions using pure datafusion exprs by @SparkApplicationMaster in chore: rewrite cardinality, size, array_size, deep_size functions using pure datafusion exprs #660
feat: bitmap scalar functions by @SparkApplicationMaster in feat: bitmap scalar functions #661
fix: pysparktableudf behaviour on empty partitions with no recordbatches by @SparkApplicationMaster in fix: pysparktableudf behaviour on empty partitions with no recordbatches #658
feat: add to_number function by @rafafrdz in feat: add to_number function #629
deps(npm): bump the all group with 9 updates by @dependabot[bot] in deps(npm): bump the all group with 9 updates #663
deps(cargo): bump tokio from 1.46.1 to 1.47.0 in the all group by @dependabot[bot] in deps(cargo): bump tokio from 1.46.1 to 1.47.0 in the all group #664
fix: map_function DataType::Null inputs by @SparkApplicationMaster in fix: map_function DataType::Null inputs #666
fix: column output naming by @SparkApplicationMaster in fix: column output naming #668
feat: catalog manager and provider redesign by @linhr in feat: catalog manager and provider redesign #632
feat: partitioned writes and data skipping for Delta Lake by @lonless9 in feat: partitioned writes and data skipping for Delta Lake #634
fix: trim, ltrim. rtrim reversed arguments by @SparkApplicationMaster in fix: trim, ltrim. rtrim reversed arguments #674
feat: to_utc_timestamp function by @SparkApplicationMaster in feat: to_utc_timestamp function #676
refactor: pass url directly for Delta Lake by @lonless9 in refactor: pass url directly for Delta Lake #677
fix: position of the sign in to_number function by @rafafrdz in fix: position of the sign in to_number function #672
feat: add functions math by @davidlghellin in feat: add functions math #635
chore: Upgrade to DataFusion 49.0.0 by @shehabgamin in chore: Upgrade to DataFusion 49.0.0 #653
fix: parquet configs by @shehabgamin in fix: parquet configs #681
fix: change div by mul in try_mult by @davidlghellin in fix: change div by mul in try_mult #683
fix: cast expr renaming by @SparkApplicationMaster in fix: cast expr renaming #685
fix: dataframe.na.drop() by @SparkApplicationMaster in fix: dataframe.na.drop() #693
fix: timestamp and interval cast from and to numeric by @SparkApplicationMaster in fix: timestamp and interval cast from and to numeric #691
feat: try_cast column method by @SparkApplicationMaster in feat: try_cast column method #694
fix: avg func exception on NullType input by @SparkApplicationMaster in fix: avg func exception on NullType input #701
chore: math functions refactor by @SparkApplicationMaster in chore: math functions refactor #699
feat: consolidate write plan resolver by @linhr in feat: consolidate write plan resolver #688
deps(cargo): bump the all group with 3 updates by @dependabot[bot] in deps(cargo): bump the all group with 3 updates #703
deps(npm): bump the all group with 6 updates by @dependabot[bot] in deps(npm): bump the all group with 6 updates #704
fix: session config options for target_partitions and collect_statistics by @shehabgamin in fix: session config options for target_partitions and collect_statistics #705
docs: update storage guide by @linhr in docs: update storage guide #706
feat: monthname, weekofyear funcs, refactor datetime funcs by @SparkApplicationMaster in feat: monthname, weekofyear funcs, refactor datetime funcs #702
feat: arrays_zip by @SparkApplicationMaster in feat: arrays_zip #698
fix: resolver logic for overwrite conditions by @linhr in fix: resolver logic for overwrite conditions #707
feat: spark_to_chrono_fmt and to_unix_timestamp funcs by @SparkApplicationMaster in feat: spark_to_chrono_fmt and to_unix_timestamp funcs #708
feat: file listing and statistics cache by @shehabgamin in feat: file listing and statistics cache #709
feat: update cache config by @linhr in feat: update cache config #712
feat: try_validate_utf8, validate_utf8, is_valid_utf8 funcs by @SparkApplicationMaster in feat: try_validate_utf8, validate_utf8, is_valid_utf8 funcs #713
feat: schema evolution for Delta Lake by @lonless9 in feat: schema evolution for Delta Lake #680
fix: clippy 1.89 warnings by @SparkApplicationMaster in fix: clippy 1.89 warnings #720
fix: resolve test path handling on Windows by @lonless9 in fix: resolve test path handling on Windows #716
feat: make_valid_utf8 func by @SparkApplicationMaster in feat: make_valid_utf8 func #719
docs: add guide for Delta Lake by @linhr in docs: add guide for Delta Lake #717
chore: prepare v0.3.2 by @linhr in chore: prepare v0.3.2 #714
feat: support Delta codec by @linhr in feat: support Delta codec #723
chore: prepare v0.3.2 by @linhr in chore: prepare v0.3.2 #721

New Contributors

@jamesfricker made their first contribution in support nullifzero and zeroifnull #613
@anhvdq made their first contribution in feat: support Spark URL functions #539

Full Changelog: v0.3.1...v0.3.2

This discussion was created from the release v0.3.2.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment