0.14.0-rc.1

This release refactors the implementation for serialization. All serializers now directly implement serde::Serializer instead of a custom trait. There are no breaking changes to the public interface, except for a polish of error messages.

• Performance improvements:
  • Avoid unncessary allocations in serialization
  • Reserve elements up front
• Improved error reporting in serialization:
  • The field and data type is now reported more consistently across builders
• More consistent handling of the serde data model in serialization:
  • Tuple variants, tuple structs can be used where tuples could be used
  • Treat newtype variants as transparents wrappers
  • Struct variants can be used where structs could be used
  • Support tuples, tuple, structs, tuple variants, bytes for FixedSizeList
  • Allow to serialize sequences for structs, similar to tuples
• Add options to consume the builder ArrayBuilder::into_arrow, ArrayBuilder::into_record_batch, ArrayBuilder::into_marrow that avoid additional allocations for metadata

0.13.7

• @lonless9: Add 57 support + Release 0.13.7 (#287)

0.13.6

• @jpopesculian added the option bytes_as_large_binary to TracingOptions (#282)
• @jpopesculian added support to serialize strings as bytes (#281)

0.13.5

• Add arrow=56 support

0.13.4

• Allow dictionaries with nullable values to be deserialized, if all values are valid

Thanks

The following people contributed to this release:

• @ryzhyk improved the null checks in dictionary deserialization (#277)

0.13.3

• Fix: Use the correct lower bound for marrow to enable arrow=55

0.13.2

• Add arrow=55 support
• Allow primitive to string conversion (#272)
  • Adds the option allow_to_string to TracerOptions to allow numbers, booleans and chars to coerce to strings (false by default)
  • Allows for numbers, booleans and chars to be serialized to strings

Thanks

The following people contributed to this release:

• @jpopesculian added support for serializing primitives to string (#272)

0.13.1

• Allow to use enums / unions in nullable structs. The arrow format requires that each child field of a null struct value contains a dummy value. For union fields such dummy values were not supported before. With this release the following dummy values for unions are used:
  • For unions, the first variant is used as a dummy
  • For dictionary encoded strings, the first non-null value is used as a dummy value or an empty string if no values are encountered

Thanks

The following people contributed to this release:

• @bartek358 discovered that unions in nullable structs are not supported and fixed the implementation in (#265)

0.13.0

• Migrate internal array abstraction to marrow
  • Breaking change: Dictionary data types no longer support sorting
  • Breaking change: serde_arrow::Error no longer implements From<arrow::error::ArrowError>
  • Breaking change: serde_arrow::Error no longer implements From<arrow2::error::Error>
• Add support for view types Utf8View, BytesView
• Add APIs to interact with marorw arrays directly. Allows to use serde_arrow with different arrow versions at the same time.
• Fix Date64 semantics: use Date64 exclusively for dates, and Timestamp for date times
  • Trace date time strings as Timestamp(Millisecond, tz) with tz either being None or Some("UTC")
  • Remove the UtcAsDate64Str and NaiveAsDate64Str strategies
• Fix bug in deserialization of sub seconds for Time32 and Time64
• Fix bug that prevented to deserialize String from Decimal arrays
• Improve performance when deserializing from sliced arrays
• Allow to treat deserializers as sequence of deserializers by iterating over them or accessing individual items

Thanks

The following people contributed to this release:

• @ryzhyk discovered and fixed a bug introduced during refactoring (#261)

0.13.0-rc.2

• Migrate internal array abstraction to marrow
  • Breaking change: Dictionary data types no longer support sorting
  • Breaking change: serde_arrow::Error no longer implements From<arrow::error::ArrowError>
  • Breaking change: serde_arrow::Error no longer implements From<arrow2::error::Error>
• Add support for view types Utf8View, BytesView
• Add APIs to interact with marorw arrays directly
• Fix Date64 semantics: use Date64 exclusively for dates, and Timestamp for date times
  • Trace date time strings as Timestamp(Millisecond, tz) with tz either being None or Some("UTC")
  • Remove the UtcAsDate64Str and NaiveAsDate64Str strategies
• Fix bug in deserialization of sub seconds for Time32 and Time64
• Fix bug that prevented to deserialize String from Decimal arrays
• Improve performance when deserializing from slide arrays
• Allow to treat deserializers as sequence of deserializers by iterating over them or accessing individual items

Thanks

The following people contributed to this release:

• @ryzhyk discovered and fixed a bug introduced during refactoring (#261)