v0.4.0

What's Changed

• io: add support for role arn and external id s3 props by @mattheusv in #553
• fix: ensure S3 and GCS integ tests are conditionally compiled only when the storage-s3 and storage-gcs features are enabled by @sdd in #552
• docs: fix main iceberg example by @jdockerty in #554
• io: add support to set assume role session name by @mattheusv in #555
• test: refactor datafusion test with memory catalog by @FANNG1 in #557
• chore: add clean job in Makefile by @ChinoUkaegbu in #561
• docs: Fix build website permission changed by @Xuanwo in #564
• Object Cache: caches parsed Manifests and ManifestLists for performance by @sdd in #512
• Update the paths by @Fokko in #569
• docs: Add links for released crates by @Xuanwo in #570
• Python: Use hatch for dependency management by @sungwy in #572
• Ensure that RestCatalog passes user config to FileIO by @sdd in #476
• Move zlib and unicode licenses to allow by @Fokko in #566
• website: Update links for 0.3.0 by @Xuanwo in #573
• feat(timestamp_ns): Implement timestamps with nanosecond precision by @Sl1mb0 in #542
• fix: correct partition-id to field-id in UnboundPartitionField by @FANNG1 in #576
• fix: Update sqlx from 0.8.0 to 0.8.1 by @FANNG1 in #584
• chore(deps): Update typed-builder requirement from 0.19 to 0.20 by @dependabot in #582
• Expose Transforms to Python Binding by @sungwy in #556
• chore(deps): Bump crate-ci/typos from 1.23.6 to 1.24.1 by @dependabot in #583
• Table Scan: Add Row Group Skipping by @sdd in #558
• chore: bump crate-ci/typos to 1.24.3 by @sdlarsen in #598
• feat: SQL Catalog - namespaces by @callum-ryan in #534
• feat: Add more fields in FileScanTask by @Xuanwo in #609
• chore(deps): Bump crate-ci/typos from 1.24.3 to 1.24.5 by @dependabot in #616
• fix: Less Panics for Snapshot timestamps by @c-thiel in #614
• feat: partition compatibility by @c-thiel in #612
• feat: SortOrder methods should take schema ref if possible by @c-thiel in #613
• feat: add client.region by @jdockerty in #623
• fix: Correctly calculate highest_field_id in schema by @liurenjie1024 in #590
• Feat: Normalize TableMetadata by @c-thiel in #611
• refactor(python): Expose transform as a submodule for pyiceberg_core by @Xuanwo in #628
• feat: support projection pushdown for datafusion iceberg by @FANNG1 in #594
• chore: Bump opendal to 0.50 by @Xuanwo in #634
• feat: add Sync to TransformFunction by @xxchan in #638
• feat: expose arrow type <-> iceberg type by @xxchan in #637
• doc: improve FileIO doc by @xxchan in #642
• chore(deps): Bump crate-ci/typos from 1.24.5 to 1.24.6 by @dependabot in #640
• Migrate to arrow-* v53 by @sdd in #626
• feat: expose remove_all in FileIO by @xxchan in #643
• feat (datafusion integration): convert datafusion expr filters to Iceberg Predicate by @a-agmon in #588
• feat: Add NamespaceIdent.parent() by @c-thiel in #641
• Table Scan: Add Row Selection Filtering by @sdd in #565
• fix: compile error due to merge stale PR by @xxchan in #646
• scan: change ErrorKind when table dont have spanshots by @mattheusv in #608
• fix: avoid to create operator of memory storage every time by @ZENOTME in #635
• feat (datafusion): making IcebergTableProvider public to be used without a catalog by @a-agmon in #650
• test (datafusion): add test for table provider creation by @a-agmon in #651
• fix: page index evaluator min/max args inverted by @sdd in #648
• chore: fix typo in FileIO Schemes by @wcy-fdu in #653
• fix: TableUpdate Snapshot deserialization for v1 by @c-thiel in #656
• feat: Reassign field ids for schema by @c-thiel in #615
• feat: add gcp oauth support by @twuebi in #654
• fix(arrow): Use new ParquetMetaDataReader instead by @Xuanwo in #661
• chore(deps): bump typos crate to 1.25.0 by @matthewwillian in #662
• RecordBatchTransformer: Handle schema migration and column re-ordering in table scans by @sdd in #602
• docs: installation of the new iceberg_catalog_rest added to the docs by @nishant-sachdeva in #355
• feat(datafusion): Support pushdown more datafusion exprs to Iceberg by @FANNG1 in #649
• feat: Derive PartialEq for FileScanTask by @Xuanwo in #660
• feat: SQL Catalog - Tables by @callum-ryan in #610
• ci: Allow install a non-debian-packaged Python package by @Xuanwo in #666
• docs: README uses iceberg-rust instead of we by @caicancai in #667
• chore(deps): Bump crate-ci/typos from 1.25.0 to 1.26.0 by @dependabot in #668
• feat: Add equality delete writer by @Dysprosium0626 in #372
• Revert "feat: Add equality delete writer (#372)" by @Xuanwo in #672
• ci: Fix CI for bindings python by @Xuanwo in #678
• fix: OpenDAL is_exist => exists by @sdd in #680
• feat: Expose ManifestEntry status by @zheilbron in #681
• feat: allow empty projection in table scan by @sundy-li in #677
• chore(deps): Bump crate-ci/typos from 1.26.0 to 1.26.8 by @dependabot in #683
• fix: bump parquet minor version by @xxchan in #684
• fix(type): fix type promote to ignore field name. by @chenzl25 in #685
• feat: implement IcebergTableProviderFactory for datafusion by @yukkit in #600
• feat: Safer PartitionSpec & SchemalessPartitionSpec by @c-thiel in #645
• chore(deps): Bump crate-ci/typos from 1.26.8 to 1.27.0 by @dependabot in #687
• feat: TableMetadata accessors for current ids of Schema, Snapshot and SortOrder by @c-thiel in #688
• chore: upgrade to DataFusion 43 by @gruuya in #691
• chore(deps): Bump crate-ci/typos from 1.27.0 to 1.27.3 by @dependabot in #693
• feat: Expose length of Iterators by @c-thiel in #692
• feat: Implement TableRequirement checks by @c-thiel in #689
• feat: Add ViewUpdate to catalog by @c-thiel in #690
• chore: update .asf.yaml by @c-thiel in #701
• datafusion: Create table provider for a snapshot. by @ryzhyk in #707
• Add Python Release Action to publish pyiceberg_core dist to Pypi by @sungwy in #705
• chore: Mark last-field-id as deprecated by @Fokko in #715
• TableMetadataBuilder by @c-thiel in #587
• Add fallback attribute to all strip_options. by @ryzhyk in #708
• fix: Remove check of last_column_id by @liurenjie1024 in #717
• Fix error running data fusion queries - Physical input schema should be the same as the one converted from logical input schema by @FANNG1 in #664
• chore: Typo in test :) by @Fokko in #727
• Derive Clone for IcebergTableProvider by @SergeiPatiakin in #722
• fix: expand arrow to iceberg schema to handle nanosecond timestamp by @jdockerty in #710
• feat: Add equality delete writer by @ZENOTME in #703
• chore: Bump upload-artifact@v3 to v4 by @sungwy in #725
• feat: support append data file and add e2e test by @ZENOTME in #349
• chore(deps): Bump actions/setup-python from 4 to 5 by @dependabot in #746
• chore: Align argument name with doc comment by @SergeiPatiakin in #750
• fix: equality delete writer field id project by @ZENOTME in #751
• feat: expose opendal S3 options for anonymous access by @gruuya in #757
• fix: current-snapshot-id serialized to -1 in TableMetadata.json by @c-thiel in #755
• feat(puffin): Add Puffin crate and CompressionCodec by @fqaiser94 in #745
• Build: Delete branch automatically on PR merge by @manuzhang in #764
• chore(deps): bump crate-ci/typos from 1.27.3 to 1.28.1 by @jonathanc-n in #769
• chore(deps): Bump crate-ci/typos from 1.27.3 to 1.28.2 by @dependabot in #767
• Add Spark for integration tests by @Fokko in #766
• refine: refine writer interface by @ZENOTME in #741
• Clean up docker Docker by @Fokko in #770
• name mapping serde by @barronw in #740
• docker: The archive seems unstable by @Fokko in #773
• doc: add RisingWave to users by @xxchan in #775
• feat: Expose disable_config_load opendal S3 option by @gruuya in #782
• Suport conversion of Arrow Int8 and Int16 to Iceberg Int by @gruuya in #787
• infra: Dismiss stale reviews by @liurenjie1024 in #779
• fix: return type for year and month transform should be int by @xxchan in #776
• feat: Allow for schema evolution by @Fokko in #786
• Retry object store reads on temporary errors. by @ryzhyk in #788
• refactor(puffin): Move puffin crate contents inside iceberg crate by @fqaiser94 in #789
• feat: Implement Decimal from/to bytes represents by @Xuanwo in #665
• feat: eagerly project the arrow schema to scope out non-selected fields by @gruuya in #785
• fix: wrong compute of partitions in manifest by @ZENOTME in #794
• fix: set key_metadata to Null by default by @feniljain in #800
• test: append partition data file by @feniljain in #742
• chore: Add more debug message inside error by @Xuanwo in #793
• fix: Error source from cache has been shadowed by @Xuanwo in #792
• fix(catalog/rest): Ensure token been reused correctly by @Xuanwo in #801
• feat!: Remove BoundPartitionSpec by @c-thiel in #771
• chore(deps): Bump crate-ci/typos from 1.28.2 to 1.28.3 by @dependabot in #805
• feat: add DataFileWriter tests for schema and partition by @jonathanc-n in #768
• fix: day transform compute by @ZENOTME in #796
• feat: TableMetadata Statistic Files by @c-thiel in #799
• Bump pyiceberg_core to 0.4.0 by @sungwy in #808
• chore(docs): Update Readme - Lakekeeper repository moved by @c-thiel in #810
• Prep 0.4.0 release by @sungwy in #809
• feat: Add RemovePartitionSpecs table update by @c-thiel in #804
• feat: Store file io props to allow re-build it by @Xuanwo in #802
• chore: Generate Changelog Dependencies for 0.4.0 release by @sungwy in #812
• chore: chmod +x on verify.py script by @sungwy in #817

New Contributors

• @mattheusv made their first contribution in #553
• @FANNG1 made their first contribution in #557
• @ChinoUkaegbu made their first contribution in #561
• @Sl1mb0 made their first contribution in #542
• @sdlarsen made their first contribution in #598
• @wcy-fdu made their first contribution in #653
• @matthewwillian made their first contribution in #662
• @nishant-sachdeva made their first contribution in #355
• @caicancai made their first contribution in #667
• @zheilbron made their first contribution in #681
• @sundy-li made their first contribution in #677
• @chenzl25 made their first contribution in #685
• @yukkit made their first contribution in #600
• @ryzhyk made their first contribution in #707
• @SergeiPatiakin made their first contribution in #722
• @jonathanc-n made their first contribution in #769
• @feniljain made their first contribution in #800

Full Changelog: v0.3.0...v0.4.0