Skip to content

Commit

Permalink
Update xlsx table (#316)
Browse files Browse the repository at this point in the history
This PR is a fix for issue #259

List of updates/fixes:
* module xlsx renamed to excel.
* Allow reading not only xlsx format but also xls, ods, xlsb
* Allow Excel DateTime format and transform it to arrow
Timestamp(Seconds, None)
* Allow using NULLs in any data types and use null value instead of
string "null"
* Fix issue with incorrect data type inference when multiple data types
are detected.
* Add possibility to specify data schema in config.
* Add new options: -
rows_range_start
 - rows_range_end
 - columns_range_start
 - columns_range_end
 - schema_inference_lines
* Make sheet_name optional and if it is not specified than use first
sheet by default

* Bump calamine crate to version 0.23.1 and add feature "dates"
(supporting for DateTime column format)

Documentation updates: roapi/docs#20
  • Loading branch information
Maks-D authored Feb 1, 2024
1 parent 9377b7c commit 81a2520
Show file tree
Hide file tree
Showing 8 changed files with 812 additions and 296 deletions.
20 changes: 11 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@ Data layer:
- [x] JSON
- [x] NDJSON
- [x] parquet
- [x] xls, xlsx, xlsm, ods: https://github.com/tafia/calamine
- [x] xls, xlsx, xlsb, ods: https://github.com/tafia/calamine
- [x] [DeltaLake](https://delta.io/)
Misc:
Expand Down
2 changes: 1 addition & 1 deletion columnq/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ reqwest = { version = "0.11", default-features = false, features = [
"blocking",
"json",
] }
calamine = "0.19.1"
calamine = {version = "0.23.1", features = ["dates"]}

tokio = { version = "1", features = ["rt-multi-thread"] }
futures = "0.3"
Expand Down
4 changes: 2 additions & 2 deletions columnq/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ pub enum ColumnQError {
#[error("Error loading Delta table: {0}")]
LoadDelta(String),

#[error("Error loading Xlsx table: {0}")]
LoadXlsx(String),
#[error("Error loading Excel table: {0}")]
LoadExcel(String),

#[error("Error loading data from HTTP store: {0}")]
HttpStore(String),
Expand Down
Loading

0 comments on commit 81a2520

Please sign in to comment.