Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(c/driver/postgresql): Interval support #908

Merged
merged 11 commits into from
Jul 28, 2023

Conversation

WillAyd
Copy link
Contributor

@WillAyd WillAyd commented Jul 16, 2023

alternate to #907 still needs some work to get the test templates working

c/driver/postgresql/postgres_copy_reader.h Outdated Show resolved Hide resolved
Comment on lines 236 to 237
NANOARROW_RETURN_NOT_OK(ArrowBufferAppend(data_, &days, sizeof(int32_t)));
NANOARROW_RETURN_NOT_OK(ArrowBufferAppend(data_, &months, sizeof(int32_t)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the order of these switched?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. Not sure the definitive place to look in arrow but modeled this after https://github.com/apache/arrow/blob/fb7fb0db60aac7e48f6434b48aa23ada5c4885a2/cpp/src/arrow/scalar.cc#L84

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(The prose docs were never updated to account for this, because I suppose we don't document the types, only the 'layouts'...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool thanks for the reference. Is what I linked a bug in arrow?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, it's merely misleading.

Unwrapping a bunch of layers of type references: https://github.com/apache/arrow/blob/fb7fb0db60aac7e48f6434b48aa23ada5c4885a2/cpp/src/arrow/type.h#L1529-L1536

The code defines it as months-days-nanos, just hashes them in a different order for some reason (alphabetically?)

@WillAyd WillAyd marked this pull request as ready for review July 28, 2023 13:51
@@ -426,6 +430,23 @@ struct BindStream {
std::memcpy(param_values[col], &value, sizeof(int64_t));
break;
}
case ArrowType::NANOARROW_TYPE_INTERVAL_MONTH_DAY_NANO: {
const auto buf =
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that is missing from https://github.com/apache/arrow-nanoarrow/pull/258/files is a ArrowArrayViewGetIntervalUnsafe function, which would help both here and in the test

Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

const int64_t time = ReadUnsafe<int64_t>(data) * 1000;
const int64_t time_usec = ReadUnsafe<int64_t>(data);

if ((time_usec > INT64_MAX / 1000) | (time_usec < INT64_MIN / 1000)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ((time_usec > INT64_MAX / 1000) | (time_usec < INT64_MIN / 1000)) {
if ((time_usec > INT64_MAX / 1000) || (time_usec < INT64_MIN / 1000)) {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, Arrow vendors https://github.com/nemequ/portable-snippets/tree/master/safe-math for overflow-safe helpers (we can do that as a followup though)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is your idea to also vendor that in this repo? Or something we should do in nanoarrow?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, in this repo.

@WillAyd
Copy link
Contributor Author

WillAyd commented Jul 28, 2023

Don't think the R failures are related?

@lidavidm lidavidm merged commit 0a5afa3 into apache:main Jul 28, 2023
56 of 65 checks passed
@lidavidm
Copy link
Member

Yeah, we can ignore that for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants