Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Events Dates as dateranges generated colum #366

Open
RudiThoeni opened this issue Jul 14, 2023 · 9 comments
Open

Events Dates as dateranges generated colum #366

RudiThoeni opened this issue Jul 14, 2023 · 9 comments
Assignees
Labels
feature:postgres E.g. postgres functions

Comments

@RudiThoeni
Copy link
Member

RudiThoeni commented Jul 14, 2023

Hi

an Event is currently defined with a Startdate and an Enddate. The actual datefilter simply returns the Event by checking if the passed daterange is IN the Event start begin daterange....

This behaviour is not always correct because there exists Events that occurs at example on 14.07 / 21.07 / 28.07
So the begindate is set to 14.07 and enddate 28.07. Now if the query from 16.07 to 20.07 with the actual datamodel the event is returned because this daterange is in the defined

This Eventdates are stored in the EventDate Property
example
"EventDate": [ { "To": "2023-04-21T00:00:00", "End": "16:14:00", "From": "2023-04-21T00:00:00", "Begin": "15:00:00", ... }, { "To": "2023-04-25T00:00:00", "End": "16:14:00", "From": "2023-04-25T00:00:00", "Begin": "15:00:00", ... } ],

another possibility is to have

"EventDate": [ { "To": "2023-04-25T00:00:00", "End": "16:14:00", "From": "2023-04-21T00:00:00", "Begin": "15:00:00", ... }, { "To": "2023-04-30T00:00:00", "End": "16:14:00", "From": "2023-04-27T00:00:00", "Begin": "15:00:00", ... } ],

The idea is to add the defined Dateranges in the EventDateField as a tsrange[] and querying this fields

@RudiThoeni RudiThoeni self-assigned this Aug 8, 2023
@toburger toburger added the feature:postgres E.g. postgres functions label Sep 6, 2023
@toburger
Copy link
Collaborator

toburger commented Sep 7, 2023

I think this should be solved with a tsmultirange instead of a tsrange[]: https://www.cybertec-postgresql.com/en/multiranges-in-postgresql-14/

@RudiThoeni
Copy link
Member Author

sounds great.... what would be nice if we create a tsmultirange generated column out of the EventDates json and use this for querying

We have to consider we have this types of EventDates

05.01.22 - 06.01.22 --> Event occurs every Day, EventDates has [05.01.22 (from/to) , 06.01.22 (from/to)]
2)
13.07.22 - 13.08.22 --> Event Occurs only on certain days, EventDates has [13.07.22 (from/to), 27.07.22 (from/to), 03.08.22 (from/to)]
3)
10.08.22 - 30.08.22 --> Event Occurs only on certain date Intervals, EventDates has [10.08.22 from - 12.08.22 to, 22.08.22 from - 30.08.22 to]

@toburger
Copy link
Collaborator

toburger commented Sep 7, 2023

The following PG function returns the DateRange as a tsrange[]:

CREATE OR REPLACE FUNCTION public.json_2_tsrange_array(jsonarray jsonb)
 RETURNS tsrange[]
 LANGUAGE plpgsql
 IMMUTABLE STRICT
AS $function$ begin if jsonarray <> 'null' then return (
  select 
    array(
      select 
        tsrange(
          (event ->> 'From'):: timestamp, 
          (event ->> 'To'):: timestamp + interval '1 day'
        ) 
      from 
        jsonb_array_elements(jsonarray) as event 
      where 
        (event ->> 'From'):: timestamp <= (event ->> 'To'):: timestamp + interval '1 day'
    )
);
else return null;
end if;
end;
$function$

Usage:

select json_2_tsrange_array(data->'EventDate') from events;

I have to add one day to the 'To' date to have the correct date span.
I have to check if the 'From' is smaller than the 'To' date. Probably some events are not providing correct dates where 'From' is bigger than 'To'.
The 'From' and 'To' part of the function is hardcoded but could be parameterized for greater flexibility.

@toburger
Copy link
Collaborator

toburger commented Sep 7, 2023

Variant that includes the time information:

CREATE OR REPLACE FUNCTION public.json_2_tsrange_array(jsonarray jsonb)
 RETURNS tsrange[]
 LANGUAGE plpgsql
 IMMUTABLE STRICT
AS $function$ begin if jsonarray <> 'null' then return (
  select 
    array(
      select 
        tsrange(
          ( (event ->> 'From')::timestamp + (event ->> 'Begin')::time), 
          ( (event ->> 'To')::timestamp + (event ->> 'End')::time)
        ) 
      from 
        jsonb_array_elements(jsonarray) as event 
      where 
        (event ->> 'From')::timestamp + (event ->> 'Begin')::time < (event ->> 'To')::timestamp + (event ->> 'End')::time
    )
);
else return null;
end if;
end;
$function$

@toburger
Copy link
Collaborator

toburger commented Sep 8, 2023

It seems that the correct solution for the date range queries ist to use tsmultirange as it allows to query the "overlap" with the && operator. The tsrange[] query with the && operator only returns a small subset of the events.

In order to convert the tsrange[] to a tsmultirange I've written the following PGSQL function:

CREATE OR REPLACE FUNCTION convert_tsrange_array_to_tsmultirange(tsrange_array tsrange[])
RETURNS tsmultirange
LANGUAGE plpgsql
 IMMUTABLE STRICT
AS $$
DECLARE
    result tsmultirange := tsmultirange();
    tsr tsrange;
BEGIN
	IF tsrange_array IS NOT NULL THEN
    -- Durchlaufen des tsrange-Arrays
    FOREACH tsr IN ARRAY tsrange_array
    LOOP
        -- Hinzufügen des tsrange-Elements zum tsmultirange
        result := result + tsmultirange(tsrange(tsr));
    END LOOP;
	END IF;

    RETURN result;
END;
$$;

To speed up the query the following generated column can be defined:

ALTER TABLE events ADD IF NOT EXISTS gen_eventdates tsmultirange GENERATED ALWAYS AS (convert_tsrange_array_to_tsmultirange(json_2_tsrange_array(data#>'{EventDate}'))) stored;

Then the following query can be written to query for a date range that overlaps with all the date ranges defined in the event document:

select id from events where gen_eventdates && tsrange('[2023-09-07 10:00,2023-09-07 12:30]');

Which returns at the time of writing 498 events.

@toburger
Copy link
Collaborator

toburger commented Sep 8, 2023

Be aware that the gen_eventdates column is solely useful to query for the date ranges. Do not use the gen_eventdates column to visualize the different date ranges of an event because a tsmultirange column merges overlapping date ranges.

@toburger
Copy link
Collaborator

toburger commented Sep 8, 2023

Be aware that the gen_eventdates column is solely useful to query for the date ranges. Do not use the gen_eventdates column to visualize the different date ranges of an event because a tsmultirange column merges overlapping date ranges.

I can be wrong on that as I can't find any documentation for tsmultirange.

@toburger
Copy link
Collaborator

toburger commented Sep 8, 2023

Be aware that the gen_eventdates column is solely useful to query for the date ranges. Do not use the gen_eventdates column to visualize the different date ranges of an event because a tsmultirange column merges overlapping date ranges.

I can be wrong on that as I can't find any documentation for tsmultirange.

I've testet it with the following SQL and it seems like the date ranges get merged:

select tsmultirange(tsrange('[2023-09-07 10:00,2023-09-07 12:30]'), tsrange('[2023-09-07 12:00,2023-09-07 14:30]'));

Result

{["2023-09-07 10:00:00","2023-09-07 14:30:00"]}

And with the +:

select tsmultirange(tsrange('[2023-09-07 10:00,2023-09-07 12:30]')) + tsmultirange(tsrange('[2023-09-07 12:00,2023-09-07 14:30]'));

Result

{["2023-09-07 10:00:00","2023-09-07 14:30:00"]}

So in fact the date ranges get merged!

So my previous statement holds:

Be aware that the gen_eventdates column is solely useful to query for the date ranges. Do not use the gen_eventdates column to visualize the different date ranges of an event because a tsmultirange column merges overlapping date ranges.

@RudiThoeni
Copy link
Member Author

it looks very good, i will make further tests and deploy on test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature:postgres E.g. postgres functions
Projects
None yet
Development

No branches or pull requests

2 participants