Create DuckDB internal schema

[akvlad]

What

Every quackpipe instance should have an internal tables describing the created "merged parquet" tables.

The tables should contain

• table name
• table base folder path on HD
• table fields and types
• table order by expression

How

Add the create table logic on quackpipe startup:

CREATE TABLE tables (
  name VARCHAR PRIMARY KEY, 
  path VARCHAR,
  field_names VARCHAR[],
  field_types VARCHAR[],
  order_by VARCHAR[],
  engine VARCHAR[],
  timestamp_field: VARCHAR[],
  timestamp_precision: VARCHAR[],
  partition_by: VARCHAR[],
);

The table should be created on startup if it does not exist

The table should persist between restarts

Test the logic:

• start a quackpipe instance. Table should be created
• insert some data into the table
• restart the instance. The table should contain the inserted data

[github-actions]

Thanks for opening an Issue! Please star this repository to motivate developers! ⭐