Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logical type alias and cleanup #44

Merged
merged 2 commits into from
Nov 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 38 additions & 78 deletions api/pkgs/@duckdb/node-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ This is a high-level API meant for applications. It depends on low-level binding
Some features are not yet complete:
- Friendlier APIs for convering results to common JS data structures.
- Friendlier APIs for converting values of specialized and complex DuckDB types to common JS types.
- Automatic memory management (i.e. avoiding the need to call `dispose` manually in most cases).
- Appending and binding advanced data types. (Additional DuckDB C API support needed.)
- Writing to data chunk vectors. (Directly writing to binary buffers is challenging to support using the Node Addon API.)
- User-defined types & functions. (Support for this was added to the DuckDB C API in v1.1.0.)
Expand Down Expand Up @@ -70,33 +69,18 @@ Set configuration options:
const instance = await DuckDBInstance.create('my_duckdb.db', { threads: '4' });
```

Dispose:
```ts
await instance.dispose();
```

### Connect

```ts
const connection = await instance.connect();
```

Dispose:
```ts
await connection.dispose();
```

### Run SQL

```ts
const result = await connection.run('from test_all_types()');
```

Dispose:
```ts
result.dispose();
```

### Parameterize SQL

```ts
Expand All @@ -106,12 +90,6 @@ prepared.bindInteger(2, 42);
const result = await prepared.run();
```

Dispose:
```ts
result.dispose();
prepared.dispose();
```

### Inspect Result

Get column names and types:
Expand Down Expand Up @@ -155,11 +133,6 @@ for (let columnIndex = 0; columnIndex < columnCount; columnIndex++) {
}
```

Dispose data chunk:
```ts
chunk.dispose();
```

### Inspect Data Types

```ts
Expand Down Expand Up @@ -197,6 +170,11 @@ if (columnType.typeId === DuckDBTypeId.UNION) {
const unionMemberTags = columnType.memberTags;
const unionMemberTypes = columnType.memberTypes;
}

// For the JSON type (https://duckdb.org/docs/data/json/json_type)
if (columnType.alias === 'JSON') {
const json = JSON.parse(columnValue);
}
```

Every type implements toString, matching DuckDB's type-to-string conversion.
Expand Down Expand Up @@ -313,25 +291,25 @@ if (columnType.typeId === DuckDBTypeId.UUID) {
### Append To Table

```ts
const createTableResult = await connection.run(`create or replace table target_table(i integer, v varchar)`);
createTableResult.dispose();
await connection.run(`create or replace table target_table(i integer, v varchar)`);

const appender = await connection.createAppender('main', 'target_table');
try {
appender.appendInteger(42);
appender.appendVarchar('duck');
appender.endRow();
appender.appendInteger(123);
appender.appendVarchar('mallard');
appender.endRow();
appender.flush();
appender.appendInteger(17);
appender.appendVarchar('goose');
appender.endRow();
appender.close(); // also flushes
} finally {
appender.dispose();
}

appender.appendInteger(42);
appender.appendVarchar('duck');
appender.endRow();

appender.appendInteger(123);
appender.appendVarchar('mallard');
appender.endRow();

appender.flush();

appender.appendInteger(17);
appender.appendVarchar('goose');
appender.endRow();

appender.close(); // also flushes
```

### Extract Statements
Expand All @@ -343,24 +321,15 @@ const extractedStatements = await connection.extractStatements(`
drop table numbers;
`);
const parameterValues = [10, 7];
try {
const statementCount = extractedStatements.count;
for (let statementIndex = 0; statementIndex < statementCount; statementIndex++) {
const prepared = await extractedStatements.prepare(statementIndex);
try {
let parameterCount = prepared.parameterCount;
for (let parameterIndex = 1; parameterIndex <= parameterCount; parameterIndex++) {
prepared.bindInteger(parameterIndex, parameterValues.shift());
}
const result = await prepared.run();
// ...
result.dispose();
} finally {
prepared.dispose();
}
const statementCount = extractedStatements.count;
for (let statementIndex = 0; statementIndex < statementCount; statementIndex++) {
const prepared = await extractedStatements.prepare(statementIndex);
let parameterCount = prepared.parameterCount;
for (let parameterIndex = 1; parameterIndex <= parameterCount; parameterIndex++) {
prepared.bindInteger(parameterIndex, parameterValues.shift());
}
} finally {
extractedStatements.dispose();
const result = await prepared.run();
// ...
}
```

Expand All @@ -376,21 +345,12 @@ async function sleep(ms) {
}

const prepared = await connection.prepare('from range(10_000_000)');
try {
const pending = prepared.start();
try {
while (pending.runTask() !== DuckDBPendingResultState.RESULT_READY) {
console.log('not ready');
await sleep(1);
}
console.log('ready');
const result = await pending.getResult();
// ...
result.dispose();
} finally {
pending.dispose();
}
} finally {
prepared.dispose();
}
const pending = prepared.start();
while (pending.runTask() !== DuckDBPendingResultState.RESULT_READY) {
console.log('not ready');
await sleep(1);
}
console.log('ready');
const result = await pending.getResult();
// ...
```
4 changes: 3 additions & 1 deletion api/src/DuckDBAppender.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@ export class DuckDBAppender {
return duckdb.appender_column_count(this.appender);
}
public columnType(columnIndex: number): DuckDBType {
return DuckDBLogicalType.consumeAsType(duckdb.appender_column_type(this.appender, columnIndex));
return DuckDBLogicalType.create(
duckdb.appender_column_type(this.appender, columnIndex)
).asType();
}
public endRow() {
duckdb.appender_end_row(this.appender);
Expand Down
82 changes: 39 additions & 43 deletions api/src/DuckDBLogicalType.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,6 @@ export class DuckDBLogicalType {
protected constructor(logical_type: duckdb.LogicalType) {
this.logical_type = logical_type;
}
static consumeAsType(logical_type: duckdb.LogicalType): DuckDBType {
const logicalType = DuckDBLogicalType.create(logical_type);
const type = logicalType.asType();
return type;
}
static create(logical_type: duckdb.LogicalType): DuckDBLogicalType {
switch (duckdb.get_type_id(logical_type)) {
case duckdb.Type.DECIMAL:
Expand Down Expand Up @@ -147,60 +142,61 @@ export class DuckDBLogicalType {
public get typeId(): DuckDBTypeId {
return duckdb.get_type_id(this.logical_type) as number as DuckDBTypeId;
}
public get alias(): string | null {
return duckdb.logical_type_get_alias(this.logical_type);
public get alias(): string | undefined {
return duckdb.logical_type_get_alias(this.logical_type) || undefined;
}
public set alias(newAlias: string) {
duckdb.logical_type_set_alias(this.logical_type, newAlias);
}
public asType(): DuckDBType {
const alias = this.alias;
switch (this.typeId) {
case DuckDBTypeId.BOOLEAN:
return DuckDBBooleanType.instance;
return DuckDBBooleanType.create(alias);
case DuckDBTypeId.TINYINT:
return DuckDBTinyIntType.instance;
return DuckDBTinyIntType.create(alias);
case DuckDBTypeId.SMALLINT:
return DuckDBSmallIntType.instance;
return DuckDBSmallIntType.create(alias);
case DuckDBTypeId.INTEGER:
return DuckDBIntegerType.instance;
return DuckDBIntegerType.create(alias);
case DuckDBTypeId.BIGINT:
return DuckDBBigIntType.instance;
return DuckDBBigIntType.create(alias);
case DuckDBTypeId.UTINYINT:
return DuckDBUTinyIntType.instance;
return DuckDBUTinyIntType.create(alias);
case DuckDBTypeId.USMALLINT:
return DuckDBUSmallIntType.instance;
return DuckDBUSmallIntType.create(alias);
case DuckDBTypeId.UINTEGER:
return DuckDBUIntegerType.instance;
return DuckDBUIntegerType.create(alias);
case DuckDBTypeId.UBIGINT:
return DuckDBUBigIntType.instance;
return DuckDBUBigIntType.create(alias);
case DuckDBTypeId.FLOAT:
return DuckDBFloatType.instance;
return DuckDBFloatType.create(alias);
case DuckDBTypeId.DOUBLE:
return DuckDBDoubleType.instance;
return DuckDBDoubleType.create(alias);
case DuckDBTypeId.TIMESTAMP:
return DuckDBTimestampType.instance;
return DuckDBTimestampType.create(alias);
case DuckDBTypeId.DATE:
return DuckDBDateType.instance;
return DuckDBDateType.create(alias);
case DuckDBTypeId.TIME:
return DuckDBTimeType.instance;
return DuckDBTimeType.create(alias);
case DuckDBTypeId.INTERVAL:
return DuckDBIntervalType.instance;
return DuckDBIntervalType.create(alias);
case DuckDBTypeId.HUGEINT:
return DuckDBHugeIntType.instance;
return DuckDBHugeIntType.create(alias);
case DuckDBTypeId.UHUGEINT:
return DuckDBUHugeIntType.instance;
return DuckDBUHugeIntType.create(alias);
case DuckDBTypeId.VARCHAR:
return DuckDBVarCharType.instance;
return DuckDBVarCharType.create(alias);
case DuckDBTypeId.BLOB:
return DuckDBBlobType.instance;
return DuckDBBlobType.create(alias);
case DuckDBTypeId.DECIMAL:
throw new Error('Expected override');
case DuckDBTypeId.TIMESTAMP_S:
return DuckDBTimestampSecondsType.instance;
return DuckDBTimestampSecondsType.create(alias);
case DuckDBTypeId.TIMESTAMP_MS:
return DuckDBTimestampMillisecondsType.instance;
return DuckDBTimestampMillisecondsType.create(alias);
case DuckDBTypeId.TIMESTAMP_NS:
return DuckDBTimestampNanosecondsType.instance;
return DuckDBTimestampNanosecondsType.create(alias);
case DuckDBTypeId.ENUM:
throw new Error('Expected override');
case DuckDBTypeId.LIST:
Expand All @@ -212,21 +208,21 @@ export class DuckDBLogicalType {
case DuckDBTypeId.ARRAY:
throw new Error('Expected override');
case DuckDBTypeId.UUID:
return DuckDBUUIDType.instance;
return DuckDBUUIDType.create(alias);
case DuckDBTypeId.UNION:
throw new Error('Expected override');
case DuckDBTypeId.BIT:
return DuckDBBitType.instance;
return DuckDBBitType.create(alias);
case DuckDBTypeId.TIME_TZ:
return DuckDBTimeTZType.instance;
return DuckDBTimeTZType.create(alias);
case DuckDBTypeId.TIMESTAMP_TZ:
return DuckDBTimestampTZType.instance;
return DuckDBTimestampTZType.create(alias);
case DuckDBTypeId.ANY:
return DuckDBAnyType.instance;
return DuckDBAnyType.create(alias);
case DuckDBTypeId.VARINT:
return DuckDBVarIntType.instance;
return DuckDBVarIntType.create(alias);
case DuckDBTypeId.SQLNULL:
return DuckDBSQLNullType.instance;
return DuckDBSQLNullType.create(alias);
default:
throw new Error(`Unexpected type id: ${this.typeId}`);
}
Expand All @@ -246,7 +242,7 @@ export class DuckDBDecimalLogicalType extends DuckDBLogicalType {
) as number as DuckDBTypeId;
}
public override asType(): DuckDBDecimalType {
return new DuckDBDecimalType(this.width, this.scale);
return new DuckDBDecimalType(this.width, this.scale, this.alias);
}
}

Expand All @@ -271,7 +267,7 @@ export class DuckDBEnumLogicalType extends DuckDBLogicalType {
) as number as DuckDBTypeId;
}
public override asType(): DuckDBEnumType {
return new DuckDBEnumType(this.values(), this.internalTypeId);
return new DuckDBEnumType(this.values(), this.internalTypeId, this.alias);
}
}

Expand All @@ -282,7 +278,7 @@ export class DuckDBListLogicalType extends DuckDBLogicalType {
);
}
public override asType(): DuckDBListType {
return new DuckDBListType(this.valueType.asType());
return new DuckDBListType(this.valueType.asType(), this.alias);
}
}

Expand Down Expand Up @@ -326,7 +322,7 @@ export class DuckDBStructLogicalType extends DuckDBLogicalType {
return valueTypes;
}
public override asType(): DuckDBStructType {
return new DuckDBStructType(this.entryNames(), this.entryTypes());
return new DuckDBStructType(this.entryNames(), this.entryTypes(), this.alias);
}
}

Expand All @@ -342,7 +338,7 @@ export class DuckDBMapLogicalType extends DuckDBLogicalType {
);
}
public override asType(): DuckDBMapType {
return new DuckDBMapType(this.keyType.asType(), this.valueType.asType());
return new DuckDBMapType(this.keyType.asType(), this.valueType.asType(), this.alias);
}
}

Expand All @@ -356,7 +352,7 @@ export class DuckDBArrayLogicalType extends DuckDBLogicalType {
return duckdb.array_type_array_size(this.logical_type);
}
public override asType(): DuckDBArrayType {
return new DuckDBArrayType(this.valueType.asType(), this.length);
return new DuckDBArrayType(this.valueType.asType(), this.length, this.alias);
}
}

Expand Down Expand Up @@ -400,6 +396,6 @@ export class DuckDBUnionLogicalType extends DuckDBLogicalType {
return valueTypes;
}
public override asType(): DuckDBUnionType {
return new DuckDBUnionType(this.memberTags(), this.memberTypes());
return new DuckDBUnionType(this.memberTags(), this.memberTypes(), this.alias);
}
}
4 changes: 2 additions & 2 deletions api/src/DuckDBResult.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ export class DuckDBResult {
);
}
public columnType(columnIndex: number): DuckDBType {
return DuckDBLogicalType.consumeAsType(
return DuckDBLogicalType.create(
duckdb.column_logical_type(this.result, columnIndex)
);
).asType();
}
public get rowsChanged(): number {
return duckdb.rows_changed(this.result);
Expand Down
Loading