From ee390abe8a4b17357300df1d8960078923f9972e Mon Sep 17 00:00:00 2001 From: Sam Ansmink Date: Wed, 11 Oct 2023 14:36:06 +0200 Subject: [PATCH 1/3] small cleanups and changes to readme --- .../workflows/MainDistributionPipeline.yml | 2 +- docs/README.md | 42 ++++++++----------- 2 files changed, 18 insertions(+), 26 deletions(-) diff --git a/.github/workflows/MainDistributionPipeline.yml b/.github/workflows/MainDistributionPipeline.yml index 002e114..3d91ab3 100644 --- a/.github/workflows/MainDistributionPipeline.yml +++ b/.github/workflows/MainDistributionPipeline.yml @@ -12,7 +12,7 @@ concurrency: cancel-in-progress: true jobs: - duckdb-stable: + duckdb-stable-build: name: Build extension binaries uses: duckdb/duckdb/.github/workflows/_extension_distribution.yml@v0.9.1 with: diff --git a/docs/README.md b/docs/README.md index d45c235..1b092e4 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,28 +1,22 @@ -# WIP Disclaimer -This template is currently a work-in-progress. Feel free to play around with it and give us feedback. Note also that this template depends on a development version of DuckDB. - -Get in contact with fellow extension developers on https://discord.duckdb.org and follow https://duckdb.org/news for more information on official launch. - # DuckDB Extension Template -The main goal of this template is to allow users to easily develop, test and distribute their own DuckDB extension. +This repository contains a template for creating a DuckDB extension. The main goal of this template is to allow users to easily develop, test and distribute their own DuckDB extension. The main branch of the template is always based on the latest stable DuckDB allowing you to try out your extension right away. ## Getting started First step to getting started is to create your own repo from this template by clicking `Use this template`. Then clone your new repository using ```sh git clone --recurse-submodules https://github.com//.git ``` -Note that `--recurse-submodules` will ensure the correct version of duckdb is pulled allowing you to get started right away. +Note that `--recurse-submodules` will ensure DuckDB is pulled which is required to build the extension. ## Building -### Dependencies -DuckDB extensions use VCPKG for dependency management. To demonstrate that, the example extension in the template links against -OpenSSL. Enabling VCPKG is very simple: follow the [installation instructions](https://vcpkg.io/en/getting-started) and export the following variable: +### Managing dependencies +DuckDB extensions uses VCPKG for dependency management. Enabling VCPKG is very simple: follow the [installation instructions](https://vcpkg.io/en/getting-started) or just run the following: ```shell -export VCPKG_TOOLCHAIN_PATH=/scripts/buildsystems/vcpkg.cmake +git clone https://github.com/Microsoft/vcpkg.git +./vcpkg/bootstrap-vcpkg.sh +export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake ``` -Note: while using VCPKG for installation is recommended, the build will still work as long as -CMake's `find_package` function is able to locate a compatible openssl version. Alternatively, feel free -to remove the OpenSSL dependency completely to build the example extension without dependencies. +Note: VCPKG is only required for extensions that want to rely on it for dependency management. If you want to develop an extension without dependencies, or want to do your own dependency management, just skip this step. Note that the example extension uses VCPKG to build with a dependency for instructive purposes, so when skipping this step the build may not work without removing the dependency. ### Build steps Now to build the extension, run: @@ -40,7 +34,7 @@ The main binaries that will be built are: - `.duckdb_extension` is the loadable binary as it would be distributed. ## Running the extension -To run the extension code, simply start the shell with `./build/release/duckdb`. +To run the extension code, simply start the shell with `./build/release/duckdb`. This shell will have the extension pre-loaded. Now we can use the features from the extension directly in DuckDB. The template contains a single scalar function `quack()` that takes a string arguments and returns a string: ``` @@ -62,7 +56,7 @@ make test ## Getting started with your own extension After creating a repository from this template, the first step is to name your extension. To rename the extension, run: ``` -python3 ./scripts/set_extension_name.py +python3 ./scripts/bootstrap-template.py ``` Feel free to delete the script after this step. @@ -131,15 +125,13 @@ LOAD ``` ### Versioning of your extension -Extension binaries will only work for the specific DuckDB version they were built for. Since you may want to support multiple -versions of DuckDB for a release of your extension, you can specify which versions to build for in the CI of this template. -By default, the CI will build your extension against the version of the DuckDB submodule, which should generally be the most -recent version of DuckDB. To build for multiple versions of DuckDB, simply add the version to the matrix variable, e.g.: -``` -strategy: - matrix: - duckdb_version: [ '', 'v0.7.0'] -``` +Extension binaries will only work for the specific DuckDB version they were built for. The version of DuckDB that is targeted +is set to the latest stable release for the main branch of the template so initially that is all you need. As new releases +of DuckDB are published however, the extension repository will need to be updated. The template comes with a workflow set-up +that will automatically build the binaries for all DuckDB target architectures that are available in the corresponding DuckDB +version. This workflow is found in `.github/workflows/MainDistributionPipeline.yml`. It is up to the extension developer to keep +this up to date with DuckDB. Note also that its possible to distribute binaries for multiple DuckDB versions in this workflow +by simply duplicating the jobs. ## Setting up CLion From 6510193c9342cd31c09b2b506f26bb9150be14c8 Mon Sep 17 00:00:00 2001 From: Sam Ansmink Date: Wed, 11 Oct 2023 14:39:51 +0200 Subject: [PATCH 2/3] rename bootstrap script --- .github/workflows/ExtensionTemplate.yml | 8 ++++---- docs/NEXT_README.md | 13 ++++++------- ...{set_extension_name.py => bootstrap-template.py} | 2 +- 3 files changed, 11 insertions(+), 12 deletions(-) rename scripts/{set_extension_name.py => bootstrap-template.py} (97%) diff --git a/.github/workflows/ExtensionTemplate.yml b/.github/workflows/ExtensionTemplate.yml index cd21c77..a00f5c1 100644 --- a/.github/workflows/ExtensionTemplate.yml +++ b/.github/workflows/ExtensionTemplate.yml @@ -1,6 +1,6 @@ # # NOTE: this workflow is for testing the extension template itself, -# this workflow will be removed when scripts/set_extension_name.py is run +# this workflow will be removed when scripts/bootstrap-template.py is run # name: Extension Template on: [push, pull_request,repository_dispatch] @@ -64,7 +64,7 @@ jobs: - name: Rename extension run: | - python3 scripts/set_extension_name.py ext_1_a_123b_b11 + python3 scripts/bootstrap-template.py ext_1_a_123b_b11 - name: Build run: | @@ -117,7 +117,7 @@ jobs: - name: Rename extension run: | - python scripts/set_extension_name.py ext_1_a_123b_b11 + python scripts/bootstrap-template.py ext_1_a_123b_b11 - name: Build run: | @@ -166,7 +166,7 @@ jobs: - name: Rename extension run: | - python scripts/set_extension_name.py ext_1_a_123b_b11 + python scripts/bootstrap-template.py ext_1_a_123b_b11 - name: Build run: | diff --git a/docs/NEXT_README.md b/docs/NEXT_README.md index 99c0c5a..96f0482 100644 --- a/docs/NEXT_README.md +++ b/docs/NEXT_README.md @@ -8,15 +8,14 @@ This extension, Quack, allow you to ... . ## Building -### Dependencies -DuckDB extensions use VCPKG for dependency management. To demonstrate that, the example extension in the template links against -OpenSSL. Enabling VCPKG is very simple: follow the [installation instructions](https://vcpkg.io/en/getting-started) and export the following variable: +### Managing dependencies +DuckDB extensions uses VCPKG for dependency management. Enabling VCPKG is very simple: follow the [installation instructions](https://vcpkg.io/en/getting-started) or just run the following: ```shell -export VCPKG_TOOLCHAIN_PATH=/scripts/buildsystems/vcpkg.cmake +git clone https://github.com/Microsoft/vcpkg.git +./vcpkg/bootstrap-vcpkg.sh +export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake ``` -Note: while using VCPKG for installation is recommended, the build will still work as long as -CMake's `find_package` function is able to locate a compatible openssl version. Alternatively, feel free -to remove the OpenSSL dependency completely to build the example extension without dependencies. +Note: VCPKG is only required for extensions that want to rely on it for dependency management. If you want to develop an extension without dependencies, or want to do your own dependency management, just skip this step. Note that the example extension uses VCPKG to build with a dependency for instructive purposes, so when skipping this step the build may not work without removing the dependency. ### Build steps Now to build the extension, run: diff --git a/scripts/set_extension_name.py b/scripts/bootstrap-template.py similarity index 97% rename from scripts/set_extension_name.py rename to scripts/bootstrap-template.py index b78a8b1..ef079e2 100755 --- a/scripts/set_extension_name.py +++ b/scripts/bootstrap-template.py @@ -6,7 +6,7 @@ shutil.copyfile(f'docs/NEXT_README.md', f'README.md') if (len(sys.argv) != 2): - raise Exception('usage: python3 set_extension_name.py ') + raise Exception('usage: python3 bootstrap-template.py ') name_extension = sys.argv[1] From b46330c1a35a582a6b41804e5e1ebf401f45d7e2 Mon Sep 17 00:00:00 2001 From: Sam Ansmink Date: Wed, 11 Oct 2023 16:31:39 +0200 Subject: [PATCH 3/3] fix minor rename issue --- .github/workflows/MainDistributionPipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/MainDistributionPipeline.yml b/.github/workflows/MainDistributionPipeline.yml index 3d91ab3..917320e 100644 --- a/.github/workflows/MainDistributionPipeline.yml +++ b/.github/workflows/MainDistributionPipeline.yml @@ -21,7 +21,7 @@ jobs: duckdb-stable-deploy: name: Deploy extension binaries - needs: duckdb-stable + needs: duckdb-stable-build uses: ./.github/workflows/_extension_deploy.yml secrets: inherit with: