Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#38: Add extension #39

Merged
merged 33 commits into from
Nov 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
198280d
add v8 to ci
pj-spoelders Oct 30, 2023
c7b3c6c
Fix version increment
kaklakariada Nov 8, 2023
086dc8f
Upgrade Exasol version
kaklakariada Nov 8, 2023
c26853d
Run PK fix
kaklakariada Nov 8, 2023
aba16c7
Configure runner OS with project keeper
kaklakariada Nov 8, 2023
905dc5f
Update references
kaklakariada Nov 8, 2023
8b278e8
Use fast startup oracle container
kaklakariada Nov 8, 2023
a013199
#38: Add extension
kaklakariada Nov 8, 2023
2d5ba26
Adapt extension tests
kaklakariada Nov 8, 2023
b7c4175
Update workflows for extension
kaklakariada Nov 8, 2023
e4c1253
Integrate extension into maven build
kaklakariada Nov 8, 2023
fbda0a4
Add extension to project keeper
kaklakariada Nov 8, 2023
db85f4f
Fix extension unit tests
kaklakariada Nov 8, 2023
db0ff8b
Fix extension integration test
kaklakariada Nov 8, 2023
d67cb4a
Don't include Oracle JDBC driver in packaged JAR
kaklakariada Nov 8, 2023
fafdd12
Simplify JDBC driver installation
kaklakariada Nov 8, 2023
6bf6150
Merge branch 'refactor/add_v8_to_ci' into add-extension
kaklakariada Nov 9, 2023
81f513a
Run PK fix
kaklakariada Nov 9, 2023
f7819a1
Fix compile error
kaklakariada Nov 9, 2023
b10a9e4
Exclude vulnerabilities without a fix
kaklakariada Nov 9, 2023
b4b3f79
Fix extension build order
kaklakariada Nov 9, 2023
d2e7789
Fix sonar warning
kaklakariada Nov 9, 2023
c969219
Add JDBC driver to required files
kaklakariada Nov 9, 2023
706afc5
Update changelog
kaklakariada Nov 9, 2023
e7fda45
#38: Add required JDBC driver
kaklakariada Nov 9, 2023
77ed49a
Improve Javadoc
kaklakariada Nov 10, 2023
ff43070
Upgrade exasol-testcontainers
kaklakariada Nov 10, 2023
3f60c9c
Skip integration tests with Exasol 8
kaklakariada Nov 10, 2023
202f988
Fix setup order
kaklakariada Nov 10, 2023
cbdfb42
Merge branch 'refactor/add_v8_to_ci' into add-extension
kaklakariada Nov 10, 2023
857eaac
Code cleanup
kaklakariada Nov 10, 2023
9dcd0d7
Merge remote-tracking branch 'origin/main' into add-extension
kaklakariada Nov 10, 2023
487e40f
Apply suggestions from code review
kaklakariada Nov 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ doc/changes/changelog.md linguist-genera
.github/workflows/ci-build-next-java.yml linguist-generated=true
.github/workflows/dependencies_check.yml linguist-generated=true
.github/workflows/release_droid_print_quick_checksum.yml linguist-generated=true
.github/workflows/release_droid_upload_github_release_assets.yml linguist-generated=true
.github/workflows/release_droid_prepare_original_checksum.yml linguist-generated=true

.settings/org.eclipse.jdt.core.prefs linguist-generated=true
.settings/org.eclipse.jdt.ui.prefs linguist-generated=true
extension/package-lock.json linguist-generated=true
17 changes: 17 additions & 0 deletions .github/workflows/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,23 @@ jobs:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: "1.20"
cache: false

- name: Cache Go modules
uses: actions/cache@v3
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('pom.xml') }}
restore-keys: |
${{ runner.os }}-go-

- name: Enable testcontainer reuse
run: echo 'testcontainers.reuse.enable=true' > "$HOME/.testcontainers.properties"
- name: Run tests and build with Maven
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/release_droid_prepare_original_checksum.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ jobs:
distribution: "temurin"
java-version: 11
cache: "maven"
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.20"
- name: Enable testcontainer reuse
run: echo 'testcontainers.reuse.enable=true' > "$HOME/.testcontainers.properties"
- name: Run tests and build with Maven
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/release_droid_upload_github_release_assets.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,21 @@ jobs:
run: |
cd target
find . -maxdepth 1 -name \*.jar -exec bash -c 'sha256sum {} > {}.sha256' \;
- name: Generate sha256sum files for extension
run: |
cd extension/dist/
sha256sum oracle-vs-extension.js > oracle-vs-extension.js.sha256

- name: Upload assets to the GitHub release draft
uses: shogo82148/actions-upload-release-asset@v1
with:
upload_url: ${{ github.event.inputs.upload_url }}
asset_path: target/*.jar
- name: Upload extension
uses: shogo82148/actions-upload-release-asset@v1
with:
upload_url: ${{ github.event.inputs.upload_url }}
asset_path: extension/dist/oracle-vs-extension.js*
- name: Upload sha256sum files
uses: shogo82148/actions-upload-release-asset@v1
with:
Expand Down
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,10 @@ pom.xml.versionsBackup
*.flattened-pom.xml
/.apt_generated/
/.apt_generated_tests/

# Extension
/extension/node_modules/
/extension/dist/
/extension/coverage/
/extension/src/extension-config.ts
kaklakariada marked this conversation as resolved.
Show resolved Hide resolved
/extension-test.properties
6 changes: 6 additions & 0 deletions .project-keeper.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ sources:
- integration_tests
- udf_coverage
- jar_artifact
- type: npm
path: extension/package.json
version:
fromSource: pom.xml
excludes:
- "E-PK-CORE-18: Outdated content: '.github/workflows/ci-build.yml'"
- "E-PK-CORE-18: Outdated content: '.github/workflows/release_droid_upload_github_release_assets.yml'"
- "E-PK-CORE-18: Outdated content: '.github/workflows/release_droid_prepare_original_checksum.yml'"
ciBuildRunnerOS: ubuntu-20.04
120 changes: 69 additions & 51 deletions dependencies.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 33 additions & 4 deletions doc/changes/changes_2.4.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,64 @@ Code name: Test with Exasol 8

## Summary

This release adds integration tests using Exasol 8.
This release adds integration tests using Exasol 8 and the extension code making VSORA available in the [Exasol Extension Manager](https://github.com/exasol/extension-manager). Please note that the VSORA extension expects the Oracle JDBC driver `ojdbc8.jar` in BucketFS. If this file is missing, the extension won't be available in Extension Manager.

**Note:** This release removes the Oracle JDBC driver from the adapter JAR file that was accidentally included in version 2.1.0. This means that you will need to define the `ADAPTER SCRIPT` specifying both the adapter JAR and the JDBC driver JAR as described in the [user guide](../user_guide/oracle_user_guide.md#installing-the-adapter-script).

**Known issues:** After adding integration tests for the extension, this release contains the following vulnerabilities in third party test dependencies:

* CVE-2023-4586 in `io.netty:netty-handler:jar:4.1.94.Final:test`
* CVE-2020-36641 in `fr.turri:aXMLRPC:jar:1.13.0:test`

## Feature

* #33: Added tests with Exasol 8
* #38: Added extension for Extension Manager

## Dependency Updates

### Compile Dependency Updates
### Virtual Schema for Oracle

#### Compile Dependency Updates

* Removed `com.oracle.database.jdbc:ojdbc8:23.3.0.23.09`

### Test Dependency Updates
#### Test Dependency Updates

* Updated `com.exasol:exasol-testcontainers:6.6.2` to `6.6.3`
* Added `com.exasol:extension-manager-integration-test-java:0.5.6`
* Updated `com.exasol:hamcrest-resultset-matcher:1.6.1` to `1.6.2`
* Added `com.oracle.database.jdbc:ojdbc8:23.3.0.23.09`
* Updated `nl.jqno.equalsverifier:equalsverifier:3.15.2` to `3.15.3`
* Updated `org.jacoco:org.jacoco.agent:0.8.10` to `0.8.11`
* Updated `org.junit.jupiter:junit-jupiter:5.10.0` to `5.10.1`
* Updated `org.mockito:mockito-junit-jupiter:5.5.0` to `5.7.0`

### Plugin Dependency Updates
#### Plugin Dependency Updates

* Updated `com.exasol:error-code-crawler-maven-plugin:1.3.0` to `1.3.1`
* Updated `com.exasol:project-keeper-maven-plugin:2.9.12` to `2.9.15`
* Updated `org.apache.maven.plugins:maven-clean-plugin:2.5` to `3.3.2`
* Updated `org.apache.maven.plugins:maven-enforcer-plugin:3.4.0` to `3.4.1`
* Added `org.codehaus.mojo:exec-maven-plugin:3.1.0`
* Updated `org.codehaus.mojo:versions-maven-plugin:2.16.0` to `2.16.1`
* Updated `org.jacoco:jacoco-maven-plugin:0.8.10` to `0.8.11`
* Updated `org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184` to `3.10.0.2594`

### Extension

#### Compile Dependency Updates

* Added `@exasol/extension-manager-interface:0.4.1`

#### Development Dependency Updates

* Added `eslint:^8.53.0`
* Added `@typescript-eslint/parser:^6.10.0`
* Added `ts-jest:^29.1.1`
* Added `@types/jest:^29.5.8`
* Added `typescript:^5.2.2`
* Added `@typescript-eslint/eslint-plugin:^6.10.0`
* Added `jest:29.7.0`
* Added `ts-node:^10.9.1`
* Added `esbuild:^0.19.5`
9 changes: 6 additions & 3 deletions doc/user_guide/oracle_user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ See also [Adapter Properties for JDBC-Based Virtual Schemas](https://github.com/

Exasol provides the `IMPORT FROM ORA` command for loading data from Oracle. It is possible to create a virtual schema that uses `IMPORT FROM ORA` instead of JDBC to communicate with Oracle. Both options are intended to support the same features. `IMPORT FROM ORA` almost always offers better performance since it is implemented natively.

This behavior is toggled by the Boolean `IMPORT_FROM_ORA` variable. Note that a JDBC connection to Oracle is still required to fetch metadata. In addition, a "direct" connection to the Oracle database is needed.
This behavior is toggled by the boolean `IMPORT_FROM_ORA` variable. Note that a JDBC connection to Oracle is still required to fetch metadata. In addition, a "direct" connection to the Oracle database is needed.

### Deploying the Oracle Instant Client

Expand All @@ -101,7 +101,7 @@ To be able to communicate with Oracle, you first need to supply Exasol with the
Having deployed the Oracle Instant Client, a connection to your Oracle database can be set up.

```sql
CREATE CONNECTION ORA_CONNECTION
CREATE OR REPLACE CONNECTION ORA_CONNECTION
TO '(DESCRIPTION =
(ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)
(HOST = <host>)
Expand Down Expand Up @@ -132,9 +132,12 @@ CREATE VIRTUAL SCHEMA <virtual schema name>
IMPORT_FROM_ORA = 'true'
ORA_CONNECTION_NAME = 'ORA_CONNECTION';
```

### Auto generated datatype mapping list while using IMPORT_FROM_ORA.

Using `IMPORT FROM ORA` might lead to some unexpected datatype mappings. Unlike for a JDBC connection there's no explicit data mapping being generated when using `IMPORT FROM ORA`.
As a current stopgap solution for this issue we now (starting from version 2.2.0) also provide a `GENERATE_JDBC_DATATYPE_MAPPING_FOR_OCI` switch you can specify and enable when creating the virtual schema.

```sql
CREATE VIRTUAL SCHEMA <virtual schema name>
USING ADAPTER.JDBC_ADAPTER
Expand All @@ -145,6 +148,7 @@ CREATE VIRTUAL SCHEMA <virtual schema name>
GENERATE_JDBC_DATATYPE_MAPPING_FOR_OCI = 'true'
ORA_CONNECTION_NAME = 'ORA_CONNECTION';
```

This will add explicit datatype mapping to the generated command when using `IMPORT FROM ORA`.

Example:
Expand Down Expand Up @@ -222,4 +226,3 @@ In the following matrix you find combinations of JDBC driver and dialect version
| 2.0.0 | Oracle XE 11g | instantclient-basic-linux | x64-12.1.0.2.0 |
| 2.4.2 | Oracle XE 21c | ojdbc8 | 23.3.0.23.09 |
| 2.4.2 | Oracle XE 21c | instantclient-basic-linux | x64-12.1.0.2.0 |

11 changes: 11 additions & 0 deletions extension/.eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
module.exports = {
root: true,
parser: '@typescript-eslint/parser',
parserOptions: {
tsconfigRootDir: __dirname,
project: ['./tsconfig.json'],
},
plugins: ['@typescript-eslint'],
extends: ['eslint:recommended', 'plugin:@typescript-eslint/recommended', 'plugin:@typescript-eslint/recommended-requiring-type-checking'],
ignorePatterns: ["src/**/*.test.ts"],
};
Loading