1.3.0: Release Artifact with Scala 2.12

Summary

In this release we added an artifact release for Scala 2.12 version.

Features

• #140: Added release artifact with Scala 2.12 variant

Bugs

• #131: Removed project-keeper check suppression on dependencies.md file

Dependency Updates

Compile Dependency Updates

• Updated com.google.protobuf:protobuf-java:3.22.1 to 3.22.2

Plugin Dependency Updates

• Updated com.diffplug.spotless:spotless-maven-plugin:2.34.0 to 2.35.0
• Updated com.exasol:project-keeper-maven-plugin:2.9.3 to 2.9.6
• Updated org.apache.maven.plugins:maven-deploy-plugin:3.0.0 to 3.1.0
• Updated org.apache.maven.plugins:maven-enforcer-plugin:3.1.0 to 3.2.1

1.2.4: Fixed remote Maven Central release bug

Summary

In this release we fixed the bug due to release to the remote Maven Central repository. Because of this users were not able to reference the spark-connector in their dependencies. Additionally, we updated dependencies and fixed known vulnerabilities.

Bug Fixes

• #123: Fixed release bug that errors when referencing artifact from maven central

Refactoring

• #129: Updated dependencies and fixed vulnerabilities

Dependency Updates

Compile Dependency Updates

• Updated com.exasol:exasol-jdbc:7.1.11 to 7.1.17
• Updated com.google.protobuf:protobuf-java:3.21.8 to 3.22.1

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:6.3.0 to 6.5.1
• Updated com.exasol:test-db-builder-java:3.4.0 to 3.4.2
• Updated org.apache.logging.log4j:log4j-1.2-api:2.19.0 to 2.20.0
• Updated org.apache.logging.log4j:log4j-api:2.19.0 to 2.20.0
• Updated org.apache.logging.log4j:log4j-core:2.19.0 to 2.20.0
• Updated org.mockito:mockito-core:4.8.0 to 5.2.0

Plugin Dependency Updates

• Updated com.diffplug.spotless:spotless-maven-plugin:2.24.0 to 2.34.0
• Updated com.exasol:artifact-reference-checker-maven-plugin:0.4.1 to 0.4.2
• Updated com.exasol:error-code-crawler-maven-plugin:1.1.2 to 1.2.2
• Updated com.exasol:project-keeper-maven-plugin:2.8.0 to 2.9.3
• Updated io.github.zlika:reproducible-build-maven-plugin:0.15 to 0.16
• Updated net.alchim31.maven:scala-maven-plugin:4.7.2 to 4.8.1
• Updated org.apache.maven.plugins:maven-compiler-plugin:3.10.1 to 3.11.0
• Updated org.apache.maven.plugins:maven-deploy-plugin:3.0.0-M1 to 3.0.0
• Updated org.apache.maven.plugins:maven-failsafe-plugin:3.0.0-M5 to 3.0.0-M8
• Updated org.apache.maven.plugins:maven-javadoc-plugin:3.4.0 to 3.4.1
• Updated org.apache.maven.plugins:maven-shade-plugin:3.4.0 to 3.4.1
• Updated org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5 to 3.0.0-M8
• Updated org.codehaus.mojo:flatten-maven-plugin:1.2.7 to 1.3.0
• Updated org.codehaus.mojo:versions-maven-plugin:2.10.0 to 2.14.2
• Removed org.scalastyle:scalastyle-maven-plugin:1.0.0

1.2.3: Spark 3.3

Summary

• Added support for Spark 3.3.
• Removed support for Spark 3.1
• Removed support for Spark 2.4.
• Removed support for Scala 2.12
• Dropped incomplete Scala 2.12 profile from build.

For Spark 3.3 the following vulnerabilities in dependencies were fixed by updating the dependencies:

• CVE-2022-42003: Uncontrolled Resource Consumption in jackson-databind
• CVE-2022-3171: Uncontrolled Resource Consumption ('Resource Exhaustion') in org.apache.commons:commons-text
• sonatype-2022-5820: Improper Restriction of XML External Entity Reference ('XXE') in hadoop-common

Known Remaining Spark 3.3.4 Vulnerabilities and Sonatype Warnings

The following vulnerabilities are known in Spark 3.3.4, but no update is available at the time of this spark-connector update, so instead we evaluate the risks here.

• The command line tool hdfs ec has the known vulnerability sonatype-2022-5732, but the connector is not affected, since it does not use this tool. For more details see HDFS-16766 on the Haddop issue tracker.
• Related to the vulnerability above the code creates DocumentBuilderFactory instances in various other locations, but the collection ticket HADOOP-18469 states that no additional issues are known as a result yet.

Spark 3.2

At the moment no Spark 3.2 version is available where vulnerabilities in dependencies can be fixed or are not exploitable. So we cannot offer a connector for these versions until the upstream project provide updates.

While you can still download older versions of the spark-connector that support Spark 3.2, we recommend updating, because there are no secure 3.2 versions available at the time of this connector release.

Features

• #125: Added support for Spark 3.3

Bugfixes

• #125: Fixed vulnerabilities in dependencies

Dependency Updates

Compile Dependency Updates

• Added com.google.protobuf:protobuf-java:3.21.8
• Added org.apache.commons:commons-text:1.10.0
• Updated org.scala-lang:scala-library:2.13.8 to 2.13.10

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:6.2.0 to 6.3.0
• Updated com.exasol:test-db-builder-java:3.3.4 to 3.4.0
• Added org.apache.logging.log4j:log4j-api:2.19.0

Plugin Dependency Updates

• Updated com.diffplug.spotless:spotless-maven-plugin:2.22.8 to 2.24.0
• Updated net.alchim31.maven:scala-maven-plugin:4.6.3 to 4.7.2
• Updated org.apache.maven.plugins:maven-shade-plugin:3.3.0 to 3.4.0
• Updated org.itsallcode:openfasttrace-maven-plugin:1.5.0 to 1.6.1
• Updated org.scalatest:scalatest-maven-plugin:2.0.2 to 2.2.0

1.2.2: Fix vulnerabilities in dependencies

Summary

This release fixes sonatype-2022-5401 in reload4j.

Features

• #121: Fixed vulnerabilities in dependencies

Dependency Updates

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:6.1.2 to 6.2.0
• Updated com.exasol:hamcrest-resultset-matcher:1.5.1 to 1.5.2
• Updated com.exasol:test-db-builder-java:3.3.3 to 3.3.4
• Updated org.apache.logging.log4j:log4j-1.2-api:2.18.0 to 2.19.0
• Updated org.apache.logging.log4j:log4j-core:2.18.0 to 2.19.0
• Updated org.mockito:mockito-core:4.6.1 to 4.8.0

Plugin Dependency Updates

• Updated com.diffplug.spotless:spotless-maven-plugin:2.22.4 to 2.22.8
• Updated com.exasol:error-code-crawler-maven-plugin:1.1.1 to 1.1.2
• Updated com.exasol:project-keeper-maven-plugin:2.5.0 to 2.8.0
• Updated net.alchim31.maven:scala-maven-plugin:4.6.1 to 4.6.3
• Updated org.apache.maven.plugins:maven-clean-plugin:3.2.0 to 2.5
• Updated org.apache.maven.plugins:maven-enforcer-plugin:3.0.0 to 3.1.0
• Updated org.apache.maven.plugins:maven-install-plugin:2.5.2 to 2.4
• Updated org.apache.maven.plugins:maven-jar-plugin:3.2.2 to 3.3.0
• Updated org.apache.maven.plugins:maven-resources-plugin:3.2.0 to 2.6
• Updated org.apache.maven.plugins:maven-site-plugin:3.12.0 to 3.3

1.2.1 Dependency updates

The Spark Exasol Connector 1.2.1, released 2022-08-04

Bugfixes

• #118: Dependency update to fix vulnerabilities reported by ossindex and broken links
  Updated com.fasterxml.jackson.core and excluded dependencies provided by
  spark-core as current there is no update available.

Dependency Updates

Compile Dependency Updates

• Updated com.exasol:exasol-jdbc:7.1.4 to 7.1.11
• Updated org.scala-lang:scala-library:2.13.5 to 2.13.8

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:5.1.1 to 6.1.2
• Updated com.exasol:test-db-builder-java:3.2.2 to 3.3.3
• Updated org.apache.logging.log4j:log4j-1.2-api:2.17.1 to 2.18.0
• Added org.apache.logging.log4j:log4j-core:2.18.0
• Updated org.mockito:mockito-core:4.2.0 to 4.6.1

Plugin Dependency Updates

• Updated com.diffplug.spotless:spotless-maven-plugin:2.20.0 to 2.22.4
• Updated com.exasol:artifact-reference-checker-maven-plugin:0.4.0 to 0.4.1
• Updated com.exasol:error-code-crawler-maven-plugin:0.7.1 to 1.1.1
• Updated com.exasol:project-keeper-maven-plugin:1.3.4 to 2.5.0
• Updated net.alchim31.maven:scala-maven-plugin:4.5.6 to 4.6.1
• Updated org.apache.maven.plugins:maven-clean-plugin:3.1.0 to 3.2.0
• Updated org.apache.maven.plugins:maven-compiler-plugin:3.9.0 to 3.10.1
• Added org.apache.maven.plugins:maven-failsafe-plugin:3.0.0-M5
• Updated org.apache.maven.plugins:maven-javadoc-plugin:3.3.1 to 3.4.0
• Updated org.apache.maven.plugins:maven-shade-plugin:3.2.4 to 3.3.0
• Updated org.apache.maven.plugins:maven-site-plugin:3.10.0 to 3.12.0
• Updated org.apache.maven.plugins:maven-surefire-plugin:2.12.4 to 3.0.0-M5
• Added org.codehaus.mojo:flatten-maven-plugin:1.2.7
• Updated org.codehaus.mojo:versions-maven-plugin:2.8.1 to 2.10.0
• Updated org.itsallcode:openfasttrace-maven-plugin:1.3.0 to 1.5.0
• Updated org.jacoco:jacoco-maven-plugin:0.8.7 to 0.8.8
• Added org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184
• Updated org.sonatype.ossindex.maven:ossindex-maven-plugin:3.1.0 to 3.2.0
• Updated org.sonatype.plugins:nexus-staging-maven-plugin:1.6.8 to 1.6.13

Added support for certificate fingerprint

Summary

In this release we added an option for providing Exasol certificate fingerprint value. Similarly, we added Spark versions to the released artifacts, improved errors and migrated to Maven based build from SBT build. We also updated Log4J dependency to fix recent CVE-2021-44228 vulnerability.

Features

• #103: Added an option so that users can provide certificate fingerprint

Bugs

• #106: Fixed Log4J CVE-2021-44228 vulnerability
• #111: Fixed issues related to cross releasing with updated release droid workflows
• #112: Updated Log4J dependency to 2.17.0 version

Refactorings

• #54: Added Spark versions to the released artifacts
• #75: Added dependencies vulnerabilities checker
• #94: Added notice about Spark Streaming
• #96: Added unified error codes
• #101: Migrated to Maven based build from SBT
• #108: Removed custom loader class from tests

Dependency Updates

Compile Dependency Updates

• Added com.exasol:error-reporting-java8:0.4.1
• Added com.exasol:exasol-jdbc:7.1.4
• Added com.exasol:sql-statement-builder-java8:4.5.0
• Added org.scala-lang:scala-library:2.13.5

Test Dependency Updates

• Added com.exasol:exasol-testcontainers:5.1.1
• Added com.exasol:hamcrest-resultset-matcher:1.5.1
• Added com.exasol:test-db-builder-java:3.2.2
• Added org.apache.logging.log4j:log4j-1.2-api:2.17.1
• Added org.mockito:mockito-core:4.2.0
• Added org.scalatestplus:scalatestplus-mockito_2.13:1.0.0-M2
• Added org.scalatest:scalatest_2.13:3.2.9

Plugin Dependency Updates

• Added com.diffplug.spotless:spotless-maven-plugin:2.20.0
• Added com.exasol:artifact-reference-checker-maven-plugin:0.4.0
• Added com.exasol:error-code-crawler-maven-plugin:0.7.1
• Added com.exasol:project-keeper-maven-plugin:1.3.4
• Added io.github.evis:scalafix-maven-plugin_2.13:0.1.4_0.9.31
• Added io.github.zlika:reproducible-build-maven-plugin:0.15
• Added net.alchim31.maven:scala-maven-plugin:4.5.6
• Added org.apache.maven.plugins:maven-clean-plugin:3.1.0
• Added org.apache.maven.plugins:maven-compiler-plugin:3.9.0
• Added org.apache.maven.plugins:maven-deploy-plugin:3.0.0-M1
• Added org.apache.maven.plugins:maven-enforcer-plugin:3.0.0
• Added org.apache.maven.plugins:maven-gpg-plugin:3.0.1
• Added org.apache.maven.plugins:maven-install-plugin:2.5.2
• Added org.apache.maven.plugins:maven-jar-plugin:3.2.2
• Added org.apache.maven.plugins:maven-javadoc-plugin:3.3.1
• Added org.apache.maven.plugins:maven-resources-plugin:3.2.0
• Added org.apache.maven.plugins:maven-shade-plugin:3.2.4
• Added org.apache.maven.plugins:maven-site-plugin:3.10.0
• Added org.apache.maven.plugins:maven-source-plugin:3.2.1
• Added org.apache.maven.plugins:maven-surefire-plugin:2.12.4
• Added org.codehaus.mojo:versions-maven-plugin:2.8.1
• Added org.itsallcode:openfasttrace-maven-plugin:1.3.0
• Added org.jacoco:jacoco-maven-plugin:0.8.7
• Added org.scalastyle:scalastyle-maven-plugin:1.0.0
• Added org.scalatest:scalatest-maven-plugin:2.0.2
• Added org.sonatype.ossindex.maven:ossindex-maven-plugin:3.1.0
• Added org.sonatype.plugins:nexus-staging-maven-plugin:1.6.8

1.1.0 - Added Support for Java8

This release adds support for Java 8 runtime environments. We updated sql-statement-builder that was released for Java 11 to sql-statement-builder-java8 version.

Features

• #92: Added support for Java 8.

Dependency Updates

Runtime Dependency Updates

• Removed com.exasol:sql-statement-builder:4.4.0
• Added com.exasol:sql-statement-builder-java8:4.4.1

Test Dependency Updates

• Updated com.exasol:exasol-testcontainers:3.5.1 to 3.5.3
• Updated com.exasol:test-db-builder-java:3.1.0 to 3.1.1
• Updated org.scalatest:scalatest:3.2.5 to 3.2.9
• Updated org.mockito:mockito-core:3.8.0 to 3.11.0

Plugin Updates

• Updated org.wartremover:sbt-wartremover:2.4.13 to 2.4.15
• Updated org.wartremover:sbt-wartremover-contrib:1.3.11 to 1.3.12
• Updated org.scoverage:sbt-scoverage:1.6.1 to 1.8.2
• Updated com.timushev.sbt:sbt-updates:0.5.1 to 0.5.3
• Updated org.xerial.sbt:sbt-sonatype:3.9.5 to 3.9.7
• Updated com.typesafe.sbt:sbt-git:1.0.0 to 1.0.1

1.0.0 - Identifier Fixes

Summary

In this release, we fixed bugs related to quoted identifiers and reserved keywords. In addition, we refactored the integration test and SQL generation using Exasol SQL Statement Builder.

Bug Fixes

• #14: Fixed issue with using Exasol reserved keywords in Spark queries (PR #88).
• #39: Fixed issue related to quoted columns in Spark queries (PR #88).

Refactoring

• #40: Added Exasol Testcontainers, refactored test environment (PR #87).
• #84: Added Exasol SQL Statement Builder for building SQL queries (PR #88).
• #89: Added missing Exasol predicates (PR #91).

Documentation

• #85: Updated documentation with configuration for the Databricks cluster (PR #86).

Dependency Updates

Runtime Dependency Updates

• Added com.exasol:sql-statement-builder:4.4.0
• Updated com.exasol:exasol-jdbc:7.0.0 to 7.0.7
• Updated org.apache.spark:spark-core:2.4.5 to 3.0.1
• Updated org.apache.spark:spark-sql:2.4.5 to 3.0.1

Test Dependency Updates

• Added com.exasol:exasol-testcontainers:3.5.1
• Added com.exasol:test-db-builder-java:3.0.0
• Added com.exasol:hamcrest-resultset-matcher:1.4.0
• Removed org.testcontainers:jdbc
• Removed com.dimafeng:testcontainers-scala
• Updated org.scalatest:scalatest:3.2.2 to 3.2.5
• Updated org.mockito:mockito-core:3.5.13 to 3.8.0
• Updated com.holdenkarau:spark-testing-base:2.4.5_0.14.0 to 3.0.1_1.0.0

Plugin Updates

• Updated sbt.version:1.3.13 to 1.4.7
• Updated org.wartremover:sbt-wartremover:2.4.10 to 2.4.13
• Updated org.wartremover:sbt-wartremover-contrib:1.3.8 to 1.3.11
• Updated com.jsuereth:sbt-pgp:2.0.1 to 2.1.1
• Updated org.xerial.sbt:sbt-sonatype:3.9.4 to 3.9.5
• Removed io.get-coursier:sbt-coursier

0.3.2

Spark Exasol Connector 0.3.2, released 2020-10-01

In this release, we add support for additional JDBC options that can be provided using jdbc_options parameter. In addition, we updated the Exasol JDBC version to 7.0.0.

Features / Improvements

• #76: Enabled extra options for JDBC connection (PR #77).
• #78: Updated Exasol JDBC version to 7.0.0 (PR #79).

Documentation

• #80: Removed auto release setup from Travis CI (PR #83).
• #81: Updated user and development guide (PR #82).

Dependency Updates

• Updated Scala version from 2.12.10 to 2.12.12.
• Updated com.exasol:exasol-jdbc from 6.2.5 to 7.0.0.
• Updated org.mockito:mockito-core from 3.5.7 to 3.5.13.
• Updated com.dimafeng:testcontainers-scala from 0.38.1 to 0.38.4.

Plugin Updates

• Added org.scoverage:sbt-coveralls plugin for additional coverage report.

v0.3.1

Spark Exasol Connector v0.3.1, released 2020-08-31

Features

• #68: Add drop table option (PR #72).

Documentation

• #59: Add dependency list to readme (PR #73).

Dependency Updates

• Updated org.scalatest:scalatest from 3.2.0 to 3.2.2.
• Updated org.mockito:mockito-core from 3.4.6 to 3.5.7.

Plugin Updates

• Removed com.eed3si9n:sbt-buildinfo. It is not used in the project.
• Removed org.danielnixon:sbt-extrawarts. It is not maintained anymore.