Skip to content

Commit

Permalink
#132 Fix wording in hands-on-guide for Minio (#133)
Browse files Browse the repository at this point in the history
  • Loading branch information
kaklakariada authored Aug 29, 2023
1 parent e97286c commit f846965
Show file tree
Hide file tree
Showing 11 changed files with 48 additions and 29 deletions.
2 changes: 1 addition & 1 deletion cloudSetup/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ variable "deputy" {

# Use a short project tag. Long tags will cause the exasol cluster creation to fail (see https://github.com/exasol/cloudformation-aws-exasol/issues/3)
variable "project" {
default = "S3VS"
default = "VSS3"
}

variable "additional_tags" {
Expand Down
1 change: 1 addition & 0 deletions doc/changes/changelog.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions doc/changes/changes_2.8.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# S3 Document Files Virtual Schema 2.8.1, released 2023-??-??

Code name:

## Summary

## Documentation

* #132: Fixed wording in hands-on-guide for MinIO

## Dependency Updates

### Virtual Schema for Document Data in Files on AWS S3

#### Plugin Dependency Updates

* Updated `com.exasol:project-keeper-maven-plugin:2.9.10` to `2.9.11`
* Updated `org.apache.maven.plugins:maven-enforcer-plugin:3.3.0` to `3.4.0`
6 changes: 3 additions & 3 deletions doc/hands_on/hands_on.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ For the document Virtual Schemas, and by that also the S3 Virtual Schema, this i
To install the Virtual Schema adapter, download its latest jar from the [releases](https://github.com/exasol/s3-document-files-virtual-schema/releases) and upload to BucketFS:

``` shell script
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
```

(If you have never used BucketFS, you can check out [its documentation](https://docs.exasol.com/database_concepts/bucketfs/bucketfs.htm))
Expand All @@ -75,7 +75,7 @@ CREATE SCHEMA ADAPTER;
--/
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/

--/
Expand All @@ -85,7 +85,7 @@ CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
CONNECTION_NAME VARCHAR(500))
EMITS(...) AS
%scriptclass com.exasol.adapter.document.UdfEntryPoint;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/
```

Expand Down
24 changes: 12 additions & 12 deletions doc/hands_on/hands_on_minio.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Introduction

VSS3 supports using two different variants of S3 buckets:
Virtual Schema for Document Files in S3 (VSS3) supports using two different variants of S3 buckets:
* Real S3 buckets in the AWS cloud
* S3 buckets as emulated by [MinIO](https://min.io)'s S3-compatible interface

Expand All @@ -18,8 +18,8 @@ This Hands-On Guide describes how to access an S3 bucket emulated by a MinIO Doc

Select a user name and a password for the MinIO Docker Container. Throughout this Hands On Guide these strings will be named as specified in column *Reference*:

| MinIO Terminology | AWS terminology | Reference |
|--------------------------------------|-----------------------|-----------|
| MinIO Terminology | AWS terminology | Reference |
|--------------------------------------|-----------------------|----------------|
| User name for MinIO Docker Container | `<AWS ACCESS KEY ID>` | `$MINIO_USER` |
| Password for MinIO Docker Container | `<AWS SECRET KEY>` | `$MINIO_PASS` |

Expand Down Expand Up @@ -50,8 +50,8 @@ export MINIO_BUCKET=my-bucket
docker run --detach --name minio \
-p 9000:9000 \
-p 9001:9001 \
-e "MINIO_ROOT_USER=$MUSER" \
-e "MINIO_ROOT_PASSWORD=$MPASS" \
-e "MINIO_ROOT_USER=$MINIO_USER" \
-e "MINIO_ROOT_PASSWORD=$MINIO_PASS" \
-it minio/minio:latest \
server /data --console-address ":9001"
```
Expand All @@ -73,7 +73,7 @@ In the output of `docker ps` you can also see the ports forwarded to your localh
For adding content to MinIO you can use the MinIO client `mc`.
Its [manual](https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart) also contains links to download a binary for various platforms.

The following following comand defines alias `myminio` for your MinIO server using the [selected credentials](#credentials):
The following following command defines alias `myminio` for your MinIO server using the [selected credentials](#credentials):

```shell
mc alias set myminio http://$MINIO_HOST:9000/ $MINIO_USER $MINIO_PASS
Expand All @@ -90,9 +90,9 @@ mc mb myminio/$MINIO_BUCKET
Download the [sample Json files](./books) and upload them to the emulated S3 bucket:

```shell
mc cp book-1.json myminio/$MBUCKET/
mc cp book-2.json myminio/$MBUCKET/
mc cp book-3.json myminio/$MBUCKET/
mc cp book-1.json myminio/$MINIO_BUCKET/
mc cp book-2.json myminio/$MINIO_BUCKET/
mc cp book-3.json myminio/$MINIO_BUCKET/
```

## Setting up the Exasol Database
Expand All @@ -102,7 +102,7 @@ mc cp book-3.json myminio/$MBUCKET/
Run Exasol database, e.g. as [Docker-DB](https://hub.docker.com/r/exasol/docker-db/tags) in a Docker Container and follow the instructions in the following sections of document [Hands-on S3-Virtual-Schema](hands_on.md) to execute the required SQL statements:

* Section [ Installation](hands_on.md#installation)
* Download S3VS latest jar and upload it to BucketFS
* Download VSS3 latest jar and upload it to BucketFS
* Create Java Adapter Script
* Create Java Set Script
* Section [Creating a Mapping Definition](hands_on.md#creating-a-mapping-definition) to create a mapping definition and upload it to BucketFS.
Expand All @@ -127,11 +127,11 @@ CREATE OR REPLACE CONNECTION S3_CONNECTION
```

Please note:
* Your SQL client (e.g. [DbVisualizer](https://confluence.dbvis.com)) may not be able to resolve shell variables `$MUSER`, `MPASS`, `$MHOST`, `$MBUCKET`.
* Your SQL client (e.g. [DbVisualizer](https://confluence.dbvis.com)) may not be able to resolve shell variables `$MINIO_USER`, `MINIO_PASS`, `$MINIO_HOST`, `$MINIO_BUCKET`.
* So please replace these variables by the actual values described in section [Variables](#variables).

### Create a Virtual Schema and Run Your First Query

Follow additional instructions in document *Hands-on S3-Virtual-Schema*:
* [Creating the Virtual Schema](#creating-the-virtual-schema)
* [Creating the Virtual Schema](hands_on.md#creating-the-virtual-schema)
* [First Query](hands_on.md#first-query) to execute your first query and watch the results in your SQL client.
6 changes: 3 additions & 3 deletions doc/hands_on/hands_on_parquet.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ For the document Virtual Schemas, and by that also the S3 Virtual Schema, this i
To install the Virtual Schema adapter, download its latest jar from the [releases](https://github.com/exasol/s3-document-files-virtual-schema/releases) and upload to BucketFS:

``` shell script
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
```

(If you have never used BucketFS, you can check out [its documentation](https://docs.exasol.com/database_concepts/bucketfs/bucketfs.htm))
Expand All @@ -59,7 +59,7 @@ CREATE SCHEMA ADAPTER;

CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/

CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
Expand All @@ -68,7 +68,7 @@ CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
CONNECTION_NAME VARCHAR(500))
EMITS(...) AS
%scriptclass com.exasol.adapter.document.UdfEntryPoint;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/
```

Expand Down
6 changes: 3 additions & 3 deletions doc/user_guide/user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Next create the Adapter Script:
```sql
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/
```

Expand All @@ -30,7 +30,7 @@ CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
CONNECTION_NAME VARCHAR(500))
EMITS(...) AS
%scriptclass com.exasol.adapter.document.UdfEntryPoint;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/
```

Expand Down Expand Up @@ -126,7 +126,7 @@ In DbVisualizer use exactly this command:
--/
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/vs/document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar;
%jar /buckets/bfsdefault/default/vs/document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar;
/
```

Expand Down
4 changes: 2 additions & 2 deletions pk_generated_parent.pom

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>s3-document-files-virtual-schema</artifactId>
<version>2.8.0</version>
<version>2.8.1</version>
<name>Virtual Schema for document data in files on AWS S3</name>
<description>Adapter for document data access from files from AWS S3.</description>
<url>https://github.com/exasol/s3-document-files-virtual-schema/</url>
Expand Down Expand Up @@ -229,7 +229,7 @@
<plugin>
<groupId>com.exasol</groupId>
<artifactId>project-keeper-maven-plugin</artifactId>
<version>2.9.10</version>
<version>2.9.11</version>
<executions>
<execution>
<goals>
Expand Down Expand Up @@ -348,7 +348,7 @@
<parent>
<artifactId>s3-document-files-virtual-schema-generated-parent</artifactId>
<groupId>com.exasol</groupId>
<version>2.8.0</version>
<version>2.8.1</version>
<relativePath>pk_generated_parent.pom</relativePath>
</parent>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@

public class IntegrationTestSetup implements AutoCloseable {
public static final Path CLOUD_SETUP_CONFIG = Path.of("cloudSetup/generated/testConfig.json");
public static final String ADAPTER_JAR = "document-files-virtual-schema-dist-7.3.3-s3-2.8.0.jar";
public static final String ADAPTER_JAR = "document-files-virtual-schema-dist-7.3.3-s3-2.8.1.jar";
public static final Path ADAPTER_JAR_LOCAL_PATH = Path.of("target", ADAPTER_JAR);
public final String s3BucketName;
private final ExasolTestSetup exasolTestSetup = new ExasolTestSetupFactory(CLOUD_SETUP_CONFIG).getTestSetup();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ private String getMappingDefinitionForSmallJsonFiles() {
private void createTestSetupWithSmallJsonFiles(final int numberOfJsonFiles) throws IOException {
createBucketIfNotExists(SMALL_JSON_FILES_FIXTURE_BUCKET);
final AwsCredentialsProvider awsCredentialsProvider = TestConfig.instance().getAwsCredentialsProvider();
final Map<String, String> tags = Map.of("exa:project", "S3VS", "exa:owner", TestConfig.instance().getOwner());
final Map<String, String> tags = Map.of("exa:project", "VSS3", "exa:owner", TestConfig.instance().getOwner());
new SmallJsonFilesTestSetup(awsCredentialsProvider, tags, SMALL_JSON_FILES_FIXTURE_BUCKET)
.setup(numberOfJsonFiles);
}
Expand Down

0 comments on commit f846965

Please sign in to comment.