Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#26: Added user guide #67

Merged
merged 31 commits into from
Oct 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
65893a2
Initiated the user guide
umitbuyuksahin Sep 29, 2022
d7a6fd6
Added a sample execution figure and explanations
umitbuyuksahin Sep 30, 2022
b665fc7
Merge branch 'main' into doc/#26-add-user-guide
ckunki Oct 7, 2024
1ce9d39
Initial review and update of existing PR
ckunki Oct 7, 2024
e861ab4
Descibed BucketFS connection in the User Guide
ckunki Oct 7, 2024
8c5899e
renamed CustomQueryHandler to ExampleQueryHandler
ckunki Oct 7, 2024
cf2872e
Renamed variable in user guide sample query
ckunki Oct 8, 2024
7655552
Merge branch 'main' into doc/#26-add-user-guide
ckunki Oct 8, 2024
80495a7
Updated user guide
ckunki Oct 9, 2024
da6c79e
Fixed bug in query_handler_runner_udf.py
ckunki Oct 9, 2024
378b1f8
Updated user guide based on empiric results
ckunki Oct 9, 2024
ced8c82
Fixed unit tests
ckunki Oct 9, 2024
df56d51
Apply suggestions from code review
ckunki Oct 10, 2024
82250db
Renamed CustomQueryHandler to ExampleQueryHandler
ckunki Oct 10, 2024
b61626c
Moved instructions for building the SLC to the developer guide
ckunki Oct 10, 2024
8ae62ad
Fixed review findings
ckunki Oct 10, 2024
6242c37
Added sample file for current experiments
ckunki Oct 10, 2024
37dc979
Fixed some review findings in the User Guide
ckunki Oct 11, 2024
81be6df
Merge branch 'main' into doc/#26-add-user-guide
ckunki Oct 11, 2024
20b3d8a
Apply suggestions from code review
ckunki Oct 11, 2024
58e717b
Updated images with sample execution
ckunki Oct 11, 2024
09a0584
Removed sample file udf-6.sql
ckunki Oct 11, 2024
e1843bd
Formatted paragraphs and remove comments
ckunki Oct 11, 2024
b466809
Remove comment in user guide
ckunki Oct 11, 2024
b335b1d
Apply suggestions from code review
ckunki Oct 14, 2024
14860aa
Fixed review findings
ckunki Oct 14, 2024
cd47569
Apply suggestions from code review
ckunki Oct 14, 2024
5ed9bba
Fixed review findings
ckunki Oct 14, 2024
616a9f8
Use python version 3.10 for version check
ckunki Oct 14, 2024
2d968f9
Update doc/user_guide/user_guide.md
ckunki Oct 15, 2024
b1372f3
Added separate section for example
ckunki Oct 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions .github/workflows/check-packaging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,31 +15,31 @@ jobs:

steps:
- uses: actions/checkout@v4

- name: Setup Python & Poetry Environment
uses: exasol/python-toolbox/.github/actions/[email protected]
with:
python-version: ${{ matrix.python-version }}
- name: Install Lua environment
run: ./scripts/install_lua_environment.sh

- name: Install Development Environment
run: poetry run -- nox -s install_dev_env

- name: Poetry install
run: ./scripts/run_in_dev_env.sh poetry install
run: poetry run -- nox -s run_in_dev_env -- poetry install

- name: Run packaging update
# re-generates / amalgate the lua script
# refactor pre-commit as nox task
# and call in pre-commit
run: bash ./githooks/pre-commit

- name: Show changes on working copy
# check if re-generated lua script is still up-to-date
run: git status --porcelain=v1 -uno

- name: Show diff on working copy
run: git diff --cached

- name: Check if packaging changed
run: |
[ -z "$(git status --porcelain=v1 -uno 2>/dev/null)" ]
2 changes: 2 additions & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ jobs:

- name: Setup Python & Poetry Environment
uses: exasol/python-toolbox/.github/actions/[email protected]
with:
python-version: "3.10"

lint-job:
name: Linting (Python-${{ matrix.python-version }})
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/tests-integration-db.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ jobs:
with:
python-version: ${{ inputs.python-version }}

- name: Install Lua environment
run: poetry run -- nox -s install_lua_environment
- name: Install Development Environment
run: poetry run -- nox -s install_dev_env

- name: Poetry install
run: poetry run -- nox -s run_in_dev_env -- poetry install
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/tests-unit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ jobs:
with:
python-version: ${{ inputs.python-version }}

- name: Install Lua environment
run: poetry run -- nox -s install_lua_environment
- name: Install Development Environment
run: poetry run -- nox -s install_dev_env

- name: Poetry install
run: poetry run -- nox -s run_in_dev_env -- poetry install
Expand Down
1 change: 1 addition & 0 deletions doc/changes/changes_0.1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ Code name:
### Documentation

* #9: Added README file
* #26: Added user guide

## Dependency Updates

Expand Down
64 changes: 61 additions & 3 deletions doc/developer_guide/developer_guide.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,66 @@
# Developer Guide


In this developer guide we explain how you can build this project.
The developer guide explains how to maintain and develop the Advanced Analytics Framework (AAF).

* [developer_environment](developer_environment.md)
* [building_documentation](building_documentation.md)

## Building and Installing the AAF Script Language Container (SLC)

The following command builds the SLC for the AAF

```shell
poetry run nox -s build_language_container
```

Installing the SLC ins described in the [AAF User Guide](../user_guide/user_guide.md#script-language-container-slc).

## Running Tests

AAF comes with different automated tests implemented in different programming languages and requiring different environments:

| Language | Category | Database | Environment |
|----------|---------------------------------|----------|-------------|
| Python | Unit tests | no | poetry |
tkilias marked this conversation as resolved.
Show resolved Hide resolved
| Python | Integration tests with database | yes | _dev_env_ |
| Python | Integration tests w/o database | no | _dev_env_ |
| Lua | Unit tests | no | _dev_env_ |

### Development Environment

For tests marked with Environment _dev_env_ you need to
* Setup a Development Environment
* Add the AAF to it
* Run the tests in the Development Environment

The Development Environment
* Activates AAF's conda environment containing a specific versino of Lua
* Sets the environment variables `LUA_PATH`, `LUA_CPATH`, and `PATH` for executing lua scripts

The following commands install the Development Environment and add the AAF
```shell
poetry run -- nox -s install_dev_env
poetry run -- nox -s run_in_dev_env -- poetry install
```

### Python Unit Tests

You can execute the unit tests without special preparation in the regular poetry environment:

```shell
poetry run pytest tests/unit_tests
```

### Python Integration Tests with and w/o database

The following commands run integration tests w/o and with database
```shell
poetry run -- nox -s run_python_test -- -- tests/integration_tests/without_db/
poetry run -- nox -s run_python_test -- -- --backend=onprem tests/integration_tests/with_db/
```

### Lua Unit Tests

The following command executes the Lua Unit Tests:
```shell
poety run nox -s run_lua_unit_tests
```
Binary file added doc/images/sample_execution.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion doc/images/system_design_diagram.drawio

This file was deleted.

Loading
Loading