Add QueryHandler Dataflow Abstraction

[ckunki]

• All files below directory logistic_regression/exasol_machine_learning_library/execution/sql_stage_graph_execution
• plus Dependencies in directory execution/stage_graph/
  • sql_stage_train_query_handler.py
  • stage_graph.py
  • stage.py without class UDFStage
• plus related Tests in tests/unit_tests/sql_stage_graph/ and subdirectories
• After that some small adaption will be required

[ckunki]

Affected test files

Clearly related

• Folder tests/unit_tests/sql_stage_graph/
  • test_data_partition.py
  • test_dataset.py
  • test_dependency.py
  • test_object_proxy_reference_counter.py
  • test_object_proxy_reference_counting_bag_with_mocks.py
  • test_sql_stage_train_query_handler_input.py
  • test_find_object_proxies.py
  • test_object_proxy_reference_counting_bag_with_query_handler_context_impl.py
  • test_sql_stage_input_output.py
  • SubFolder stage_graph_execution_query_handler/
    • assert_helper.py
    • state_test_setup.py
    • test_query_handler_using_mock_state.py
    • test_state_diamond.py
    • test_state_reference_counting.py
    • test_state_single_stage.py
    • test_state_two_stage.py
    • test_query_handler_integration.py

Other files below folder tests/ with imports from sql_stage_graph_execution

@tkilias: None of these files is related and hence we don't need to move any of these.

File	Imports
test_output_generator.py	SQLStageInputOutput
create_model_runner_asserts.py	SQLStageGraphExecutionInput, SQLStageInputOutput
create_model_runner_test_setup.py	Dataset, SQLStageGraphExecutionQueryHandler
test_create_model_input_to_dataset_converter.py	Dataset
test_create_model_runner.py	SQLStageInputOutput
test_output_generator.py	DataPartition, Dataset, SQLStageInputOutput
test_create_model_runner_integration.py	DataPartition, Dataset, SQLStageInputOutput
stage_graph_split_test_utils.py	SQLStageInputOutput
test_pydantic_estimator_to_sql_stage_graph.py	SQLStageInputOutput

[ckunki]

Dependencies

from exasol_machine_learning_library.execution

The files marked with ✅ had already been are listed as dependencies needing to be moved, too, in the main comment of the current ticket above.

File	Import	Comment
sql_stage_graph.sql_stage_graph	SQLStageGraph	✅
stage_graph.stage	SQLStage	✅
stage_graph.sql_stage_train_query_handler	SQLStageTrainQueryHandlerInput	✅ Added to list of dependencies above

from exasol_machine_learning_library.execution.sql_stage_graph_execution

✅ These files are planned to be moved, anyway

File	Import
data_partition	DataPartition
dataset	Dataset
dependency	Dependencies
find_object_proxies	find_object_proxies
object_proxy_reference_counter	ObjectProxyReferenceCounter, ReferenceCounterStatus
object_proxy_reference_counting_bag	ObjectProxyReferenceCountingBag
sql_stage_graph_execution_input	SQLStageGraphExecutionInput
sql_stage_graph_execution_query_handler_state	SQLStageGraphExecutionQueryHandlerState, ResultHandlerReturnValue
sql_stage_input_output	SQLStageInputOutput

from exasol_advanced_analytics_framework

✅ These files are already available in AAF

File	Import
query_handler.context.proxy.db_object_name_proxy	ObjectProxy
query_handler.context.proxy.object_proxy	ObjectProxy
query_handler.context.scope_query_handler_context	ScopeQueryHandlerContext
query_handler.query_handler	QueryHandler
query_handler.result	Continue, Finish
query_result.query_result	QueryResult

from exasol_bucketfs_utils_python.abstract_bucketfs_location import AbstractBucketFSLocation

from exasol_data_science_utils_python.schema.column

✅ These files are already available in AAF

File	Import
schema.column	Column
schema.table_like	TableLike
utils.data_classes_runtime_type_check	check_dataclass_types

[ckunki]

Observations
MLL test_find_object_proxies.py skips some test cases

• test_object_proxy_in_table[VIEW_NAME]
• test_object_proxy_in_table[BUCKETFS_LOCATION]
• test_object_proxy_in_view[TABLE_NAME]
• test_object_proxy_in_view[BUCKETFS_LOCATION]
• test_object_proxy_in_column_name[VIEW_NAME]
• test_object_proxy_in_column_name[BUCKETFS_LOCATION]
• test_object_proxy_in_column[VIEW_NAME]
• test_object_proxy_in_column[BUCKETFS_LOCATION]

This is due to pytest.skip() in some test cases