Spark inference mode error

[wanweiqiangintel]

After I run training mode with Spark database and get results/spark.sqlite successfully, then run the inference mode and get the error:

I found the method get_plan_preprocessor just implemented in Presto, should I implement it in Spark?

[christophanneser]

Thank you for opening the issue. Yes, currently, the plan preprocessor has just been implemented for Presto. I would highly appreciate it if you could implement the preprocessor also for Spark query plans. Thank you!

[wanweiqiangintel]

@christophanneser, Thank you for your comments! I am wondering if there is a way to reproduce the performance gain in SparkSQL TPC-DS workload as shown in the paper without implementing preprocessor?

[christophanneser]

You can run AutoSteer's training mode, which does not require the query plan preprocessor. Unfortunately, I must not share SparkSQL's exact configuration as it was/is internally used.