Version 0.5.0

What's Changed

• Fix incorrect result in show index statement by @dai-chen in #332
• Bump Flint version to 0.5.0 by @dai-chen in #343
• Enhance index monitor to terminate streaming job on consecutive errors by @dai-chen in #346
• Updating security reachout email by @varun-lodaya in #340
• Transition Flint index state to Failed upon refresh job termination by @dai-chen in #362
• Pre-validate duplicate columns in materialized view query by @dai-chen in #359
• Fix index state stuck in refreshing when streaming job exits early by @dai-chen in #370
• Read dataSourceName from FlintOptions and avoid passing as args by @seankao-az in #378
• Refactor static method for OpenSearch client utils by @seankao-az in #377
• [Refactor] Introduce flint-commons for models and interfaces by @noCharger in #373
• Extract metadata log operations from FlintClient into FlintMetadataLogService by @seankao-az in #379
• Support nested indexed field in Flint skipping index SQL statement by @dai-chen in #366
• Support custom metadata log service implementation by @seankao-az in #389
• Abstracting source relations for enhanced covering index rewriting by @dai-chen in #391
• Add OpenSearchCatalog to enable direct access OpenSearch index in Spark SQL by @penghuo in #399
• Enhance Flint Spark API error reporting with centralized handler by @dai-chen in #348
• Unquote text and identifiers in PPL parsing by @seankao-az in #393
• handle MetaException with glue AccessDeniedException by @noCharger in #410
• support shard level split on read path by @penghuo in #402
• Pre-validate checkpoint location write permission by @dai-chen in #414
• Separate metadata log entry data model and persistence by @seankao-az in #406
• Add scheduler_mode index option by @noCharger in #415
• Enhance query rewriter rule to support partial covering index by @dai-chen in #409
• Store error message for streaming job execution in Flint metadata log by @dai-chen in #433
• Add create Pit api and fix sigv4 bug by @penghuo in #434
• Support custom extension conf by @noCharger in #438
• Revert OpenSearch Version to 2.6 by @penghuo in #444
• Remove unimplemented syntax by @ykmr1224 in #439
• Update README to reflect available commands by @ykmr1224 in #447
• Add FlintJob integration test with EMR serverless by @penghuo in #449
• [Bugfix] Insights on query execution error by @noCharger in #475
• Disable unsupported PPL function expressions by @ykmr1224 in #478
• Add error output column to show Flint index statement by @dai-chen in #436
• [Doc] Checklist to fix issue "could not find Docker environment" on macOS by @LantaoJin in #477
• Translate PPL-builtin functions to Spark-builtin functions by @LantaoJin in #448
• Translate Eval Command by @LantaoJin in #499
• Fix SigV4 signature when connecting to OpenSearchServerless by @ykmr1224 in #473
• Support more PPL builtin functions by adding a name mapping by @LantaoJin in #504
• Add OpenSearchTable in flint core by @penghuo in #479
• Reorganize IT directory to prevent unintentional execution from UT by @dai-chen in #501
• Add config on query loop execution frequency by @noCharger in #411
• Use refresh policy from config by @ykmr1224 in #530
• Add PPL describe command by @YANG-DB in #541
• Terminate streaming job when index data is deleted by @dai-chen in #500
• Upgrade Spark 3.5.1 by @penghuo in #525
• [Backport 0.5] Add statement timeout by @opensearch-trigger-bot in #550
• [Backport 0.5] Remove query rewrite for LogsTable skipping index by @opensearch-trigger-bot in #553
• [Backport 0.5] Fix refresh policy back to WAIT_FOR other than writing query result by @opensearch-trigger-bot in #557
• [Backport 0.5] Lateral eval expressions are supported after Spark upgrading by @opensearch-trigger-bot in #561
• [Backport 0.5] Translate PPL dedup Command Part 1: allowedDuplication=1 by @opensearch-trigger-bot in #566
• [Backport 0.5] Add rate limiter for bulk request by @opensearch-trigger-bot in #571
• [Backport 0.5] Abstract service for accessing Flint index metadata by @opensearch-trigger-bot in #575
• [Backport 0.5] Update PPL describe command by @opensearch-trigger-bot in #578
• [Backport 0.5] Implement stddev_samp and stddev_pop ppl stats function by @opensearch-trigger-bot in #581
• [Backport 0.5] Top & Rare PPL commands support by @opensearch-trigger-bot in #583
• [Backport 0.5] Nested fields query fix by @opensearch-trigger-bot in #585
• [Backport 0.5] Add percentile PPL function by @opensearch-trigger-bot in #587
• [Backport 0.5] Add conf for specifying flint checkpoint location by @opensearch-trigger-bot in #589
• [Backport 0.5] Fix distinct_count ppl stats function by @opensearch-trigger-bot in #590
• [Backport 0.5] PPL Parse command by @opensearch-trigger-bot in #597
• [Backport 0.5] Add support of GroK command including default patterns by @opensearch-trigger-bot in #610
• [Backport 0.5] Translate PPL dedup Command Part 2: allowedDuplication>1 by @opensearch-trigger-bot in #615
• [Backport 0.5] Add UT and IT for 2+ level aggregations PPL command by @opensearch-trigger-bot in #613
• [Backport 0.5] Remove checkpoint folder when vacuuming index by @opensearch-trigger-bot in #629
• [Backport 0.5] [Refactor] Extend REPL to support external metadata storage and data storage by @opensearch-trigger-bot in #604
• [Backport 0.5] Update checkpoint location on alter path by @opensearch-trigger-bot in #631
• [Backport 0.5] Refactor FlintJob with FlintStatement and StatementExecutionManager by @opensearch-trigger-bot in #636
• [Backport 0.5] Ppl patterns command by @opensearch-trigger-bot in #639
• [Backport 0.5] Disable timeout params for deleteIndex API only for serverless by @opensearch-trigger-bot in #649
• [Backport 0.5] Add langType to FlintStatement model by @opensearch-trigger-bot in #665
• [Back...

Version 0.4.1

What's Changed

• Bump Flint version to 0.4.1 by @dai-chen in #360
• [Backport 0.4] Transition Flint index state to Failed upon refresh job termination by @opensearch-trigger-bot in #364
• [Backport 0.4] Fix index state stuck in refreshing when streaming job exits early by @opensearch-trigger-bot in #374
• [Backport 0.4] Support nested indexed field in Flint skipping index SQL statement by @opensearch-trigger-bot in #388
• [Backport 0.4] Enhance Flint Spark API error reporting with centralized handler by @dai-chen in #401
• [Backport 0.4] handle MetaException with glue AccessDeniedException by @opensearch-trigger-bot in #412
• [Backport 0.4] [Bugfix] Insights on query execution error by @opensearch-trigger-bot in #486

Full Changelog: v0.4.0...v0.4.1

Version 0.4.0

What's Changed

• Improve pre-validation for Flint index refresh options by @dai-chen in #297
• Remove query log from job executor by @seankao-az in #308
• Adding support to run integ tests on iceberg tables by @asuresh8 in #301
• Bump Flint version to 0.4.0 by @seankao-az in #311
• Allow non-existent checkpoint location path in index validation by @dai-chen in #313
• Clean shuffle data by @penghuo in #312
• Introduce aws sigv4a request signer by @noCharger in #303
• Add covering index based query rewriter rule by @dai-chen in #318
• Add maxExecutors configuration for streaming queries by @penghuo in #326
• add batch_bytes configuration for Flint by @penghuo in #329
• Improve flint error handling by @noCharger in #335
• Apply new logging format to record exceptions by @noCharger in #314
• [Backport 0.4] Enhance index monitor to terminate streaming job on consecutive errors by @opensearch-trigger-bot in #347

Full Changelog: v0.3.0...v0.4.0

Version 0.3.0

What's Changed

• Bump Flint version to 0.3.0 by @penghuo in #258
• Add sql grammar support for show flint index statement by @seankao-az in #266
• Implement BloomFilter query rewrite (without pushdown optimization) by @dai-chen in #248
• Refactor flint log format by @noCharger in #263
• Implement BloomFilter query pushdown optimization by @dai-chen in #271
• Add grammar files for alter index by @seankao-az in #279
• Implement adaptive BloomFilter algorithm by @dai-chen in #251
• Move query from entry point to SparkConf by @noCharger in #274
• Fix spark extension path in README. by @asuresh8 in #282
• Implement show flint index statement by @seankao-az in #276
• Add BloomFilter skipping index SQL support by @dai-chen in #283
• Implement analyze skipping index statement by @rupal-bq in #284
• Reduce default inactivity limit to 3min by @penghuo in #287
• Fix shutdown bug due to non-daemon thread in driver by @kaituo in #292
• Rule out logical deleted skipping index in query rewrite by @dai-chen in #289
• Ignore non-Flint index in show and describe index statement by @dai-chen in #296
• Implement Alter Index SQL statement by @seankao-az in #286
• Welcome new maintainer Louis Chu by @penghuo in #299
• Add AWS credentials provider for metadata access by @noCharger in #285
• Unescape query from EMR spark submit parameter by @seankao-az in #306
• [Backport 0.3] Remove query log from job executor by @opensearch-trigger-bot in #310
• [Backport 0.3] Clean shuffle data by @opensearch-trigger-bot in #322
• [Backport 0.3] Introduce aws sigv4a request signer by @opensearch-trigger-bot in #323
• [Backport 0.3] Add maxExecutors configuration for streaming queries by @opensearch-trigger-bot in #328
• [Manual Backport 0.3] Improve flint error handling (#335) by @noCharger in #338

Full Changelog: v0.2.0...v0.3.0

Version 0.2.0

What's Changed

• Ppl spark join command by @YANG-DB in #69
• Fixed the GitHub id column for Yang-DB in Maintainers by @dtaivpp in #205
• Bump Flint version to 0.2.0 by @dai-chen in #183
• Add vacuum index API and SQL support by @dai-chen in #189
• Restrict the maximum size of value set by default limit by @dai-chen in #208
• bug fix, support array datatype in MV by @penghuo in #211
• Percent-encode invalid flint index characters by @seankao-az in #215
• GHA fix for backport and snapshot-publish by @seankao-az in #222
• Quote table name with backticks to escape special characters in spark.read.table() by @seankao-az in #224
• Configure value set max size in SQL statement by @dai-chen in #210
• Change delete index API to logical delete by @dai-chen in #191
• Trigger barkport workflow when a pull request merges by @noCharger in #230
• Changes for adding default dimensions in CWSink. by @vamsi-amazon in #209
• Refactor Flint index refresh mode by @dai-chen in #228
• Fix Lychee Link Checker Error by @noCharger in #236
• Add OpenSearch metrics by @noCharger in #229
• Fix recover index bug when Flint data index is deleted accidentally by @dai-chen in #241
• Support struct field as indexed column by @dai-chen in #213
• Implement BloomFilter skipping index building logic by @dai-chen in #242
• Support dimension sets in config by @noCharger in #238
• Support on-demand incremental refresh by @dai-chen in #234
• Fix Session state bug and improve Query Efficiency in REPL by @kaituo in #245
• Add interactive job metrics by @noCharger in #240
• Add more flint metrics by @noCharger in #255

Version 0.1.0

Preview release