Added new Setting property `UnmodifiableOnRestore` to prevent updating settings on restore snapshot #16957

anntians · 2025-01-06T18:45:34Z

Description

This PR adds a new Setting property UnmodifiableOnRestore to prevent customers from updating certain settings on restore snapshot. The motivation behind this change is to prevent customers from updating index.knn setting when restoring a snapshot. The PR is related to another PR (opensearch-project/k-NN#2348) which fixes a bug by making index.knn setting immutable after index creation. However, that PR doesn't cover the case for snapshots, so this PR adds Setting property UnmodifiableOnRestore so that index.knn will be immutable when restoring an index from a snapshot.

Related Issues

Resolves Issue 2334 in KNN repository: opensearch-project/k-NN#2334
#17019

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…napshot Signed-off-by: AnnTian Shao <[email protected]>

Signed-off-by: Tommy Shao <[email protected]>

github-actions · 2025-01-06T19:52:12Z

✅ Gradle check result for 4a659c0: SUCCESS

codecov · 2025-01-06T21:28:20Z

Codecov Report

Attention: Patch coverage is 76.47059% with 4 lines in your changes missing coverage. Please review.

Project coverage is 72.23%. Comparing base (13ab4ec) to head (9b3e7ce).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
.../java/org/opensearch/snapshots/RestoreService.java	42.85%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16957      +/-   ##
============================================
- Coverage     72.32%   72.23%   -0.10%     
+ Complexity    65477    65393      -84     
============================================
  Files          5309     5309              
  Lines        304306   304319      +13     
  Branches      44124    44128       +4     
============================================
- Hits         220098   219813     -285     
- Misses        66159    66502     +343     
+ Partials      18049    18004      -45

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Tommy Shao <[email protected]>

github-actions · 2025-01-07T18:24:17Z

❌ Gradle check result for f33c9bc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-01-23T03:26:53Z

❌ Gradle check result for f487f95: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-01-23T04:49:01Z

❌ Gradle check result for c1c3593: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…ady registered, will raise separate PR for settings not yet registered. Add validation check in Setting.java. Add UnmodifiableOnRestore settings cannot be removed on restore Signed-off-by: AnnTian Shao <[email protected]>

github-actions · 2025-01-23T08:37:58Z

✅ Gradle check result for dc281ce: SUCCESS

jed326

Had a few comments but none are blocking, LGTM. @dblock would you like to take another look too?

server/src/main/java/org/opensearch/common/settings/Setting.java

server/src/internalClusterTest/java/org/opensearch/remotestore/RemoteRestoreSnapshotIT.java

server/src/internalClusterTest/java/org/opensearch/remotestore/RestoreShallowSnapshotV2IT.java

server/src/internalClusterTest/java/org/opensearch/remotestore/RemoteRestoreSnapshotIT.java

server/src/main/java/org/opensearch/common/settings/Setting.java

server/src/internalClusterTest/java/org/opensearch/remotestore/RemoteRestoreSnapshotIT.java

server/src/internalClusterTest/java/org/opensearch/remotestore/RestoreShallowSnapshotV2IT.java

Requested changes adapted.

github-actions · 2025-01-23T22:48:04Z

❌ Gradle check result for 3227d08: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: AnnTian Shao <[email protected]>

github-actions · 2025-01-24T00:22:11Z

❌ Gradle check result for b55cbed: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: AnnTian Shao <[email protected]>

sohami

Thanks for the changes, LGTM

github-actions · 2025-01-24T02:30:01Z

✅ Gradle check result for 5a836e3: SUCCESS

Signed-off-by: Tommy Shao <[email protected]>

github-actions · 2025-01-24T17:39:31Z

✅ Gradle check result for 9b3e7ce: SUCCESS

opensearch-trigger-bot · 2025-01-24T19:04:45Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-16957-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 822f80cf2cdc8398d6861827bca2d53f91329f8d
# Push it to GitHub
git push --set-upstream origin backport/backport-16957-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-16957-to-2.x.

…g settings on restore snapshot (#16957) (#17116) * Add index.knn setting to list of unmodifiable settings when restore snapshot Signed-off-by: AnnTian Shao <[email protected]> * Add index.knn setting to list of unmodifiable settings when restore snapshot Signed-off-by: AnnTian Shao <[email protected]> * Add new Setting property UnmodifiableOnRestore to mark setting as immutable on restore snapshot Signed-off-by: AnnTian Shao <[email protected]> * Add tests for new Setting property UnmodifiableOnRestore Signed-off-by: AnnTian Shao <[email protected]> * fixes and added more tests for new setting property UnmodifiableOnRestore Signed-off-by: AnnTian Shao <[email protected]> * fix test failures Signed-off-by: AnnTian Shao <[email protected]> * Revert "fix test failures" This reverts commit 252100c. Signed-off-by: AnnTian Shao <[email protected]> * fixes by adding back USER_UNMODIFIABLE_SETTINGS for settings without Setting implementation Signed-off-by: AnnTian Shao <[email protected]> * testing CI config for registering plugin settings Signed-off-by: AnnTian Shao <[email protected]> * Revert "testing CI config for registering plugin settings" This reverts commit 9ebab5a. Signed-off-by: AnnTian Shao <[email protected]> * Add UnmodifiableOnRestore only to unmodifiable settings that are already registered, will raise separate PR for settings not yet registered. Add validation check in Setting.java. Add UnmodifiableOnRestore settings cannot be removed on restore Signed-off-by: AnnTian Shao <[email protected]> * fixes and move tests to RestoreSnapshotIT Signed-off-by: AnnTian Shao <[email protected]> * Add back testInvalidRestoreRequestScenarios test method Signed-off-by: AnnTian Shao <[email protected]> --------- Signed-off-by: AnnTian Shao <[email protected]> Signed-off-by: Tommy Shao <[email protected]> Co-authored-by: AnnTian Shao <[email protected]> (cherry picked from commit 822f80c) Signed-off-by: Tommy Shao <[email protected]>

AnnTian Shao added 3 commits January 6, 2025 10:02

Add index.knn setting to list of unmodifiable settings when restore s…

457898a

…napshot Signed-off-by: AnnTian Shao <[email protected]>

Add index.knn setting to list of unmodifiable settings when restore s…

5f70915

…napshot Signed-off-by: AnnTian Shao <[email protected]>

Merge branch 'main' of github.com:anntians/OpenSearch

736696c

Merge branch 'main' into main

4a659c0

Signed-off-by: Tommy Shao <[email protected]>

Merge branch 'main' into main

f33c9bc

Signed-off-by: Tommy Shao <[email protected]>

opensearch-ci-bot mentioned this pull request Jan 22, 2025

[AUTOCUT] Gradle Check Flaky Test Report for SpecificClusterManagerNodesIT #15944

Open

anntians force-pushed the main branch from f487f95 to c1c3593 Compare January 23, 2025 03:52

anntians force-pushed the main branch from c1c3593 to dc281ce Compare January 23, 2025 07:42

anntians requested a review from dblock January 23, 2025 17:16

jed326 approved these changes Jan 23, 2025

View reviewed changes

navneet1v added the backport 2.x Backport to 2.x branch label Jan 23, 2025

sohami reviewed Jan 23, 2025

View reviewed changes

fixes and move tests to RestoreSnapshotIT

b55cbed

Signed-off-by: AnnTian Shao <[email protected]>

anntians force-pushed the main branch from 3227d08 to b55cbed Compare January 23, 2025 23:25

opensearch-ci-bot mentioned this pull request Jan 24, 2025

[AUTOCUT] Gradle Check Flaky Test Report for IndexingIT #14302

Open

Add back testInvalidRestoreRequestScenarios test method

accd092

Signed-off-by: AnnTian Shao <[email protected]>

sohami approved these changes Jan 24, 2025

View reviewed changes

Merge branch 'main' into main

9b3e7ce

Signed-off-by: Tommy Shao <[email protected]>

anntians force-pushed the main branch from 5a836e3 to 9b3e7ce Compare January 24, 2025 16:43

sohami approved these changes Jan 24, 2025

View reviewed changes

sohami merged commit 822f80c into opensearch-project:main Jan 24, 2025
30 checks passed

opensearch-trigger-bot bot added the backport-failed label Jan 24, 2025

This was referenced Jan 25, 2025

Register UnmodifiableOnRestore settings #17121

Open

Add UnmodifiableOnRestore property to index.knn setting opensearch-project/k-NN#2445

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added new Setting property `UnmodifiableOnRestore` to prevent updating settings on restore snapshot #16957

Added new Setting property `UnmodifiableOnRestore` to prevent updating settings on restore snapshot #16957

anntians commented Jan 6, 2025 •

edited

Loading

github-actions bot commented Jan 6, 2025

codecov bot commented Jan 6, 2025 •

edited

Loading

github-actions bot commented Jan 7, 2025

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 23, 2025

jed326 left a comment

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 24, 2025

sohami left a comment

github-actions bot commented Jan 24, 2025

github-actions bot commented Jan 24, 2025

opensearch-trigger-bot bot commented Jan 24, 2025

Added new Setting property UnmodifiableOnRestore to prevent updating settings on restore snapshot #16957

Added new Setting property UnmodifiableOnRestore to prevent updating settings on restore snapshot #16957

Conversation

anntians commented Jan 6, 2025 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Jan 6, 2025

codecov bot commented Jan 6, 2025 • edited Loading

Codecov Report

github-actions bot commented Jan 7, 2025

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 23, 2025

jed326 left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 23, 2025

github-actions bot commented Jan 24, 2025

sohami left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 24, 2025

github-actions bot commented Jan 24, 2025

opensearch-trigger-bot bot commented Jan 24, 2025

Added new Setting property `UnmodifiableOnRestore` to prevent updating settings on restore snapshot #16957

Added new Setting property `UnmodifiableOnRestore` to prevent updating settings on restore snapshot #16957

anntians commented Jan 6, 2025 •

edited

Loading

codecov bot commented Jan 6, 2025 •

edited

Loading