-
Notifications
You must be signed in to change notification settings - Fork 277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove OpenSearch build.sh
: moved to OpenSearch repo
#2835
base: main
Are you sure you want to change the base?
remove OpenSearch build.sh
: moved to OpenSearch repo
#2835
Conversation
important: i'm not aware of a way to properly test this manually, thus this remains untested! |
6f52e99
to
e6904be
Compare
`build.sh` has been copied over to all active branches(*) of the OpenSearch repo and can now consequently be deleted here. this means, that from now on, the scripts from the respository will be picked up. for more information on the order of evaluation of the custom build scripts, see the [documentation][]. this gives the advantage to have branch-specific new logic in place there. this has already been used to add the no-jdk targets to the builds on main and 2.x. two tests were based on the assumption that there would be a `build.sh` for OpenSearch in this repository, however these are generic tests and just used OpenSearch as an example. they have been switched to be based on OpenSearch-Dashboards instead which - for the time being - still has its own `build.sh` here. (*): the following branches have been considered as active: - main - 2.x - 2.3 - 1.x - 1.3 this is part of opensearch-project#99 [documentation]: https://github.com/opensearch-project/opensearch-build/tree/main/src/build_workflow#custom-build-scripts Signed-off-by: Ralph Ursprung <[email protected]>
e6904be
to
e6288a3
Compare
Codecov Report
@@ Coverage Diff @@
## main #2835 +/- ##
=======================================
Coverage 94.15% 94.15%
=======================================
Files 157 157
Lines 4242 4242
=======================================
Hits 3994 3994
Misses 248 248 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@peterzhuamazon any issues?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @rursprung
I wonder if we can block this until 2.4.0 is released to public.
We are at the edge of the release in about a week or so.
I want to make sure this has minimum impact.
Let me know what you think.
Thanks.
i was sort-of hoping to get this in for 2.4.0 😄. |
@peterzhuamazon it shouldn't break anything should it? we just moved a bash file |
We are building release candidate of 2.4.0 tomorrow. |
Hi @rursprung we have seen there is changes to add no-jdk build in build.sh.
We need some additional time to review and plan the no-jdk addition. Thanks again for the contribution and let us discuss more on this. Thanks. |
yes, that was added in opensearch-project/OpenSearch#4902 and ported to 2.x in opensearch-project/OpenSearch#4993 |
Based on the offline conversation I will test the build on local to ensure that no-jdk is not affecting the normal jdk build. Thanks. |
I have confirmed it will not impact the existing code base. But, instead of building both jdk and no-jdk in one take, we should add a --no-jdk param in build repo, so the target will be properly set as
instead of
This will also resolve the follow up assemble issue with no-jdk bundle. There are still a lot of steps to go, I would recommend hold this PR for a bit longer, and start changing the build.sh on OpenSearch Core repo. Ex:
Thanks. |
I also want to mention this seems significantly more complex than initially anticipated. Thanks. |
I am confused. We just need an old build.sh in those old opensearch branches. Why is this any more complicated? |
i guess the problem was that we (well, I) mixed two things:
it might make sense to just revert the latter (no-jdk) and get this one here merged to at least close the move job. and then we can look into the no-jdk discussion again? |
The issue is more that instead of adding a no-jdk, the original implementation is to run both jdk + no-jdk build in parallel, which is not supposed to be as the current code base does not support the actions after that. |
I am with you @rursprung. Let's focus on this PR! Can we merge just deleting the OpenSearch build.sh from this repo? |
…ch-project#4902)" This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]>
…ch-project#4902)" This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]>
…ch-project#4902)" This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]>
i've created the reversal PRs and they're ready for review:
but i just realised that one more step will be needed before this here can be done: |
…5467) This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]> Signed-off-by: Ralph Ursprung <[email protected]>
…5465) This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]> Signed-off-by: Ralph Ursprung <[email protected]>
…5466) This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]> Signed-off-by: Ralph Ursprung <[email protected]>
…5465) This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]> Signed-off-by: Ralph Ursprung <[email protected]>
…ature/identity (#5581) * Fix flaky ShardIndexingPressureConcurrentExecutionTests (#5439) Add conditional check on assertNull to fix flaky tests. Signed-off-by: Rishikesh1159 <[email protected]> * Fix bwc for cluster manager throttling settings (#5305) Signed-off-by: Dhwanil Patel <[email protected]> * Update ingest-attachment plugin dependencies: Apache Tika 3.6.0, Apache Mime4j 0.8.8, Apache Poi 5.2.3, Apache PdfBox 2.0.27 (#5448) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Enhance CheckpointState to support no-op replication (#5282) * CheckpointState enhanced to support no-op replication Signed-off-by: Ashish Singh <[email protected]> Co-authored-by: Bukhtawar Khan<[email protected]> * [BUG] org.opensearch.repositories.s3.RepositoryS3ClientYamlTestSuiteIT/test {yaml=repository_s3/20_repository_permanent_credentials/Snapshot and Restore with repository-s3 using permanent credentials} flaky: randomizing basePath (#5482) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * [Bug] fix case sensitivity for wildcard queries (#5462) Fixes the wildcard query to not normalize the pattern when case_insensitive is set by the user. This is achieved by creating a new normalizedWildcardQuery method so that query_string queries (which do not support case sensitivity) can still normalize the pattern when the default analyzer is used; maintaining existing behavior. Signed-off-by: Nicholas Walter Knize <[email protected]> * Support OpenSSL Provider with default Netty allocator (#5460) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Revert "build no-jdk distributions as part of release build (#4902)" (#5465) This reverts commit 8c9ca4e. It seems that this wasn't entirely the correct way and is currently blocking us from removing the `build.sh` from the `opensearch-build` repository (i.e. this `build.sh` here is not yet being used). See the discussion in opensearch-project/opensearch-build#2835 for further details. Signed-off-by: Ralph Ursprung <[email protected]> Signed-off-by: Ralph Ursprung <[email protected]> * Add max_shard_size parameter for Shrink API (fix supported version after backport) (#5503) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Sync CODEOWNERS with MAINTAINERS. (#5501) Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> * Added jackson dependency to server (#5366) * Added jackson dependency to server Signed-off-by: Ryan Bogan <[email protected]> * Updated CHANGELOG Signed-off-by: Ryan Bogan <[email protected]> * Update build.gradle files Signed-off-by: Ryan Bogan <[email protected]> * Add RuntimePermission to fix errors Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Ryan Bogan <[email protected]> * Fix flaky test BulkIntegrationIT.testDeleteIndexWhileIndexing (#5491) Signed-off-by: Poojita Raj <[email protected]> Signed-off-by: Poojita Raj <[email protected]> * Add release notes for 2.4.1 (#5488) Signed-off-by: Xue Zhou <[email protected]> Signed-off-by: Xue Zhou <[email protected]> * Properly skip OnDemandBlockSnapshotIndexInputTests.testVariousBlockSize on Windows. (#5511) PR #5397 skipped this test in @before block but still frequently throws a TestCouldNotBeSkippedException. This is caused by the after block still executing and throwing an exception while cleaning the directory created at the path in @before. Moving the assumption to the individual test prevents this exception by ensuring the path exists. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]> * Merge first batch of feature/extensions into main (#5347) * Merge first batch of feature/extensions into main Signed-off-by: Ryan Bogan <[email protected]> * Fixed CHANGELOG Signed-off-by: Ryan Bogan <[email protected]> * Fixed newline errors Signed-off-by: Ryan Bogan <[email protected]> * Renaming and CHANGELOG fixes Signed-off-by: Ryan Bogan <[email protected]> * Refactor extension loading into private method Signed-off-by: Ryan Bogan <[email protected]> * Removed skipValidation and added connectToExtensionNode method Signed-off-by: Ryan Bogan <[email protected]> * Remove unnecessary feature flag calls Signed-off-by: Ryan Bogan <[email protected]> * Renaming and exception handling Signed-off-by: Ryan Bogan <[email protected]> * Change latches to CompletableFuture Signed-off-by: Ryan Bogan <[email protected]> * Removed unnecessary validateSettingKey call Signed-off-by: Ryan Bogan <[email protected]> * Fix azure-core dependency Signed-off-by: Ryan Bogan <[email protected]> * Update SHAs Signed-off-by: Ryan Bogan <[email protected]> * Remove unintended dependency changes Signed-off-by: Ryan Bogan <[email protected]> * Removed dynamic settings regitration, removed info() method, and added NoopExtensionsManager Signed-off-by: Ryan Bogan <[email protected]> * Add javadoc Signed-off-by: Ryan Bogan <[email protected]> * Fixed spotless failure Signed-off-by: Ryan Bogan <[email protected]> * Removed NoopExtensionsManager Signed-off-by: Ryan Bogan <[email protected]> * Added functioning NoopExtensionsManager Signed-off-by: Ryan Bogan <[email protected]> * Added missing javadoc Signed-off-by: Ryan Bogan <[email protected]> * Remove forbiddenAPI Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Change logger.info to logger.error in handleException Signed-off-by: Ryan Bogan <[email protected]> * Fix ExtensionsManagerTests Signed-off-by: Ryan Bogan <[email protected]> * Removing unrelated change Signed-off-by: Ryan Bogan <[email protected]> * Update SHAs Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Ryan Bogan <[email protected]> * Bump commons-compress from 1.21 to 1.22 (#5520) Bumps commons-compress from 1.21 to 1.22. --- updated-dependencies: - dependency-name: org.apache.commons:commons-compress dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [Segment Replication] Trigger a round of replication for replica shards during peer recovery when segment replication is enabled (#5332) * Fix new added replica shards falling behind primary. Signed-off-by: Rishikesh1159 <[email protected]> * Trigger a round of replication during peer recovery when segment replication is enabled. Signed-off-by: Rishikesh1159 <[email protected]> * Remove unnecessary start replication overloaded method. Signed-off-by: Rishikesh1159 <[email protected]> * Add test for failure case and refactor some code. Signed-off-by: Rishikesh1159 <[email protected]> * Apply spotless check. Signed-off-by: Rishikesh1159 <[email protected]> * Addressing comments on the PR. Signed-off-by: Rishikesh1159 <[email protected]> * Remove unnecessary condition check. Signed-off-by: Rishikesh1159 <[email protected]> * Apply spotless check. Signed-off-by: Rishikesh1159 <[email protected]> * Add step listeners to resolve forcing round of segment replication. Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> * Adding support to register settings dynamically (#5495) * Adding support to register settings dynamically Signed-off-by: Ryan Bogan <[email protected]> * Update CHANGELOG Signed-off-by: Ryan Bogan <[email protected]> * Removed unnecessary registerSetting methods Signed-off-by: Ryan Bogan <[email protected]> * Change setting registration order Signed-off-by: Ryan Bogan <[email protected]> * Add unregisterSettings method Signed-off-by: Ryan Bogan <[email protected]> * Remove unnecessary feature flag Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Ryan Bogan <[email protected]> * Updated 1.3.7 release notes date (#5536) Signed-off-by: owaiskazi19 <[email protected]> Signed-off-by: owaiskazi19 <[email protected]> * Pre conditions check before updating weighted routing metadata (#4955) * Pre conditions check to allow weight updates for non decommissioned attribute Signed-off-by: Rishab Nahata <[email protected]> * Atomically update cluster state with decommission status and corresponding action (#5093) * Atomically update the cluster state with decommission status and its corresponding action in the same execute call Signed-off-by: Rishab Nahata <[email protected]> * Update Netty to 4.1.86.Final (#5529) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Update release date in 2.4.1 release notes (#5549) Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]> * Update 2.4.1 release notes (#5552) Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Refactor fuzziness interface on query builders (#5433) * Refactor Object to Fuzziness type for all query builders Signed-off-by: noCharger <[email protected]> * Revise on bwc Signed-off-by: noCharger <[email protected]> * Update change log Signed-off-by: noCharger <[email protected]> Signed-off-by: noCharger <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> * Upgrade lucene version (#5570) * Added bwc version 2.4.2 Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> * Added 2.4.2. Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> * Update Lucene snapshot to 9.5.0-snapshot-d5cef1c Signed-off-by: Suraj Singh <[email protected]> * Update changelog entry Signed-off-by: Suraj Singh <[email protected]> * Add 2.4.2 bwc version Signed-off-by: Suraj Singh <[email protected]> * Internal changes post lucene upgrade Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> * Add CI bundle pattern to distribution download (#5348) * Add CI bundle pattern for ivy repo Signed-off-by: Zelin Hao <[email protected]> * Gradle update Signed-off-by: Zelin Hao <[email protected]> * Extract path Signed-off-by: Zelin Hao <[email protected]> * Change with customDistributionDownloadType Signed-off-by: Zelin Hao <[email protected]> * Add default for exception handle Signed-off-by: Zelin Hao <[email protected]> * Add documentations Signed-off-by: Zelin Hao <[email protected]> Signed-off-by: Zelin Hao <[email protected]> * Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs (#5519) * Bump protobuf-java from 3.21.9 to 3.21.11 in /plugins/repository-hdfs Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.9 to 3.21.11. - [Release notes](https://github.com/protocolbuffers/protobuf/releases) - [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py) - [Commits](protocolbuffers/protobuf@v3.21.9...v3.21.11) --- updated-dependencies: - dependency-name: com.google.protobuf:protobuf-java dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Updated changelog Signed-off-by: Owais Kazi <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Owais Kazi <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Dhwanil Patel <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Nicholas Walter Knize <[email protected]> Signed-off-by: Ralph Ursprung <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Poojita Raj <[email protected]> Signed-off-by: Xue Zhou <[email protected]> Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: owaiskazi19 <[email protected]> Signed-off-by: Rishab Nahata <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: noCharger <[email protected]> Signed-off-by: Zelin Hao <[email protected]> Signed-off-by: Owais Kazi <[email protected]> Co-authored-by: Rishikesh Pasham <[email protected]> Co-authored-by: Dhwanil Patel <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: Ashish <[email protected]> Co-authored-by: Nick Knize <[email protected]> Co-authored-by: Ralph Ursprung <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Ryan Bogan <[email protected]> Co-authored-by: Poojita Raj <[email protected]> Co-authored-by: Xue Zhou <[email protected]> Co-authored-by: Marc Handalian <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Rishab Nahata <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Zelin Hao <[email protected]> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Hi @peterzhuamazon Should we move ahead with this PR? |
@peterzhuamazon @rursprung What's the next step on this PR? Please provide your comments. |
between the moment where i added the |
@peterzhuamazon What would you like to do with this (finish/close)? |
build.sh
has been copied over to all active branches(*) of the OpenSearch repo and can now consequently be deleted here. this means, that from now on, the scripts from the respository will be picked up.for more information on the order of evaluation of the custom build scripts, see the documentation.
this gives the advantage to have branch-specific new logic in place there. this has already been used to add the no-jdk targets to the builds on main and 2.x.
two tests were based on the assumption that there would be a
build.sh
for OpenSearch in this repository, however these are generic tests and just used OpenSearch as an example. they have been switched to be based on OpenSearch-Dashboards instead which - for the time being - still has its ownbuild.sh
here.(*): the following branches have been considered as active:
this is part of #99
Signed-off-by: Ralph Ursprung [email protected]
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.