This document explains the release strategy for artifacts in this organization, including large distributions, such as OpenSearch with OpenSearch Dashboards, and smaller distributions, such as clients.
Projects create a new branch when they need to start working on 2 separate versions of the product, with the main branch being the furthermost release.
Most repos in this organization, including OpenSearch and OpenSearch-Dashboards, typically track 3 releases in parallel. For example, given the last major release of 2.0, OpenSearch and all its components in this organization maintains the following active branches.
- main: The next major release, currently 3.0. This is the branch where all merges take place, and code moves fast.
-
- 2.x: The next minor release. Once a change is merged into
main, decide whether to backport it to2.x.
- 2.x: The next minor release. Once a change is merged into
-
- 2.17: The last minor release and the next patch release. Once a change is merged into
2.x, it may be backported to, for example,2.17for the next 2.17 release candidate, or as a patch for a2.12.1.
- 2.17: The last minor release and the next patch release. Once a change is merged into
- 1.3: The next patch release. Once a change is merged into
main, decide whether to backport it to1.3.
Label PRs with the next major version label (e.g. 3.0.0) and merge changes into main. Label PRs that you believe need to be backported as 2.x, 1.x and 1.3. Backport PRs by checking out the versioned branch, cherry-pick changes and open a PR against each target backport branch.
OpenSearch and OpenSearch Dashboards plugins are bundled and shipped together along with the OpenSearch distribution for every release. Plugin branching follows OpenSearch branching described above that allows working on 3 releases at the same time.
Frequently released artifacts, such as most clients, follow a simpler branching model with the next release always living on main and no patch branches.
Do not creating branches in the upstream repo, use your fork, for the exception of long lasting feature branches that require active collaboration from multiple developers. Name feature branches feature/<thing>. Once the work is merged to main, please make sure to delete the feature branch.
Backwards-incompatible changes always result in a new major version and will never be backported. Small improvements and features will be backported to a new minor version (e.g. 2.9.0). High severity security or critical bug fixes will be backported to a new patch version (e.g. 2.9.1). Repos in this organization typically use a backport workflow where you can label a PR, e.g. backport 2.x, and the workflow will attempt an automatic backport and open a new PR. If the backport fails, it is the contributor's responsibility to do a manual backport following the instructions in the failed backport error message.
All distributions in this organization follow SemVer. A user-facing breaking change can only be made in a major release. Any regression that breaks SemVer is considered a high severity bug.
The build number of the engine is 3-digit major.minor.patch (e.g. 2.9.0), while plugins use 4 digits (2.9.0.45). See OpenSearch#1093 for a proposal to remove this difference.
The version number of all releasable artifacts is 3-digit major.minor.patch (e.g. 2.9.0).
Versions are incremented as soon as development starts on a given version to avoid confusion. In the examples above versions are as follows.
- OpenSearch:
main= 3.0.0,2.x= 2.10.01.x= 1.4.0, and1.2= 1.2.5 - job-scheduler:
main= 3.0.0.0,2.x= 2.10.0.0 ,1.x= 1.4.0.0 and1.2= 1.2.5.0
Major releases contain breaking changes, new features, and bug fixes. All artifacts released in this organization only do major releases when there are significant breaking changes ready for release that warrant a major upgrade. To avoid major upgrades we strive to build features in a backwards compatible way as much as possible, therefore it's common for a distribution to go multiple years without a major release.
For example, the OpenSearch bundle that contains OpenSearch and OpenSearch Dashboards releases new major versions after Lucene major version releases. This is announced in the forums at least 4 weeks prior to the release date. Once we become aware of the need for a major version, we start a major version release window which is similar to a minor release, except for two things: 1) participation is mandatory for all components, and 2) the release window is at least 4 weeks long to accommodate the testing required.
Minor releases for large distributions such as OpenSearch are scheduled on regular intervals, contain new features and bug fixes, and must not contain any breaking changes.
Patch releases are reserved for addressing high-severity CVEs, critical bug fixes (e.g., issues causing users to roll back after an upgrade), or significant regressions in non-experimental features (e.g., feature rendered unusable). These releases must not introduce any new features.
We follow OpenSSF's best practices for patching publicly known vulnerabilities and we make sure that there are no unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days in our actively maintained versions.
Whether a fix qualifies as a critical bug should be proposed by the PR contributor or issue creator, and triaged by the maintainers of the repo with input from the community. The severity and impact of a bug in defining criticality of an issue can vary. For example, back-end APIs are fairly straightforward to triage, while a UX bug may prevent users from using a feature and may require some domain expertise to assess.
See Releasing OpenSearch for the standard operating procedures of making an OpenSearch release.
OpenSearch uses a "release-train" model. For minor version, that train leaves approximately every six weeks we release a new minor version which includes all the new features and fixes that are ready to go. Having a set release schedule makes sure OpenSearch is releasing in a predictable way and prevents a backlog of unreleased changes.
At the beginning of every year, the project will publish on opensearch.org the "release windows start" dates for the year for all the large distributions, including OpenSearch. These dates will be spaced out ~6 weeks.
On release window start date:
- We generate the first release candidate with all plug-ins/components that have met the entrance criteria. If a plug-in/component hasn't met all of the criteria, we'll version bump the last released version and release that. Once the release window opens and the first candidate is generated, no additional features can be added, and we will not delay the start of a release window for any plug-in/component.
- During the release window we will conduct final quality testing, documentation updates and performance testing. Bug fixes can be added in during this time, but no new features will be included.
- We will generate a new release candidate every day and post on the release issue about the status of the exit criteria. When all the exit criteria have been met, we'll announce that the candidate is ready and release it 2 days later (unless that falls on Friday, in which case we'll release on Monday).
- If we cannot clear the exit criteria within 2 weeks from the start of the window (1 week for patch releases), we will cancel that release window and try again in the next window.
Please note: this process is for regularly scheduled minor and patch releases. For "hot" patches (patches required for security or other urgent issues) we will build, test and release as quickly as possible.
- Documentation draft PRs are up and in tech review for all component changes.
- Sanity testing is done for all components.
- Code coverage has not decreased (all new code has tests).
- Release notes are ready and available for all components.
- Roadmap is up-to-date (information is available to create release highlights).
- Release ticket is cut, and there's a forum post announcing the start of the window.
- Any necessary security reviews are complete.
- Performance tests are run, results are posted to the release ticket and there no unexpected regressions
- Documentation has been fully reviewed and signed off by the documentation community.
- All integration tests are passing
- Release blog is ready
Look for a RELEASING.md file in each repo, e.g. opensearch-java/RELEASING.md. This is typically a 1-click release process in which a maintainer pushes a tag to the repo, followed by automation performing the actual release.
Create tags after a release that match the version number, major.minor.patch, without a v prefix.
For a discussion on whether to add a prefixing v to release tags, see #35.
Repositories create consistent release labels, such as v2.9.0, v1.0.0, v1.1.0 and v2.0.0, as well as patch and backport. Use release labels to target an issue or a PR for a given release. See MAINTAINERS for more information on triaging issues.
For large distributions such as OpenSearch, we strive for consistent and predictable release schedule as multiple organizations and users depend on the software for their own projects and businesses. However, sometimes a release date needs to move to accommodate engineering delays in critical components that affect the key properties of the software such as performance, reliability, availability, or security. In order to move a release date, we will ensure:
- There is a publicly documented justification for moving the release date.
- The justification is circulated 2 weeks or more prior to the original release date.
- The organization coordinating the release is in favor of moving the release date.
- The leadership committee has voted to move the release date with a simple majority.
Smaller distributions, such as clients, release as often as possible. If you see changes that have not been released and you need a release, please open an issue titled "Release v. Next" in the corresponding repo, e.g. opensearch-java#1093.
If you discover a potential security issue in this project we ask that you notify the OpenSearch Security Team by email at [email protected]. Please do not create a public GitHub issue. See SECURITY.md for more information on the security response process.
The OpenSearch Project currently performs security reviews before releasing signed artifacts. These are typically conducted for any of the following:
- Releases from a new GitHub repository, such as a new plugin or extension, client, or tool.
- Major new features added to an existing application, including significant UX or API changes.
- Changes to authentication, authorization, cryptography, or other security-impacting functions.
- New software or infrastructure deployed to support the project, such as CI/CD.
- New 3rd-party providers or vendors being used by the project.
The review process consists of building a threat model for the proposed change and optionally engaging a specialist to perform additional testing, such as a penetration testing. This process is done in parallel and in private within the project, during development, and usually takes 4-10 weeks. A repository maintainer will assess the scope of the new changes, initiate and manage a security review, provide public updates, and, if needed, communicate privately by email with the contributors. Please add a note in your pull request if you believe a security review is warranted.
Please see opensearch-project/.github#81 for a discussion on improving this and other security-related processes.