Skip to content

Commit

Permalink
Adding elastic_integration upgrade guidelines.
Browse files Browse the repository at this point in the history
  • Loading branch information
mashhurs committed Jan 28, 2025
1 parent 6943df5 commit a954400
Showing 1 changed file with 38 additions and 5 deletions.
43 changes: 38 additions & 5 deletions docs/static/upgrading.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ and becomes a new node in the monitoring data.
===========================================

If you're upgrading other products in the stack, also read the
{stack-ref}/index.html[Elastic Stack Installation and Upgrade Guide].
{stack-ref}/index.html[Elastic Stack Installation and Upgrade Guide].

See the following topics for information about upgrading Logstash:

Expand All @@ -36,7 +36,8 @@ will happily receive data from earlier versions of Logstash via the default
HTTP communication layer. This provides some flexibility to decide when to
upgrade Logstash relative to an Elasticsearch upgrade. It may or may not be
convenient for you to upgrade them together, and it is not required to be done
at the same time as long as Elasticsearch is upgraded first.
at the same time as long as Elasticsearch is upgraded first. However, there are special plugin cases for example, if your pipeline includes <<plugins-filters-elastic_integration,elastic_integration filter>> plugin.
See <<upgrading-when-elastic_integration-in-pipeline,when `elastic_integration` is in {ls} pipeline>> section for details.

You should upgrade in a timely manner to get the performance improvements that
come with Logstash 8.0, but do so in the way that makes the most sense for your
Expand All @@ -50,14 +51,46 @@ before upgrading.

Although we make great efforts to ensure compatibility, Logstash 8.0 is not completely backwards compatible.
As noted in the Elastic Stack upgrade guide, you should not upgrade Logstash 8.0 before you upgrade Elasticsearch 8.0.
This is both
practical and because some Logstash 8.0 plugins may attempt to use features of Elasticsearch 8.0 that did not exist
This is both practical and because some Logstash 8.0 plugins may attempt to use features of Elasticsearch 8.0 that did not exist
in earlier versions.

For example, if you attempt to send the 8.x template to a cluster before
Elasticsearch 8.0, then all indexing likely fail.
If you use your own custom template with Logstash, then this issue can be ignored.

Another example is when your pipeline utilizes the <<plugins-filters-elastic_integration,`elastic_integration` filter>> plugin.
In such cases, the plugin may encounter issues loading and executing deprecated integrations or features that have been removed in newer versions.
This can lead to disruptions in your pipeline's functionality, especially if your workflow relies on these outdated components.
For a comprehensive understanding of how to handle such scenarios and ensure compatibility, refer to the <<upgrading-when-elastic_integration-in-pipeline,when `elastic_integration` is in {ls} pipeline>> section in this documentation.

[discrete]
[[upgrading-when-elastic_integration-in-pipeline]]
==== When `elastic_integration` is in {ls} pipeline

<<plugins-filters-elastic_integration,elastic_integration filter>> plugin requires a special attention due to its dependencies on various components of the stack such as {es}, {kib} and {ls}.
Any updates, deprecations, or changes in the stack products can directly impact the functionality of the plugin.

* when upgrading {es}

This plugin is compiled with a specific version of {es} and embeds {es} Ingest Node components that match the `major.minor` stack version. Therefore, we recommend using a plugin version that aligns with the `major.minor` version of your stack.

If the versions do not match, the plugin may encounter issues such as failing to load or execute pipelines. For example, if your {es} version is newer than the plugin, the plugin may not support new features introduced in the updated {es} version.
Conversely, if your {es} version is older, the plugin may rely on features that have been deprecated or removed in your {es} version.

* when upgrading {kib}

When you upgrade {kib}, {kib} downloads the latest version of the integrations through {fleet-guide}/fleet-overview.html#package-registry-intro[Elastic Package Registry].
As part of the upgrade process, you will also have the opportunity to review and upgrade your currently installed integrations to their latest versions.
However, we strongly recommend upgrading the <<plugins-filters-elastic_integration,elastic_integration filter>> plugin before upgrading {kib} and {es}.
This is because <<plugins-filters-elastic_integration,elastic_integration filter>> plugin pulls and processes the ingest pipelines associated with the installed integrations.
These pipelines are then executed using the {es} Ingest Node components that the plugin was compiled with.
If {es} or {es} is upgraded first, there is a risk of incompatibility between the plugin's ingest componenets and the newer versions of {es}'s Ingest Node features or {kib}'s integration definitions.

* when upgrading {ls}

This plugin is by default embedded in {ls} core. When you upgrade {ls}, new version of the plugin is installed.
The plugin is backward compatible accross {ls} 8.x versions. However, if you are considering to upgrade {ls} only (not the plugin), there are exceptions cases, such as JDK compatibility which require matching certain {ls} versions.
We recommend visiting <<plugins-filters-elastic_integration-requirements, elastic_integration plugin requirements>> guide considering the {ls} version you are upgrading to.

[[upgrading-using-package-managers]]
=== Upgrading using package managers
Expand Down Expand Up @@ -119,7 +152,7 @@ There you can find info on these topics and more:
If you are installing Logstash with other components in the Elastic Stack, also see the
{stack-ref}/index.html[Elastic Stack installation and upgrade documentation].

NOTE: Upgrading between non-consecutive major versions (6.x to 8.x, for example) is not supported.
NOTE: Upgrading between non-consecutive major versions (6.x to 8.x, for example) is not supported.
We recommend that you upgrade to {prev-major-last}, and then upgrade to 8.0.

[discrete]
Expand Down

0 comments on commit a954400

Please sign in to comment.