Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove Lambda ESM v1 feature flag #1555

Merged
merged 1 commit into from
Nov 15, 2024
Merged

Remove Lambda ESM v1 feature flag #1555

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/en/references/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,6 @@ Please consult the [migration guide]({{< ref "user-guide/aws/lambda#migrating-to
| `LAMBDA_DOCKER_FLAGS` | `-e KEY=VALUE`, `-v host:container`, `-p host:container`, `--add-host domain:ip` | Additional flags passed to Docker `run`\|`create` commands. Supports environment variables (also with `--env-file`, but the file has to be mounted into the LocalStack container), ports, volume mounts, extra hosts, networks, DNS servers, labels, ulimits, user, platform, and privileged mode. The `--env-file` argument for Docker `run` and Docker Compose have different feature sets. To provide both, we support the `--env-file` for environment files with the docker run syntax, while `--compose-env-file` supports the full docker compose features, like placeholders with `${}`, replacing quotes, etc. |
| `LAMBDA_DOCKER_NETWORK` | `bridge` (Docker default) | [Docker network driver](https://docs.docker.com/network/) for the Lambda and ECS containers. Needs to be set to the network the LocalStack container is connected to. Limitation: `host` mode currently not supported. |
| `LAMBDA_DOWNLOAD_AWS_LAYERS` | `1` (default, pro) | Whether to download public Lambda layers from AWS through a LocalStack proxy when creating or updating functions. |
| `LAMBDA_EVENT_SOURCE_MAPPING` | `v2` | Use the new Lambda Event Source Mapping (ESM) implementation. (preview) |
| `LAMBDA_IGNORE_ARCHITECTURE` | `0` (default) | Whether to ignore the AWS architectures (x86_64 or arm64) configured for the lambda function. Set to `1` to run cross-platform compatible lambda functions natively (i.e., Docker selects architecture). |
| `LAMBDA_K8S_IMAGE_PREFIX` | `amazon/aws-lambda-` (default, pro) | Prefix for images that will be used to execute Lambda functions in Kubernetes. |
| `LAMBDA_K8S_INIT_IMAGE` | | Specify the image for downloading the init binary from LocalStack. The image must include the `curl` and `chmod` commands. This is only relevant for container-based Lambdas on Kubernetes |
Expand Down Expand Up @@ -431,6 +430,7 @@ These configurations have already been removed and **won't have any effect** on

| Variable | Removed in | Example Values | Description |
| - | - | - | - |
| `LAMBDA_EVENT_SOURCE_MAPPING` | 4.0.0 | `v2` (default since [3.8.0](https://blog.localstack.cloud/localstack-release-v-3-8-0/#new-default-lambda-event-source-mapping-implementation)) \| `v1` | Feature flag to switch Lambda Event Source Mapping (ESM) implementations. |
| `S3_DIR` | 4.0.0 | `/path/to/root` | This was only supported for the `legacy_v2` provider. Configure a global parent directory that contains all buckets as sub-directories (`S3_DIR=/path/to/root`) or an individual directory that will get mounted as special bucket names (`S3_DIR=/path/to/root/bucket1:bucket1`). Only available for Localstack Pro.
| `<SERVICE>_BACKEND` | 3.0.0 | `http://localhost:7577` | Custom endpoint URL to use for a specific service, where `<SERVICE>` is the uppercase service name. |
| `<SERVICE>_PORT_EXTERNAL` | 3.0.0 | `4567` | Port number to expose a specific service externally . `SQS_PORT_EXTERNAL`, e.g. , is used when returning queue URLs from the SQS service to the client. |
Expand Down
26 changes: 4 additions & 22 deletions content/en/user-guide/aws/lambda/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,11 +167,6 @@ Since [3.4.0](https://discuss.localstack.cloud/t/localstack-release-v3-4-0/871#n
You can [configure]({{< ref "configuration" >}}) `EVENT_RULE_ENGINE=java` (preview) to use the AWS [event-ruler](https://github.com/aws/event-ruler), which offers better parity.
{{< /callout >}}

{{< callout >}}
Since [3.7.0](https://blog.localstack.cloud/2024-08-29-localstack-release-v-3-7-0/#new-lambda-event-source-mapping-implementation), LocalStack supports a Event Source Mapping (ESM) implementation.
You can [configure]({{< ref "configuration" >}}) `LAMBDA_EVENT_SOURCE_MAPPING=v2` (preview) to use the new ESM implementation.
{{< /callout >}}

[Lambda event source mappings](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventsourcemapping.html) allows you to connect Lambda functions to other AWS services.
The following event sources are supported in LocalStack:

Expand All @@ -180,31 +175,18 @@ The following event sources are supported in LocalStack:
- [Managed Streaming for Apache Kafka (MSK)](https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html)
- [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html)

### New Lambda Event Source Mapping implementation

Since v3.8, LocalStack uses a new default implementation for [Lambda Event Source Mapping](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventsourcemapping.html) (ESM) with improved reliability, performance, and AWS parity.
The new ESM v2 implementation is also compatible with the Java-based event pattern rule engine (`EVENT_RULE_ENGINE=java`) for better event filtering.

The improvements over ESM v1 include:
- Improved reliability through internal retries and separation of concern such that single exceptions or timeouts don't affect other event source mappings.
- Improved performance by enabling concurrent event source mappings rather than having a single thread handling everything.
- Improved AWS parity, related to events filtering, configuring failure destinations, (partial) batch failure handling, and SQS, Kinesis, & DynamoDB Streams events polling.
- Improved Kafka support for Amazon MSK and Self-Managed Kafka.

Create a [GitHub issue](https://github.com/localstack/localstack/issues/new/choose) or reach out to [LocalStack support](https://docs.localstack.cloud/getting-started/help-and-support/) if you experience any challenges.
You can temporarily switch back to the old implementation using the configuration `LAMBDA_EVENT_SOURCE_MAPPING=v1`.
The old implementation won't be available in the next major release.

The limitations compared to AWS include:
- Lambda Success Destinations are not supported.
- Only very basic validations are performed upon creating and updating ESM.
- Streaming Pollers for Kinesis and DynamoDB do not implement features like:
- Only basic validations are performed upon creating and updating ESM.
- Streaming Pollers for Kinesis and DynamoDB do not implement the following batching behavior configurations:
- `BisectBatchOnFunctionError`
- `MaximumBatchingWindowInSeconds`
- `ParallelizationFactor`
- `ScalingConfig`
- `TumblingWindowInSeconds`.

Create a [GitHub issue](https://github.com/localstack/localstack/issues/new/choose) or reach out to [LocalStack support](https://docs.localstack.cloud/getting-started/help-and-support/) if you experience any challenges.

## Lambda Layers (Pro)

[Lambda layers](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) let you include additional code and dependencies in your Lambda functions.
Expand Down