feat(alarm): add minSampleCountToEvaluateDatapoint #453
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
miloszwatroba
force-pushed
the
fix/min-samples
branch
from
58a1cc2 to
917eee0
Compare
miloszwatroba
changed the title
miloszwatroba
commented
Nov 10, 2023
| // create primary alarm | ||
|
|
||
| const primaryAlarm = adjustedMetric.createAlarm( | ||
| this.alarmScope, | ||
| const primaryAlarm = alarmMetric.createAlarm(this.alarmScope, alarmName, { |
There was a problem hiding this comment.
adjustedMetric changed to alarmMetric; the rest was automatically reformatted by yarn build without any changes
miloszwatroba
force-pushed
the
fix/min-samples
branch
from
917eee0 to
ccb010b
Compare
echeung-amzn
approved these changes
Nov 10, 2023
Comment on lines
+576
to
+578
| "minSampleCountToEvaluateDatapoint is not supported for MathExpressions. " + | ||
| "If you already use MathExpression, you can extend your expression to evaluate " + | ||
| "the sample count using IF statement, e.g. IF(sampleCount > X, mathExpression)." |
There was a problem hiding this comment.
I wonder if it's worth abstracting that at some point anyway just for convenience?
echeung-amzn
pushed a commit
that referenced
this pull request
Nov 16, 2023
#458) Fixes #452 Follow up to #453: * (feat) Exposing the new `minSampleCountToEvaluateDatapoint` through CustomMonitoring * (fix) Fixing the `minSampleCountToEvaluateDatapoint` MathExpression's period as it defaults to 5 minutes. This didn't come up during testing as I tested it using 5 minute period. Apparently, if we don't set the period on MathExpression explicitly, it overrides all child metrics to 5 minute, [reference](https://github.com/aws/aws-cdk/blob/db21fefc2dc76eb4ff306fa41652ab6a6cc95e42/packages/aws-cdk-lib/aws-cloudwatch/lib/metric.ts#L606). To avoid similar situations in the future, extended the unit test to cover custom periods. --- _By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license_
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #452
Currently, when using
minMetricSamplesToAlarmthe number of samples is evaluated for a different period than the main alarm. This makes monitoring sensitive to false positives as not every breaching datapoint must have sufficient number of samples (see #452 for more details).Moreover, the current approach for adjusting alarms to respect
minMetricSamplesToAlarmis to create 2 extra alarms - one forNoSamplesand one for a top-level composite. Each of these monitors incurs extra costs ($0.10 forNoSamplesmonitor and $0.50 for the Composite, see https://aws.amazon.com/cloudwatch/pricing/ for reference). This means that usingminMetricSamplesToAlarmincreases the cost from $0.10 per alarm to $0.70 per alarm ($0.60 of overhead!).It's possible to use Math Expression instead. Instead of adding separate alarm for
NoSamples, we can model it a Sample Count metric, and instead of the Composite, we can use the MathExpression that conditionally emits the data based on the number of samples. The charge for Math Expression-based alarms is per metric in the Math Expression, so that comes down to $0.20 per alarm. That's a 70% cost improvement. Additionally, it reduces the overall number of alarms, effectively making it easier to fit your alarming in the CloudWatch quota and decluttering the UI.To avoid breaking any customers that rely on
minMetricSamplesToAlarmgenerating alarms (e.g. #403), deprecating it and addingminSampleCountToEvaluateDatapointwith updated behaviour next to it.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license