feat(validator): add support to validate essential metrics produced by Kepler

[vprashar2929]

This commit introduces functionality to validate essential metrics produced by Kepler
The following comparisons are included:

• Node Exporter Comparison

  • Validates node_rapl_<package|core|dram> metrics against kepler_node_<package|core|dram>{dev}

• Kepler Process Comparison

  • Compares kepler_process_<package|core|dram|platform|other|uncore>{latest} metrics to
    kepler_process_<package|core|dram|platform|other|uncore>{dev}

• Kepler Node Comparison

  • Validates kepler_node_<package|core|dram|platform|other|uncore>{latest} against
    kepler_node_<package|core|dram|platform|other|uncore>{dev}

Additionally, a stressor script has been added to include system load, allowing for real-time validation of Kepler under stress conditions.

[github-actions]

🤖 SeineSailor

Here is a concise summary of the pull request changes:

Summary: This pull request introduces a new feature to validate Kepler metrics in the validator module, enhancing its capabilities for handling and validating metrics. The changes include:

• Adding a new validate_metrics command to the validator CLI with --duration and --report-dir options.
• Modifying the PrometheusJob named tuple and the load function to include new fields and initialize them when loading configuration from a file.
• Introducing new functions, such as validate_metrics, ScriptResult, and write_md_report, and adding an optional dependency on click.exceptions.Exit.
• Creating a new bash script, regression-stressor.sh, to simulate CPU load and measure its impact on the system using the stress-ng tool.

Impact: These changes expand the validator module's capabilities for handling and validating Kepler metrics, but do not affect the signatures of exported functions or global data structures. However, the introduction of the new script might affect the overall system behavior when executed.

Observations and Suggestions:

• The changes seem to be well-structured and follow a logical flow, making it easier to understand the new feature's implementation.
• It would be beneficial to include additional tests to ensure the new validate_metrics command and the regression-stressor.sh script work as expected.
• Consider adding documentation or comments to explain the purpose and usage of the new functions and script, especially for users who may not be familiar with the validator module or Kepler metrics.
• Review the error handling and logging mechanisms to ensure they are robust and provide useful information in case of failures or errors.

[sthaha]

Can we also validate these invariants in the same version of dev

• kepler_node_<pkg|core|uncore|dram|other>{dev} = sum of ( process_<pkg|core|uncore|dram|other>{dev} )
• kepler_node_<pkg|core|..> = node_exporter_rapl_<pkg|core...>
  *sum( kepler_process_bpf_cpu ) = node_exporter_cpu_time

[vprashar2929]

for kepler_node_<pkg|core|dram...>{dev} = sum of (process_<pkg|core|dram....>){dev} do you mean
MAE of sum(rate(kepler_node<pkg|core|dram>){dev}[20s]) and sum(rate(process_<pkg|core|dram>{dev}[20s]))?

[sthaha]

can't we just return 0 or 1 ?

[vprashar2929]

by default when a Click script is invoked as command line application (through BaseCommand.main()) the return value is ignored unless the standalone_mode is disabled in which case it’s bubbled through.
https://click.palletsprojects.com/en/stable/commands/#command-return-values