core: aggregate errors that happen too often during infra loading

[eckter]

Fix #8773

After:

[15:46:49,574] [INFO]          [ValidateInfra] parsing json
[15:46:52,963] [INFO]          [ValidateInfra] parsing RailJSON
[15:46:57,358] [WARN]      [RawInfraRJSParser] Electrification conflict on track-range TRACK_ID[53m, 1168m]: 25000V != 3000V
[15:46:57,358] [WARN]      [RawInfraRJSParser] Electrification conflict on track-range TRACK_ID[23m, 487m]: 25000V != 3000V
[15:46:57,358] [WARN]      [RawInfraRJSParser] Electrification conflict on track-range TRACK_ID[487m, 974m]: 25000V != 3000V
[15:46:57,371] [WARN]      [RawInfraRJSParser] ... and 90 other similar errors
[15:47:10,282] [INFO]          [ValidateInfra] loading signals
[15:47:10,502] [INFO]          [ValidateInfra] building blocks
[15:47:12,318] [DEBUG]          [BlockBuilder] no signal at non buffer stop detector.DETECTOR_ID:DECREASING
[15:47:12,318] [DEBUG]          [BlockBuilder] no signal at non buffer stop detector.DETECTOR_ID:INCREASING
[15:47:12,318] [DEBUG]          [BlockBuilder] no signal at non buffer stop detector.DETECTOR_ID:INCREASING
[15:47:12,328] [DEBUG]          [BlockBuilder] ... and 1256 other similar errors
[15:47:12,536] [DEBUG] [SignalingSimulatorImpl] error in block BLOCK_ID: too_few_signals
[15:47:12,536] [DEBUG] [SignalingSimulatorImpl] error in block BLOCK_ID: too_few_signals
[15:47:12,536] [DEBUG] [SignalingSimulatorImpl] error in block BLOCK_ID: too_few_signals
[15:47:12,537] [DEBUG] [SignalingSimulatorImpl] ... and 955 other similar errors
[15:47:12,537] [INFO]          [ValidateInfra] done

Before: same thing but over 2k2 lines

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.63%. Comparing base (675d279) to head (666849a).
Report is 1 commits behind head on dev.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev   #10345      +/-   ##
==========================================
- Coverage   81.64%   81.63%   -0.01%     
==========================================
  Files        1066     1066              
  Lines      105735   105735              
  Branches      727      727              
==========================================
- Hits        86328    86321       -7     
- Misses      19366    19373       +7     
  Partials       41       41              

Flag	Coverage Δ
editoast	73.65% <ø> (-0.03%)	⬇️
front	89.33% <ø> (ø)
gateway	2.18% <ø> (ø)
osrdyne	3.28% <ø> (ø)
railjson_generator	87.50% <ø> (ø)
tests	87.05% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bougue-pe]

The solution is quite elegant.

But how about just raising the log level when deployed (or even default to info)?
Then we can also consider lowering log-level for electrification (and move this to a more important tracking on the data-production side).
If we are able to target specific log-levels on specific areas of the code (like warning for infra import and debug for the rest), it may also avoid revision of all log-levels of the component.

[bougue-pe]

It's OK to merge IMO.
The previous comment about adjusting log-level is still "alive" IMO, but any pick is OK.

[eckter]

But how about just raising the log level when deployed (or even default to info)?
Then we can also consider lowering log-level for electrification (and move this to a more important tracking on the data-production side).
If we are able to target specific log-levels on specific areas of the code (like warning for infra import and debug for the rest), it may also avoid revision of all log-levels of the component.

We should probably look into this at some point (refining the log levels and what we want in prod).

But I don't think it would be enough, some errors are important enough to be logged in prod environments, without necessarily having all the details.

Like I've just added an aggregation for this error that happens in prod but (usually) not in local, it's quite important to know that it's there but there's no need to log it dozens of times per request.

[Erashin]

Works for me, GJ!