New section 4 templates and tables for verification scores #255

sebvi · 2024-05-14T12:20:47Z

Initial request

ECMWF is interested in storing verification scores in GRIB2 in a near future.

With the existing GRIB2 templates, it is not possible to properly encode verification scores. The following templates are a proposal of how verification scores could be encoded in GRIB2.

We are not aiming to get this validated for FT 2024-2, we would like to start the discussion and eventually get it as experimental so that we can refine the proposal.

Amendment details

ADD to code table 4.0 Product definition template number

Code	Description
137	Verification scores for analysis or forecast at a horizontal level or in a horizontal layer at a point in time
138	Verification scores for average, accumulation, and/or extreme values or other statistically processed values at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval
139	Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time
140	Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval
141	Verification scores for derived forecast based on all ensemble members at a horizontal level or in a horizontal layer at a point in time
142	Verification scores for derived forecasts based on all ensemble members at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

ADD Template 4.137 Verification scores for analysis or forecast at a horizontal level or in a horizontal layer at a point in time

Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-32	4	Scaled Value of Second Fixed Surface
35-36	2	Verification Score (see code table 4.120)
37	1	Type of Reference Dataset for Verification (see code table 4.121)
38	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
39	1	Type of Threshold Operator for Verification Score (see code table 4.91)
40	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
41	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=0:NA – can be 0 times if no parameters are necessary (na=0)
42 + (na-1)*5	1	Scale factor of additional Argument for Verification
43 + (na-1)5 to 47+(na-1)5	4	Scaled value of additional Argument for Verification
42 + NA5 to 43+ NA5	2	Year of Start Of Verification Period
44 + NA*5	1	Month of Start of Verification Period
45 + NA*5	1	Day of Start of Verification Period
46 + NA*5	1	Hour of Start of Verification Period
47 + NA*5	1	Minute of Start of Verification Period
48 + NA*5	1	Second of Start of Verification Period
49 + NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
50 + NA5+ (nv-1)11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
51 + NA5(nv-1)11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
52 + NA5(nv-1)11 to 55 + NA5(nv-1)11	4	Length of Time Range for Verification Period
56 + NA5(nv-1)11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
57 + NA5(nv-1)11 to 60 + NA5(nv-1)11	4	Time Increment for Verification Period
61 + NA5+NV11 to 62 + NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.138 Verification scores for average, accumulation, and/or extreme values or other statistically processed values at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35-36	2	Year of End of Overall Time Interval
37	1	Month of end of Overall Time Interval
38	1	Day of end Of Overall Time Interval
39	1	Hour of end of Overall Time Interval
40	1	Minute of end of Overall Time Interval
41	1	Second of end of Overall Time Interval
42	1	Number of Time Range – NR
43-46	4	Number of Missing in Statistical Process
		The next six entries are repeated NR times nr=1:NR
47 + (nr-1)*12	1	Type of Statistical Processing (see code table 4.10)
48 + (nr-1)*12	1	Type of Time Increment (see code table 4.11)
49 + (nr-1)*12	1	Indicator of Unit for Time Range (see code table 4.4)
50 + (nr-1)12 to 53 + (nr-1)12	4	Length of Time Range
54 + (nr-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
55 + (nr-1)12 to 58 + (nr-1)12	4	Time Increment
		End of sequence repeated NT times
59 + (NR-1)12 to 60 + (NR-1)12	2	Verification Score (see code table 4.120)
61 + (NR-1)*12	1	Type of Reference Dataset for Verification (see code table 4.121)
62 + (NR-1)*12	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
63 + (NR-1)*12	1	Type of Threshold Operator for Verification Score (see code table 4.91)
64 + (NR-1)*12	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
65 + (NR-1)*12	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=0:NA – can be 0 times if no parameters are necessary (na=0)
66 + (NR-1)12 + (na-1)5	1	Scale factor of additional Argument for Verification
67 + (NR-1)12 + (na-1)5 to 70 + (NR-1)12 +(na-1)5	4	Scaled value of additional Argument for Verification
66 + (NR-1)12 + NA5 to 67 + (NR-1)12 + NA5	2	Year of Start Of Verification Period
68 + (NR-1)12 + NA5	1	Month of Start of Verification Period
69 + (NR-1)12 + NA5	1	Day of Start of Verification Period
70 + (NR-1)12 + NA5	1	Hour of Start of Verification Period
71 + (NR-1)12 + NA5	1	Minute of Start of Verification Period
72 + (NR-1)12 + NA5	1	Second of Start of Verification Period
73 + NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
74 + (NR-1)12 1+ NA5+ (nv-1)*11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
75 + (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
76 + (NR-1)12 + NA5(nv-1)11 to 79 + (NR-1)12 + NA5(nv-1)11	4	Length of Time Range for Verification Period
80 + (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
81 + (NR-1)12 + NA5(nv-1)11 to 84 + (NR-1)12 + NA5(nv-1)11	4	Time Increment for Verification Period
85 + (NR-1)12 + NA5+NV11 to 86+ (NR-1)12 + NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.139 Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time

Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35	1	Type of Ensemble Forecast (see code table 4.6)
36-39	4	Perturbation Number
40-43	4	Number of Forecasts in Ensemble
44-45	2	Verification Score (see code table 4.120)
46	1	Type of Reference Dataset for Verification (see code table 4.121)
47	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
48	1	Type of Threshold Operator for Verification Score (see code table 4.91)
49	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
50	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=0:NA – can be 0 times if no parameters are necessary (na=0)
51 + (na-1)*5	1	Scale factor of additional Argument for Verification
52 + (na-1)5 to 55 + (na-1)5	4	Scaled value of additional Argument for Verification
51 + NA5 to 52 + NA5	2	Year of Start Of Verification Period
53 + NA*5	1	Month of Start of Verification Period
54 + NA*5	1	Day of Start of Verification Period
55 + NA*5	1	Hour of Start of Verification Period
56 + NA*5	1	Minute of Start of Verification Period
57 + NA*5	1	Second of Start of Verification Period
58 + NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
59 + NA5+ (nv-1)11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
60 + NA5(nv-1)11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
61 + NA5(nv-1)11 to 64 + NA5(nv-1)11	4	Length of Time Range for Verification Period
65 + NA5(nv-1)11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
66 + NA5(nv-1)11 to 69 + NA5(nv-1)11	4	Time Increment for Verification Period
70+ NA5+NV11 to 71 + NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.140 Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
244	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35	1	Type of Ensemble Forecast (see code table 4.6)
36-39	4	Perturbation Number
40-43	4	Number of Forecasts in Ensemble
44-45	2	Year of End of Overall Time Interval
46	1	Month of end of Overall Time Interval
47	1	Day of end Of Overall Time Interval
48	1	Hour of end of Overall Time Interval
49	1	Minute of end of Overall Time Interval
50	1	Second of end of Overall Time Interval
51	1	Number of Time Range – NR
52-55	4	Number of Missing in Statistical Process
		The next six entries are repeated NR times nr=1:NR
56 + (nr-1)*12	1	Type of Statistical Processing (see code table 4.10)
57 + (nr-1)*12	1	Type of Time Increment (see code table 4.11)
58 + (nr-1)*12	1	Indicator of Unit for Time Range (see code table 4.4)
59 + (nr-1)12 to 62 + (nr-1)12	4	Length of Time Range
63 + (nr-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
64 + (nr-1)12 to 67 + (nr-1)12	4	Time Increment
		End of sequence repeated NT times
68 + (NR-1)12 to 69 + (NR-1)12	2	Verification Score (see code table 4.120)
70 + (NR-1)*12	1	Type of Reference Dataset for Verification (see code table 4.121)
71 + (NR-1)*12	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
72 + (NR-1)*12	1	Type of Threshold Operator for Verification Score (see code table 4.91)
73 + (NR-1)*12	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
74 + (NR-1)*12	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=0:NA – can be 0 times if no parameters are necessary (na=0)
75 + (NR-1)12 + (na-1)5	1	Scale factor of additional Argument for Verification
76 + (NR-1)12 + (na-1)5 to 79 + (NR-1)12 +(na-1)5	4	Scaled value of additional Argument for Verification
75 + (NR-1)12 + NA5 to 76+ (NR-1)12 + NA5	2	Year of Start Of Verification Period
77 + (NR-1)12 + NA5	1	Month of Start of Verification Period
78 + (NR-1)12 + NA5	1	Day of Start of Verification Period
79 + (NR-1)12 + NA5	1	Hour of Start of Verification Period
80 + (NR-1)12 + NA5	1	Minute of Start of Verification Period
81 + (NR-1)12 + NA5	1	Second of Start of Verification Period
82 + NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
83 + (NR-1)12 1+ NA5+ (nv-1)*11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
84 + (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
85 + (NR-1)12 + NA5(nv-1)11 to 88 + (NR-1)12 + NA5(nv-1)11	4	Length of Time Range for Verification Period
89 + (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
90 + (NR-1)12 + NA5(nv-1)11 to 93 + (NR-1)12 + NA5(nv-1)11	4	Time Increment for Verification Period
94 + (NR-1)12 + NA5+NV11 to 95 + (NR-1)12 + NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.141 Verification scores for derived forecast based on all ensemble members at a horizontal level or in a horizontal layer at a point in time

Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35	1	Derived Forecast (see code table 4.7)
36-39	4	Number of Forecasts in Ensemble
40-41	2	Verification Score (see code table 4.120)
42	1	Type of Reference Dataset for Verification (see code table 4.121)
43	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
44	1	Type of Threshold Operator for Verification Score (see code table 4.91)
45	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
46	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=0:NA – can be 0 times if no parameters are necessary (na=0)
47+ (na-1)*5	1	Scale factor of additional Argument for Verification
48+ (na-1)5 to 51+(na-1)5	4	Scaled value of additional Argument for Verification
47+ NA5 to 48+ NA5	2	Year of Start Of Verification Period
49+ NA*5	1	Month of Start of Verification Period
50+ NA*5	1	Day of Start of Verification Period
51+ NA*5	1	Hour of Start of Verification Period
52+ NA*5	1	Minute of Start of Verification Period
53+ NA*5	1	Second of Start of Verification Period
54+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
55+ NA5+ (nv-1)11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
56+ NA5(nv-1)11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
57+ NA5(nv-1)11 to 60+ NA5(nv-1)11	4	Length of Time Range for Verification Period
61+ NA5(nv-1)11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
62+ NA5(nv-1)11 to 65+ NA5(nv-1)11	4	Time Increment for Verification Period
66+ NA5+NV11 to 67+ NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.142 Verification scores for derived forecasts based on all ensemble members at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15-16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19-22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25-28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35	1	Derived Forecast (see code table 4.7)
36-39	4	Number of Forecasts in Ensemble
40-41	2	Year of End of Overall Time Interval
42	1	Month of end of Overall Time Interval
43	1	Day of end Of Overall Time Interval
44	1	Hour of end of Overall Time Interval
45	1	Minute of end of Overall Time Interval
46	1	Second of end of Overall Time Interval
47	1	Number of Time Range – NR
48-51	4	Number of Missing in Statistical Process
		The next six entries are repeated NR times nr=1:NR
52 + (nr-1)*12	1	Type of Statistical Processing (see code table 4.10)
53 + (nr-1)*12	1	Type of Time Increment (see code table 4.11)
54 + (nr-1)*12	1	Indicator of Unit for Time Range (see code table 4.4)
55 + (nr-1)12 to 58 + (nr-1)12	4	Length of Time Range
59 + (nr-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
60 + (nr-1)12 to 63 + (nr-1)12	4	Time Increment
		End of sequence repeated NT times
64 + (NR-1)12 to 65+ (NR-1)12	2	Verification Score (see code table 4.120)
66 + (NR-1)*12	1	Type of Reference Dataset for Verification (see code table 4.121)
67 + (NR-1)*12	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
68 + (NR-1)*12	1	Type of Threshold Operator for Verification Score (see code table 4.91)
69 + (NR-1)*12	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
70 + (NR-1)*12	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=0:NA – can be 0 times if no parameters are necessary (na=0)
71 + (NR-1)12 + (na-1)5	1	Scale factor of additional Argument for Verification
72 + (NR-1)12 + (na-1)5 to 75 + (NR-1)12 +(na-1)5	4	Scaled value of additional Argument for Verification
71 + (NR-1)12 + NA5 to 72 + (NR-1)12 + NA5	2	Year of Start Of Verification Period
73 + (NR-1)12 + NA5	1	Month of Start of Verification Period
74 + (NR-1)12 + NA5	1	Day of Start of Verification Period
75 + (NR-1)12 + NA5	1	Hour of Start of Verification Period
76 + (NR-1)12 + NA5	1	Minute of Start of Verification Period
77 + (NR-1)12 + NA5	1	Second of Start of Verification Period
78 + NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
79+ (NR-1)12 1+ NA5+ (nv-1)*11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
80 + (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
81 + (NR-1)12 + NA5(nv-1)11 to 84 + (NR-1)12 + NA5(nv-1)11	4	Length of Time Range for Verification Period
85 + (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
86 + (NR-1)12 + NA5(nv-1)11 to 89 + (NR-1)12 + NA5(nv-1)11	4	Time Increment for Verification Period
90 + (NR-1)12 + NA5+NV11 to 91 + (NR-1)12 + NA5+NV11	2	Number of Forecasts in Verification

ADD NEW code table 4.120 Verification scores

Code	Description
0	Mean error
1	Root mean squared error
2	Continuous ranked probability score (CRPS)
3	Fair continuous ranked probability score
4	Continuous logarithmic ignorance score
5	Fair continuous logarithmic ignorance score
6	Ranked probability score
7-99	Reserved for future use
100	Contingency Table (deterministic) - hits
101	Contingency Table (deterministic) - false alarms
102	Contingency Table (deterministic) - misses
103	Contingency Table (deterministic) - correct negatives
104	Contingency Table (probabilistic) - Forecast-by-n-members and observed
105	Contingency Table (probabilistic) - Forecast-by-n-members and not observed
106- 62999	Reserved
63000-65534	For experimental use at local level
65535	Missing

Note: The entries 100 to 105 need a threshold. The number of additional keys is equal to the number of arguments specified for the code number set regarding table 4.122 in the key ‘Kind of additional arguments for verification score’. For entries 104 and 105 and ensemble template must be used to define the total number of members and n.

ADD NEW code table 4.121 Type of reference dataset for verification

Code	Description
0	Analysis interpolated on forecast grid
1	Analysis with forecast interpolated on analysis grid
2	Analysis interpolated on a common grid with the forecast but different to original grid
3	Gridded observation on forecast grid
4	Gridded observation with forecast interpolated on observation grid
5-254	Reserved
255	Missing

ADD NEW code table 4.122 Kind of additional arguments for verification score

Code	Description
0	Fixed threshold value
1	Threshold specified by a quantile of the climatology
2	Threshold specified multiple of standard deviations of the climatology
3-254	Reserved
255	Missing

Note: For #1: NA =1 or 2 with one scale factor / scaled value for a fixed threshold value if ‘type of limit for verification score’ includes only one value otherwise 2 for lower / upper limit

For #2: NA=1 or 2 1st tuple of scale factor / scaled value defined the total number of quantiles / 2nd tuple the quantile value if ‘type of limit for verification score’ includes only one value otherwise 4 for lower quantile / upper quantile

For #3: NA=1or 2 with scale factor / scaled value specifying the factor of standard deviation of the climatology if ‘type of limit for verification score’ includes only one value otherwise 2 for lower / upper limit

Comments

No response

Requestor(s)

Sebastien Villaume (ECMWF)
Robert Osinski (ECMWF)

Stakeholder(s)

ECMWF

Publication(s)

Manual on Code: new templates for section 4
Manual on Code: new tables and new entries in code Table 4.2

Expected impact of change

None

Collaborators

No response

References

No response

Validation

No response

amilan17 · 2024-05-16T12:46:47Z

https://github.com/wmo-im/CCT/wiki/Teleconference.16.May.2024 notes:
Sebastien introduced the proposal; Sibylle commented that there should always be two bytes;

amilan17 · 2024-06-05T14:08:18Z

https://github.com/wmo-im/CCT/wiki/Teleconference.16.May.2024 notes:
available for discussion probably for next fast-track;

sebvi · 2024-07-04T08:53:35Z

we have an ecCodes branch if someone is interested to test our proposed implementation and give feedback on this: feature/ft2024-2_255

sebvi · 2024-10-23T08:36:32Z

Hi @wmo-im/tt-tdcf ,

we haven't received comments or suggestions in relation to our prosposal for encoding verification scores. We are planning to go ahead with this during this fast track.

edouardsandrine · 2024-10-27T21:16:07Z

Really sorry for the delay.

I have a comment from one of our CCMEP experts:

In the table 4.120, it seems to be taken for granted that the contingency tables are 2x2 (hits, false alarms, misses and correct negatives). This seems restrictive to them. Shouldn't we consider swapping NxN tables whose 2x2 case is easy to obtain if need be by adding rows and columns so as to merge categories?

Thanks.

sebvi · 2024-11-09T16:26:39Z

Dear @edouardsandrine ,

thank you for your comments. I passed it on to our verification experts. Do you have examples of contingency tables of higher dimensions than 2x2? The new Code Table 4.120 would be able to cater for several contigency tables if it was needed.

edouardsandrine · 2024-11-10T00:44:41Z

I will try to find such examples and come back to you. Thanks for your comment!

edouardsandrine · 2024-11-12T14:20:54Z

Please, find below some examples from a reference for the protocol for exchanging performance metrics between centers that provides examples of 3x3 and 4x4 contingency tables:

https://confluence.ecmwf.int/display/WLD/Data+format+of+reports+of+station-based+scores

amilan17 · 2024-11-12T14:28:26Z

https://github.com/wmo-im/tt-tdcf/wiki/2024.11.12.tt.tdcf notes:
Sebastien believes that the proposal is ready for this FT; Jeff shared it with NCEP folks with no response so far; ECMWF uses these but currently in a different format

@sebvi provide samples with eccodes branch
validate

sebvi · 2024-12-03T11:02:14Z

We have prepared examples and an ecCodes version able to encode/decode the data
FT2025-1_255.tar.gz
eccodes-2.37.0-Source.tar.gz

amilan17 · 2024-12-03T13:40:23Z

https://github.com/wmo-im/et-data/wiki/2024.12.03.et.data notes:

@sebvi branch is ready for updating
@SibylleK will validate

Robert briefly explained the proposal;

amilan17 · 2025-01-14T13:43:39Z

https://github.com/wmo-im/et-data/wiki/2025.01.14.et.data notes:

Robert explained how the proposal has changed (will add description in comments) -- New examples and updated branch will be needed for validation;the proposal summary also needs to be updated;
@sebvi will update the proposal and branch and Robert will prepare the examples by next week;

rdosinski · 2025-01-15T16:15:36Z

Please find attached ASCII files with section 4 grib_dumps as discussed in the meeting from 14/01/2025 :
dumps_255.tar.gz

The suggested change in the proposal is to remove the octet 12 and add instead an entry anomaly into code table 4.3.

Please remove code table 4.119 Additional specification of verified parameter from the proposal and add instead an entry to table 4.3 for
Type of generating process | 23 | Anomaly

Please also remove octet 12, Additional Specification of verified parameter (see code table 4.119) from all of the six proposed templates and subtract one byte accordingly for all subsequent bytes from 13 onwards.

amilan17 · 2025-01-23T13:31:06Z

@sebvi to update branch and proposal

sebvi self-assigned this May 14, 2024

amilan17 modified the milestones: FT2024-1, FT2024-2, noTargetMilestone May 30, 2024

amilan17 added this to GRIB2 Amendments Aug 22, 2024

amilan17 moved this to In discussion in GRIB2 Amendments Aug 22, 2024

amilan17 modified the milestones: noTargetMilestone, FT2025-1 Nov 12, 2024

sebvi added a commit that referenced this issue Dec 20, 2024

issue #255 - adding new code tables 4.120, 34.121. 4.122

65a7391

amilan17 moved this from In progress to In validation / review in GRIB2 Amendments Jan 14, 2025

amilan17 linked a pull request Jan 23, 2025 that will close this issue

issue #255 - adding new code tables 4.120, 34.121. 4.122 #304

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New section 4 templates and tables for verification scores #255

New section 4 templates and tables for verification scores #255

sebvi commented May 14, 2024 •

edited

Loading

amilan17 commented May 16, 2024 •

edited

Loading

amilan17 commented Jun 5, 2024

sebvi commented Jul 4, 2024

sebvi commented Oct 23, 2024

edouardsandrine commented Oct 27, 2024

sebvi commented Nov 9, 2024

edouardsandrine commented Nov 10, 2024

edouardsandrine commented Nov 12, 2024 •

edited

Loading

amilan17 commented Nov 12, 2024 •

edited

Loading

sebvi commented Dec 3, 2024

amilan17 commented Dec 3, 2024 •

edited

Loading

amilan17 commented Jan 14, 2025 •

edited

Loading

rdosinski commented Jan 15, 2025

amilan17 commented Jan 23, 2025

New section 4 templates and tables for verification scores #255

New section 4 templates and tables for verification scores #255

Comments

sebvi commented May 14, 2024 • edited Loading

Initial request

Amendment details

Comments

Requestor(s)

Stakeholder(s)

Publication(s)

Expected impact of change

Collaborators

References

Validation

amilan17 commented May 16, 2024 • edited Loading

amilan17 commented Jun 5, 2024

sebvi commented Jul 4, 2024

sebvi commented Oct 23, 2024

edouardsandrine commented Oct 27, 2024

sebvi commented Nov 9, 2024

edouardsandrine commented Nov 10, 2024

edouardsandrine commented Nov 12, 2024 • edited Loading

amilan17 commented Nov 12, 2024 • edited Loading

sebvi commented Dec 3, 2024

amilan17 commented Dec 3, 2024 • edited Loading

amilan17 commented Jan 14, 2025 • edited Loading

rdosinski commented Jan 15, 2025

amilan17 commented Jan 23, 2025

sebvi commented May 14, 2024 •

edited

Loading

amilan17 commented May 16, 2024 •

edited

Loading

edouardsandrine commented Nov 12, 2024 •

edited

Loading

amilan17 commented Nov 12, 2024 •

edited

Loading

amilan17 commented Dec 3, 2024 •

edited

Loading

amilan17 commented Jan 14, 2025 •

edited

Loading