Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

taxonomy: boolean added for european countries + ecobalyse id is adapted for organic ingredients #10929

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open

taxonomy: boolean added for european countries + ecobalyse id is adapted for organic ingredients #10929

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
44cf545
taxonomy: a new function assign_property_to_ingredients has been adde…
very-smartin Oct 9, 2024
41f3fd1
Merge branch 'main' into ecobalyse
very-smartin Oct 10, 2024
b5ab63e
updated tests
stephanegigandet Oct 11, 2024
49c07ef
lint
stephanegigandet Oct 11, 2024
e4d6e54
Merge branch 'main' into ecobalyse
very-smartin Oct 11, 2024
9b29b75
Merge branch 'main' into ecobalyse
very-smartin Oct 11, 2024
a606d32
Merge branch 'main' into ecobalyse
very-smartin Oct 14, 2024
98560d5
taxonomy: a new key has been added in country.txt in order to know if…
very-smartin Oct 21, 2024
ebb3add
Erratum in MakeFile
very-smartin Oct 21, 2024
9b2741b
ecobalyse: a boolean has been added in the country taxonomy for europ…
very-smartin Oct 24, 2024
2458c0c
ecobalyse: a boolean has been added in the country taxonomy for europ…
very-smartin Oct 24, 2024
8d449e7
Merge branch 'main' into ecobalyse
very-smartin Oct 24, 2024
fed86f1
test: Update tests results
Oct 24, 2024
150348f
chore: Linting changes
Oct 25, 2024
34ad737
Merge branch 'main' into ecobalyse
very-smartin Oct 25, 2024
f1803f6
taxonomy: changing the function assign_property_to_ingredient
very-smartin Oct 29, 2024
1273fd3
taxonomy: resolving conflicts
very-smartin Oct 29, 2024
43448e0
taxonomy: changing the function assign_property_to_ingredient
very-smartin Oct 29, 2024
7302c37
Merge branch 'main' into ecobalyse
very-smartin Oct 29, 2024
2206fea
taxonomy; new version of the get_missing_ecobalyse_ids function
very-smartin Oct 30, 2024
678a2a7
Merge branch 'main' into ecobalyse
very-smartin Oct 30, 2024
28d0a36
Merge branch 'main' into ecobalyse
very-smartin Oct 31, 2024
7034337
taxonomy: adressing bugsin Ingredients.pm
very-smartin Oct 31, 2024
ca5d35f
Merge branch 'main' into ecobalyse
very-smartin Oct 31, 2024
96b0cfd
tackling (again) Perl errors
very-smartin Nov 4, 2024
d5ae6fa
Merge branch 'main' into ecobalyse
very-smartin Nov 4, 2024
995ade1
tackling (again) Perl errors
very-smartin Nov 4, 2024
97695e3
Merge branch 'ecobalyse' of https://github.com/openfoodfacts/openfood…
very-smartin Nov 4, 2024
0828916
tackling (again) Perl errors
very-smartin Nov 4, 2024
5e5a8e2
chore: Linting changes
Nov 4, 2024
be2e361
Merge branch 'main' into ecobalyse
very-smartin Nov 12, 2024
1b69d54
Correcting tests/unit for ecobalyse
very-smartin Nov 12, 2024
bea6622
Merge branch 'main' into ecobalyse
very-smartin Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 36 additions & 16 deletions lib/ProductOpener/Ingredients.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3368,23 +3368,43 @@ sub get_missing_ecobalyse_ids ($ingredients_ref) {
delete $ingredient_ref->{ecobalyse_code};
delete $ingredient_ref->{ecobalyse_proxy_code};

# Getting properties from product
#my $product_origin = has_specific_ingredient_property()

# Getting the correct ecobalyse comake devde
my $ecobalyse_code = get_inherited_property("ingredients", $ingredient_ref->{id}, "ecobalyse:en");
if (defined $ecobalyse_code) {
$ingredient_ref->{ecobalyse_code} = $ecobalyse_code;
}
else {
my $ecobalyse_proxy_code
= get_inherited_property("ingredients", $ingredient_ref->{id}, "ecobalyse_proxy:en");
if (defined $ecobalyse_proxy_code) {
$ingredient_ref->{ecobalyse_proxy_code} = $ecobalyse_proxy_code;
}
else {
push(@ingredients_without_ecobalyse_ids, $ingredient_ref->{id});
# We are now looking for the appropriate ecobalyse id :
# ecobalyse_origins_france_label_organic (if the product comes from france, and is organic)
# ecobalyse_origins_european-union_label_organic (if the product comes from europe, and is organic)
# ecobalyse_label_organic (if the product is organic)
# ecobalyse_origins_france (if the product comes from france)
# ecobalyse_origins_european-union (if the product comes from the Europe region)
# ecobalyse (else)

# List of suffixes
my @suffixes = ();
# If the ingredient is organic...
if ($ingredient_ref->{labels} =~ /\ben:organic\b/) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach won't work if we have an organic ingredient for which we have a general ecobalyse match, but no organic ecobalyse match.

e.g. if we have "organic pear", in the ingredients, we don't have it in Ecobalyse:

ecobalyse:en: pear-eu
ecobalyse_origins_en_france:en: pear-fr

but we need to fallback on pear-eu.

So I suggest a different approach:

Based on the labels, origins etc. of the product, construct a list of prioritized ecobalyse suffixes.

e.g. if the product is "organic pear from france", we might want to get:

  1. ecobalyse_origins_france_label_organic
  2. ecobalyse_origins_european-union_label_organic
  3. ecobalyse_label_organic
  4. ecobalyse_origins_france
  5. ecobalyse_origins_european-uinion
  6. ecobalyse

(or maybe a different order, depending on what we consider more important: the origin or the label)

Then we check if we have those as ecobalyse properties for "en:pear", and we stop at the first one we find.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point indeed, i'll change my code consequently

push @suffixes, "_labels_organic";
}
push @suffixes, '';
push @suffixes, '_proxy';

# Loop through each suffix and proxy combination to retrieve ecobalyse code
foreach my $suffix (@suffixes) {
# Construct the property name using the proxy and suffix
my $property_name = "ecobalyse" . $suffix . ":en";

# Attempt to retrieve the ecobalyse code for the current property name
my $ecobalyse_code = get_inherited_property("ingredients", $ingredient_ref->{id}, $property_name);

if (defined $ecobalyse_code) {
# Assign the ecobalyse code if found
$ingredient_ref->{ecobalyse_code} = $ecobalyse_code;
last;
}
# Exit the loop if a valid ecobalyse code was found
last if defined $ingredient_ref->{ecobalyse_code};
}

# If no ecobalyse code was found, add ingredient ID to list of missing codes
if (!defined $ingredient_ref->{ecobalyse_code}) {
push(@ingredients_without_ecobalyse_ids, $ingredient_ref->{id});
}

#ecobalyse:en
Expand Down
28 changes: 28 additions & 0 deletions taxonomies/countries.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3127,6 +3127,7 @@ diq: Awıstırya
dsb: Awstriska
dv: އޮސްޓްރިއާ
dz: ཨས་ཊི་ཡ
ecobalyse_is_part_of_eu: yes
ee: Austria
el: Αυστρία
en-ca:Austria
Expand Down Expand Up @@ -4435,6 +4436,7 @@ diq: Belçıka
dsb: Belgiska
dv: ބެލްޖިއަމް
dz: བེལ་ཇིཡམ
ecobalyse_is_part_of_eu: yes
ee: Belgium
el: Βέλγιο
eml: Bélgi
Expand Down Expand Up @@ -6817,6 +6819,7 @@ de-ch:Bulgarien
diq: Bulğarıstan, Bulgarıstan
dsb: Bulgarska
dv: ބަލްގޭރިއާ
ecobalyse_is_part_of_eu: yes
ee: Bulgaria
el: Βουλγαρία, BUL, Boύλγαροι
eml: Bulgarî, Bulgaria
Expand Down Expand Up @@ -10250,6 +10253,7 @@ de: Kroatien
diq: Xırvatıstan
dsb: Chorwatska
dv: ކުރޮއޭޝިއާ
ecobalyse_is_part_of_eu: yes
ee: Croatia
el: Κροατία
en-ca:Croatia
Expand Down Expand Up @@ -10779,6 +10783,7 @@ da: Cypern
de: Republik Zypern, Südzypern, Kıbrıs Cumhuriyeti, Kypriakí Dimokratía, Zypern
diq: Qıbrıs
dsb: Cypern
ecobalyse_is_part_of_eu: yes
ee: Cyprus
el: Κύπρος, Κυπριακή Δημοκρατία
en-ca:Cyprus
Expand Down Expand Up @@ -11002,6 +11007,7 @@ diq: Çekıstan
dsb: Česka
dv: ޗެކް ޖުމްހޫރިއްޔާ
dz: ཅེཀ་རི་པབ་ལིཀ་
ecobalyse_is_part_of_eu: yes
ee: Czech Republic
el: Τσεχία
en-ca:Czech Republic
Expand Down Expand Up @@ -11621,6 +11627,7 @@ diq: Danimarka
dsb: Dańska
dv: ޑެންމާކު
dz: ཌེན་མཱཀ་
ecobalyse_is_part_of_eu: yes
ee: Denmark
el: Δανία
en-ca:Denmark
Expand Down Expand Up @@ -13507,6 +13514,7 @@ de: Estland, Republik Estland
diq: Estonya
dsb: Estniska
dz: ཨིསི་ཊོ་ནི་ཡ།
ecobalyse_is_part_of_eu: yes
ee: Estonia
el: Εσθονία
eml: Estògna
Expand Down Expand Up @@ -14765,6 +14773,7 @@ diq: Finlanda
dsb: Finska
dv: ފިންލޭންޑު
dz: ཕིན་ལེནཌ་
ecobalyse_is_part_of_eu: yes
ee: Finland
el: Φινλανδία
en-ca:Finland
Expand Down Expand Up @@ -15024,6 +15033,7 @@ diq: Fransa
dsb: Francojska
dv: ފަރަންސޭސިވިލާތް
dz: ཕརཱནསི་
ecobalyse_is_part_of_eu: yes
ee: France
el: Γαλλία
eml: Franza
Expand Down Expand Up @@ -16249,6 +16259,7 @@ diq: Almanya
dsb: Nimska
dv: ޖަރުމަނުވިލާތް
dz: ཇཱར་མ་ནི་
ecobalyse_is_part_of_eu: yes
ee: Germany
el: Γερμανία, Ομοσπονδιακή Δημοκρατία της Γερμανίας
eml: Germâgna
Expand Down Expand Up @@ -16872,6 +16883,7 @@ diq: Yunanıstan
dsb: Grichiska
dv: ޔޫނާން
dz: གིརིསི་
ecobalyse_is_part_of_eu: yes
ee: Greece
el: Ελλάδα, Ελλάς, Ελληνική Δημοκρατία, Ἑλλάς
en-ca:Greece
Expand Down Expand Up @@ -19126,6 +19138,7 @@ de-ch:Ungarn
diq: Macarıstan
dsb: Hungorska
dv: ހަންގޭރީ
ecobalyse_is_part_of_eu: yes
ee: Hungary
el: Ουγγαρία
eml: Ungherî
Expand Down Expand Up @@ -20955,6 +20968,7 @@ diq: İtalya
dsb: Italska
dv: އިޓަލީވިލާތް
dz: ཨྀཊ་ལི་
ecobalyse_is_part_of_eu: yes
ee: Italy
el: Ιταλία
eml: Itâglia
Expand Down Expand Up @@ -23221,6 +23235,7 @@ diq: Letonya
dsb: Letiska
dv: ލެޓުވިއާ
dz: ལེཊི་བི་ཡ།
ecobalyse_is_part_of_eu: yes
ee: Latvia
el: Λεττονία
en-ca:Latvia
Expand Down Expand Up @@ -24464,6 +24479,7 @@ diq: Litwanya
dsb: Litawska
dv: ލިތުއޭނިއާ
dz: ལི་ཐུ་ནི་ཡ།
ecobalyse_is_part_of_eu: yes
ee: Lithuania
el: Λιθουανία
eml: Lituàgna
Expand Down Expand Up @@ -24713,6 +24729,7 @@ de: Luxemburg
de-ch:Luxemburg
diq: Luksemburg
dsb: Luxemburgska
ecobalyse_is_part_of_eu: yes
ee: Luxembourg
el: Λουξεμβούργο
eml: Lussembûrgh, Lussembûrg, Lusembûrgh
Expand Down Expand Up @@ -26048,6 +26065,7 @@ de-ch:Malta
diq: Malta
dsb: Malta
dv: މޯލްޓާ
ecobalyse_is_part_of_eu: yes
ee: Malta
el: Μάλτα
eml: Malta
Expand Down Expand Up @@ -29463,6 +29481,7 @@ diq: Hollanda
dsb: Nižozemska
dv: ނެދަލޭންޑު
dz: ནེ་དར་ལེནཌསི་
ecobalyse_is_part_of_eu: yes
ee: Netherlands
el: Ολλανδία, Κάτω Χώρες
en-ca:Netherlands
Expand Down Expand Up @@ -33245,6 +33264,7 @@ de-ch:Polen
diq: Polonya
dsb: Pólska
dv: ޕޮލެންޑު
ecobalyse_is_part_of_eu: yes
ee: Poland
el: Πολωνία
eml: Pulógna
Expand Down Expand Up @@ -33510,6 +33530,7 @@ diq: Portekiz
dsb: Portugalska
dv: ޕޯޗުގަލް
dz: པོར་ཅུ་གལ་
ecobalyse_is_part_of_eu: yes
ee: Portugal
el: Πορτογαλία
en-ca:Portugal
Expand Down Expand Up @@ -34276,6 +34297,7 @@ de-ch:Irland
diq: İrlanda
dsb: Irska
dv: އަޔަލޭންޑުގެ ޖުމްހޫރިއްޔާ
ecobalyse_is_part_of_eu: yes
ee: Ireland
el: Δημοκρατία της Ιρλανδίας, Έιρε, Ιρλανδική Δημοκρατία
en-ca:Republic of Ireland
Expand Down Expand Up @@ -34751,6 +34773,7 @@ de-formal:Rumänien
diq: Romanya
dsb: Rumuńska
dv: ރުމޭނިއާ
ecobalyse_is_part_of_eu: yes
ee: Romania
el: Ρουμανία
eml: Rumanî
Expand Down Expand Up @@ -38268,6 +38291,7 @@ de: Slowakei
diq: Slowakya
dsb: Słowakska
dv: ސުލޮވާކިއާ
ecobalyse_is_part_of_eu: yes
ee: Slovakia
el: Σλοβακία
eo: Slovakio, Slovaka respubliko
Expand Down Expand Up @@ -38492,6 +38516,7 @@ de-ch:Slowenien
diq: Slowenya
dsb: Słowjeńska
dv: ސުލޮވީނިއާ
ecobalyse_is_part_of_eu: yes
ee: Slovenia
el: Σλοβενία
en-ca:Slovenia
Expand Down Expand Up @@ -39967,6 +39992,7 @@ diq: İspanya
dsb: Špańska
dv: އިސްޕެއިން
dz: སིཔཱེན་
ecobalyse_is_part_of_eu: yes
ee: Spain
el: Ισπανία
eml: Spaggna
Expand Down Expand Up @@ -41103,6 +41129,7 @@ diq: İswec
dsb: Šwedska
dv: ސުވިޑަން
dz: སུའི་ཌན་
ecobalyse_is_part_of_eu: yes
ee: Sweden
el: Σουηδία
en-ca:Sweden
Expand Down Expand Up @@ -44911,6 +44938,7 @@ diq: Britanyaya Gırde, Britanya Gırd, Britanya, İngıltere, Britanya Gırdı,
dsb: Wjelika Britaniska
dv: ޔުނައިޓެޑް ކިންގްޑަމް
dz: ཡུ་ནའི་ཊེཊ་ཀིང་ཌམ
ecobalyse_is_part_of_eu: yes
ee: United Kingdom
el: Ηνωμένο Βασίλειο
eml: Régn Unî
Expand Down
7 changes: 4 additions & 3 deletions ...tion/expected_test_results/convert_and_import_excel_file/test/products/3270190128403.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,7 +630,6 @@
"ingredients" : [
{
"ciqual_food_code" : "20505",
"ecobalyse_code" : "lentils-uncooked-eu",
"id" : "en:green-lentils",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand Down Expand Up @@ -685,8 +684,10 @@
"ingredients_with_unspecified_percent_sum" : 100,
"ingredients_without_ciqual_codes" : [],
"ingredients_without_ciqual_codes_n" : 0,
"ingredients_without_ecobalyse_ids" : [],
"ingredients_without_ecobalyse_ids_n" : 0,
"ingredients_without_ecobalyse_ids" : [
"en:green-lentils"
],
"ingredients_without_ecobalyse_ids_n" : 1,
"interface_version_created" : "import_csv_file - version 2019/09/17",
"known_ingredients_n" : 5,
"labels" : "Organic, EU Organic, FR-BIO-01, AB Agriculture Biologique, Agriculture France",
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/expected_test_results/attributes/fr-palm-oil-free.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,7 +630,7 @@
},
{
"ciqual_proxy_food_code" : "9410",
"ecobalyse_proxy_code" : "flour",
"ecobalyse_code" : "flour",
"id" : "en:flour",
"is_in_taxonomy" : 1,
"percent_estimate" : 18.75,
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/expected_test_results/ingredients/es-procedente-e-agricultura-biologica.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"ingredients" : [
{
"ciqual_food_code" : "19023",
"ecobalyse_code" : "milk",
"ecobalyse_code" : "milk-organic",
"id" : "en:pasteurised-whole-cow-s-milk",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/expected_test_results/ingredients/fi-organic-label-part-of-ingredient.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
},
{
"ciqual_proxy_food_code" : "19051",
"ecobalyse_code" : "milk",
"ecobalyse_code" : "milk-organic",
"id" : "en:milk",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand Down
6 changes: 3 additions & 3 deletions tests/unit/expected_test_results/ingredients/fi-origins.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@
},
{
"ciqual_proxy_food_code" : "13034",
"ecobalyse_code" : "orange-non-eu",
"id" : "en:orange",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand Down Expand Up @@ -130,9 +129,10 @@
"ingredients_without_ecobalyse_ids" : [
"en:blackcurrant",
"en:cocoa",
"en:cocoa-butter"
"en:cocoa-butter",
"en:orange"
],
"ingredients_without_ecobalyse_ids_n" : 3,
"ingredients_without_ecobalyse_ids_n" : 4,
"known_ingredients_n" : 10,
"lc" : "fi",
"nutriments" : {
Expand Down
10 changes: 5 additions & 5 deletions tests/unit/expected_test_results/ingredients/fr-origins-labels.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@
},
{
"ciqual_proxy_food_code" : "13034",
"ecobalyse_code" : "orange-non-eu",
"id" : "en:orange",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand All @@ -65,7 +64,7 @@
},
{
"ciqual_proxy_food_code" : "13009",
"ecobalyse_code" : "lemon-eu",
"ecobalyse_code" : "lemon-organic",
"id" : "en:lemon",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand Down Expand Up @@ -102,7 +101,7 @@
},
{
"ciqual_food_code" : "13008",
"ecobalyse_code" : "cherry-tr",
"ecobalyse_code" : "cherry-organic",
"id" : "en:cherry",
"is_in_taxonomy" : 1,
"labels" : "en:organic",
Expand Down Expand Up @@ -180,9 +179,10 @@
"en:blackcurrant",
"en:cocoa",
"en:cocoa-butter",
"en:grapefruit"
"en:grapefruit",
"en:orange"
],
"ingredients_without_ecobalyse_ids_n" : 4,
"ingredients_without_ecobalyse_ids_n" : 5,
"known_ingredients_n" : 14,
"lc" : "fr",
"nutriments" : {
Expand Down
Loading
Loading