How to store multilingual data in OSM.

[c933103]

Currently, most multilingual data are stored wiht data for each languages of each tag entered as its own individual tag. For example name:en=, name:ja=, and so on, and that's not just name but also other tags where local language information and/or their translation of the object is expected. However it is not exactly easy to identify, translate, store, and present these data to people who use multiple languages in the current system. It also cannot avoid the problem of same language data being not up to date if multiple language tag data are entered into OSM database. How can the data model be improved to deal with this?

[sdicke]

Even if you have the ability to represent multiple language versions of a key value, there is no guaranty that an editor (a human using an editing software) updates all the languages variants even if you show them to him and ask to review. Why? Firstly the is only interested in a limited set of language variants (say his native language and maybe English in addition), secondly he/she/it has limited time and a set of information to insert into the database, for example notes he made for himself on the ground.

The data model could allow the associate multiple values with an key, say 7000 to match every language used by humans nowadays plus multiple local and regional variants (it is not guarantied that every object has only one local name). For editors it could be useful if locations have multiple names (such as many villages and towns in the German region of Nordfriesland where locations have often multiple names, one in the local dialect of the native Frisian language and one in the prevailing language). But the same effect can be achieved if editing software offers suitable interfaces for that. The mapping of multiple values to one key is possible, but it needs an additional indirection which make the software logic on the server side and on the client side more complicated and breaks many software used nowadays to process OpenStreetMap data.

If have no statistics about the usage of multilanguage usage in the OSM database, but I assume that only a very small amount of elements have name:LANGUAGE tags or similar tags.

[c933103]

Borrowing from other translation system, I guess it might be possible to: - Force values of some key, like name=*, be stored with language tag (Editing software can apply national language default so that when user entering data for like the US users don't need to specify English when they are typing English name) - When a user update default language value, flag other language data as outdated until they are being verified or updated by another user. - Editing software show transliteration of non-latin text in latin characters like what wikidata's editor is doing, to allow easy verification and deletion of inappropriate or outdated data

[matkoniecz]

Note that OSM is storing names of objects, and changing name in one language in general does not mean that name changed in other languages.

[c933103]

Note that OSM is storing names of objects, and changing name in one language in general does not mean that name changed in other languages.

Yes and I am talk about potential ways to encourage nonupdated values to be updated, or potential ways to make data users be aware that some values are less updated than others?

[Dimitar5555]

I think that at least for some object (countries, provinces, states, counties, regions, municipalities, cities, towns and villages) the names should be stored on Wikidata and fetched as needed (or on a regular basis). Just to give an example, the relation for the USA has 333 name tags (including short names, official names and old names), England has 227 names, Germany has 306 names. It's pretty much impossible to maintain such amount of data. Wikidata most certainly has a lot more names in diffrent languages and I see no point in duplicating this data.

Even if it's not possible to do drop all of these names from OSM, there can be a bot which watches the Wikidata entries and updates OSM on regular basis.

[matkoniecz]

the names should be stored on Wikidata and fetched as needed (or on a regular basis)

See https://community.openstreetmap.org/t/is-there-support-for-removing-all-name-tags-on-objects-with-a-wikidata-qid-present/8374 (that went even further but everything except notability issues applies also to this proposal)

It's pretty much impossible to maintain such amount of data.

It is quite possible.

[Dimitar5555]

the names should be stored on Wikidata and fetched as needed (or on a regular basis)

See https://community.openstreetmap.org/t/is-there-support-for-removing-all-name-tags-on-objects-with-a-wikidata-qid-present/8374 (that went even further but everything except notability issues applies also to this proposal)

Thanks for the link, I will check it out.

It's pretty much impossible to maintain such amount of data.

It is quite possible.

Let me reiterate it: it takes far too much scrolling (in iD, JOSM, Level0 and possibly other editors) to find the language that you are looking for. Not to mention the fact that some countries have multiple name related tags in the same language (official_name:XX, name:XX, alt_name:XX...) which makes finding all of "your" languages even harder.

[matkoniecz]

Nothing stops us from having exactly the same interface as Wikidata, that is just an implementation detail.

it takes far too much scrolling (in iD, JOSM, Level0 and possibly other editors)

at least in level0 you definitely should use ctrl+f

in JOSM you should be able to use custom filtering/labelling/searching