Introducing “Super Mapper“ Status in HOT Tasking Manager

[ramyaragupathy]

High-Level Overview

The introduction of a Super Mapper status in the HOT Tasking Manager proposes a new category of mapper above the current ‘Advanced’ level. Unlike the existing system, this new level aims to reflect genuine expertise and experience, addressing issues with the current criteria for advanced mappers.

Purpose: Create a globally recognized pool of highly skilled mappers to support complex projects and reduce the burden on validators.

Related: #3713

Ralph's proposal

Eligibility: Enrolment will be based on peer assessment, supported by an automated flagging system for candidates meeting predefined criteria.

Key Benefits:

• Ensure higher data quality for complex projects.
• Reduce validator workload by creating a specialized mapping pool.
• Recognize and retain core contributors to humanitarian mapping.
• Showing a pathway up to super mapper should encourage more to map and to reach advanced+ status

Discussion Points

Eligibility Criteria:

• Should Super Mapper status be based on general expertise across mapping or specialized skills (e.g., buildings, waterways)?
• What thresholds should trigger automated flagging for peer assessment?

Assessment Process:

• How should the peer-assessment process be designed to ensure fairness and transparency?
• Should validators play a role in assessing candidates?

Recognition Mechanisms:

• Would badge or stars for specific expertise areas (e.g., “Buildings Expert”) enhance mapper engagement?
• How can we ensure that recognition is meaningful without adding unnecessary complexity?

Integration with Tasking Manager:

• What changes are needed to the project creation/editing interface to utilize the Super Mapper status effectively?
• How can filtering and sorting options best accommodate this new status?

Community Impact:

• How can we ensure this change is seen as a positive progression rather than exclusionary?
• What steps can be taken to encourage mappers to strive for this status ?

[ramyaragupathy]

Ralph's super-mapper proposal

High Level Overview

Simply put, the Super Mapper status will involve creating a new level of mapper which is one step higher than ‘Advanced’ in Tasking Manager (this is NOT a Team). Enrollment to Super Mapper status will be by peer assessment based on experience (super mappers will be allowed to add other super mappers) but there will be an automatic flagging system for marking those that have met the required edits to be peer assessed.

Currently a mapper can reach Advanced status without ever having drawn a building. This makes the status of “Advanced” mapper highly questionable and does not alone reflect on the expertise of the mapper. This means that when creators of complex projects restrict mapping to intermediate / advanced level there are frequently data quality issues. Giving project managers the option to restrict mapping to a genuinely experienced GLOBAL pool will make all the difference.

The HOT Global Validator team is currently the best equivalent we have right now for ‘Super Mappers’ this means that they are often called to do complex mapping work and this pulls them away from vital validation efforts. Having a pool of Super Mappers should ease this pressure on validators, allowing them to focus more on validation!

The Super Mapper pool should become larger than the HOT Global Validators team and will include mappers that only use iD. The Super Mapper status will be a meaningful way of recognizing and publicly labelling the core contributors to humanitarian mapping — which will also go far to support retention of these committed volunteers.

Preamble

At present, we have three categories of status for mappers on the HOT Tasking Manager
Beginner, Intermediate and Advanced.

This status is allocated to each individual mapper without the need to belong to a team and appears on their HOT TM Profile.

We rely on this status of the mapper for deciding who can map on a project depending on the difficulty of the project. We also rely on this status to help identify mappers with the experience to be validators.

The criteria for the automated allocation of this status is
Beginner Mapper: …………….. 0 – 249 changesets
Intermediate Mapper:……….. 250 – 499 changesets
Advanced Mapper: …………... More than 500 changesets

A changeset can comprise one building or more than 100 buildings. Or it can comprise of one place of interest or more than 100 places of interest. This means that a mapper can become Advanced without ever having drawn a building or, if they only do one building per changeset, they can become an Advanced mapper with only having drawn 500 buildings, no roads and no waterways.
This makes the status of “Advanced” mapper highly questionable and does not on it’s own reflect on the expertise of the mapper or their understanding of what they are doing.

If we adjust the criteria for identifying the Advanced Mapper to better reflect the amount of mapping that mapper has done it will mean that a lot of existing mappers who already have Advanced status will drop down to Intermediate. This can be a negative act in the minds of those mappers and could have a demoralising effect.

The only realistic solution seems to be to create another level of mapper that is above that of Advanced Mapper and better reflects the knowledge and expertise of those mappers who are genuinely Advanced.

---

What will this move affect?

1. There are a number of areas that utilise this mapper status and would need to be addressed if this new status is instated:
2. The automated process for flagging those mappers that have achieved the required criteria.
   The status that appears on the mapper’s profile page.

3. The PERMISSIONS page in the Project creation/Edit mode.

4. The drop down menu filter in the CONTRIBUTIONS of the tasks page of the Project.

5. The stars against the mappers usernames in the CONTRIBUTIONS of the tasks page of the Project.

6. The circle graph on the Statistics page.

7. The filter on the Manage Users page.

8. The Set mapper level option on the Manage Users page.

---

What options do we have?

There are many aspects of what we do that needs specialist knowledge. If we only apply Super Mapper status to mappers that fit all criteria the pool will be very small and will exclude many mappers that may have the building knowledge and expertise but not the required level for other features.

Most projects are for buildings only so to exclude a buildings expert just because they have not been drawing waterways does not make good sense. Can we qualify them according to their area of expertise? Would this be a “Super Mapper” or an intermediate stage such as

• Buildings expert?
• Highway expert?
• Waterway expert?
• Landuse expert?
• PoI expert?

This could be a form of acquiring “stars” or “badges” recognising the quantity of work that a mapper has contributed as an Advanced mapper on the way to becoming a Super Mapper and would still help in identifying individuals for special or complex projects.

[RAytoun]

Would this be a “Super Mapper” or an intermediate stage such as
This would be in the form of a star or badges alongside their Advanced mapper status on their profile page indicating that they have achieved the criteria for Super Mapper status in that feature. We will have far more Buildings badges
Advanced
than highways or waterways
Advanced
Advanced
but this is a good step forward and will mean that we can use them on mapping of more complex buildings project just as we can use a mapper with the waterways badge can be selected for more complex mapping of waterways projects.

[spwoodcock]

Love this idea!

The key question for me (which I guess hasn't been decided yet), is the criteria at which a user is eligible for the 'super mapper' tag.

• Is the designation for TM edits only, or for edits globally on OpenStreetMap?
  It's probably easier to track for the whole of OpenStreetMap.

• Do we count new geometries added, or also edits, deletions?
  Probably we count everything.

For example, for buildings, we could: (1) get all changesets for user --> (2) iterate changesets for ways,nodes,relations --> (3) filter ways where building=*

That sort of thing could be repeated for different tags, for all users, on a schedule.

It's not so important the data is realtime, so these stats could be calculated on a daily cron.
The stats themselves may not even be relevant, as long as we determine the super mapper eligibility.

Implementation Ideas (DB)

Just throwing out ideas.

In the db we could add a new table and enum

• badge_type enum: SUPER_MAPPER_ROADS, SUPER_MAPPER_HIGHWAYS, SUPER_MAPPER_WATERWAYS, OTHER_FUTURE_BADGE, etc
• user_badges table: user_id, badge_type, contribution_count, is_approved.

If an entry is made for a user in the table, they are eligible, and we display the eligibility on the frontend + notify other super mappers to confirm.

If the status is approved, then we check the is_approved field and display the badge on their profile.

ignore this if there is already a plan!

[adiatmad]

Thanks for initiating this discussion!

• I love the badge idea!
• I also suggested including a quality assurance section and responsiveness to changeset discussions and TM discussions, a similar concept to what's shown in HDYC.

With specific criteria, we can still recognize more experienced mappers as Super Mappers.

[SColchester]

Good question @spwoodcock I'd argue it should count global edits to OSM (i.e. not just filter for TM edits), for two reasons:

• Edits to OSM via any path are all relevant
• Tracking just TM edits will be an extra tech lift that may not be worth the hassle

[ramyaragupathy]

agrew with @SColchester to not filter just TM edits

[michael63-osm]

I'd like to add another angle. I'm a bit reluctant to accept that we just need another threshold in some object count. We've seen people who supposedly adjusted their upload style in order to optimize their effort for obtaining a higher level. Even if you count objects and not changesets there are areas where these are low hanging fruits and others where mapping is really tricky. And if I understand the proposal correctly then mapping these places is where the new category should come into play.

If I should define a "task force" for difficult mapping projects then I would follow these lines:

• highway mappers who have a feeling for what a road network is, know how to deal with relations as numbered highways often belong to a relation for the complete highway number and there are public transport relations in more developed areas
• waterway mappers who know how to determine the direction of flow, are proficient in multipolygons (the lake on an island in a broad riverbed), know how to tag intermittent features and how to spot them
• building mappers who are not just able to deliver squared buildings but also more complex multipolygons, buildings with different height levels, buildings with building passages
• landuse mappers who do not go micromapping but developed a feeling for what a neighbourhood or a natural area looks like, what the predominant use is

Do not expect me to come up with a formula for how to calculate such levels. I also know it will be pretty unrealistic to only accept mappers after some vetting of their mapping history (who is going to do that?). "Our" map is getting better every day. Really unmapped areas which pose a mapping challenge are hard to find. Otherwise a solution to the selection problem would be something like "set up a private project there, invite possible candidates, tell each one to map 5 tasks, see what they come up with".

I just want to add this angle because I think we have to determine our skill requirements first before diving into how to measure that precisely.

[frozenrabi]

Better late than never. A very much-needed initiation to distinguish and rank distinguished mapper who had contributed their heart and soul in mapping/validation. My friend who just created an OSM account achieved Advanced Mapper in few hours. So this was a necessary step.
I propose the following requirements for this

• Buildings >100k
• Highway+Waterway > 10k Km
• 50+ Changesets discussion
• OSM Contribution for at least 30 days in the previous year
• Tasks Mapped and Validated: 1k tiles each
• Mapped in at least not less than 25 countries.

And being this a Supper Mapper status in HOT TM, I also propose not to tally the aforementioned stats based on contribution using HOT TM as HOTOSM is just a tool and there are many Super Mappers who silently contribute their edits using other editors independently. This will be unfair to the OSM Community members. And as this is just a concept, I also request the team to rethink about this to rename this as Level 1-10 mapper categorizing them on the basis of predefined stats instead of defining someone as Super/Casual/Beginner mapper. It is already ranked in various OSM sites including HDYC. This is just my idea, let's hear from other fellow mappers/contributors as well.

[emi420]

Tech & Design Draft Proposal

Hi everyone, I've been working on an idea, trying to make this feature in a way that is dynamic and cover the different use cases, from validators, project managers and experience mappers to beginners, including users of HOT's Tasking Manager but also users of other Tasking Manager instances.

(All designs are just mockups for expressing the idea)

Achievements

I think we should create a single concept called achievements and put everything inside it. This way, the "beginner", "intermediate" and "advanced" mapper levels would be also achievements. The users that gained this status will keep it. Then, we'll also add a new achievement called "Super mapper".

The text displayed under the user's name (currently the mapper's level) could be optionally changed by the user. If no text is defined by the user, the achievement with the higher value would be displayed as default (ex: beginner, intermediate, advanced or super mapper)

All these achievements , or recognition, should be configurable from the Tasking Manager admin, defining what are the requirements for each one (ex: number of ChangeSets, buildings created, time from first mapping, number of projects, etc).

Some achievements could require also a peer review and approval (one or more). In those cases, the user has to reach certain metrics but the achievement will not be assigned automatically, but an alert will be sent to a list of selected users (ex: validators, admins or users with certain achievement like Super Mapper) and one (or more) of those users must approve the achievement.

Finally, the achievements can be used for mapping permissions on projects.

Future changes

The achievements requirements can change, but once an user receives a recognition of an achievement, this will remain and not be changed.

Metrics available can also change (we can add new sources of data), this system will enable changes without losing assigned recognition, but the requirements can be adjusted as needed.

[emi420]

👍👍

Is not a final design at all but mockups to express the idea. It could be possible but I'm not sure if we need to have progress for each achievement, the idea is to display only the badges or achievements that are earned by users. Also, the requirements for achievements can change, so having a "progress" indicator could add more complexity.

[SColchester]

Nice this is looking good @emi420 - would only challenge the ability to edit the text shown below usernames, because it stating clearly the mapper level is super useful and if it can be edited it might lead to an erosion of trust in the mapper level being stated for someone

[SColchester]

@emi420 I would also definitely keep 'Super mapper' to be understood as a 'mapper level', effectively the fourth mapper level beyond Advanced - as mapper levels currently work in TM. I'm not sure if I'm reading the above wrong and that is already mentioned - but important to stress.

[emi420]

@SColchester the idea is to be more flexible and have more options for the future of HOT's TM and other instances, but at the same time to not make big changes on the current systems that works.

The proposal of using "achievements" is that they can be used in the same way as mapper level but adding the possibility of customization so we can also have badges in the same system. This is also because there is a lot of discussion about how to use the current system of "mapping levels" and how the new level will be assigned, as this level will be based not only on quantitative metrics but also on qualitative indicators that needs human intervention.

In summary, current levels will be maintained, a new level will be added (super-mapper), and also a badge system, but in the back everything will be moved to a single system called "achievements", with different options of customization and usage.

[SColchester]

Hi @emi420 yes think we are aligned here in concept terms, I've created a quick visual to show how we would use a proposed sysyem like this to implement the super mapper level (and to check that I've understood things). On terminology, i'm not 100% on using 'Achievements', thinking more along the lines of 'Recognition' or 'Expertise', but will give this a deeper think.

[emi420]

As the Super Mapper feature has the higher priority, I would suggest to start defining a list of quantitative metrics that are interesting for identifying candidates to this new status.

[SColchester]

Use the 'created' numbers for each user (from ohsomeNow Stats, i.e. ignore the 'mapped' stats)

• Created buildings
• KM created highways
• KM created waterways

[martien-176]

I gathered a few quotes from this Github and the Slack discussion that i want to comment on.

• "Ensure higher data quality for complex projects" -> i wonder, what are complex projects in this respect? Urgent projects with many to-be-mapped features and a deadline? Or projects containing complex objects like route or waterway relations?

• "mappers who are contributing to the HOT Tasking Manager projects" -> agree with this

• "They can be using JOSM, iD or Potlatch" -> i agree partially. Using JOSM will imo better ensure quality and speed of mapping

• "100,000 buildings, 1000Km highway and 1000 Km (or 500 Km) waterway" -> i would favor splitting these up in separate super mapper categories. That makes vetting by peers less complicated and will ensure the recruitment of new master mappers be better guaranteed

• "The Super Mapper pool should become larger than the HOT Global Validators team" -> if requirements are too strict this will, i fear, be hard to achieve

• "peer-assessment process be designed to ensure fairness and transparency" -> having experience with vetting candidate-validators myself i know this could be a complex process. Keeping the requirements as basic as possible could mitigate this. After all, the vetting has to be done by normal mappers with possibly no knowledge of eg OSMCha or the TM API (with possible danger: who will do that after all; maybe nobody?).
  Therefor there should be a strict and clear procedure for peer assessment. Eg a form with questionnaire listing the requirements. The vetter(s) could fill in the form, the form could guide the vetting process. Also thus the vetting process will be documented.
  And imo there should be at least 2 vetters. This to prevent "favoritism".