Support multiple field lookups based on year

[ctmay4]

From @schusslern

So right now there are 4 grade fields defined as SSDIs
(Grade Clin, Grade Path, Grade Post Tx Clin, Grade Post Tx Path)

They are discussing changing the lookups in 2025 and adding a new grade (Grade 2025).

They will then have:

	2018-2024 lookup	2025+ lookup
Grade Clin	All cases, all standard setters	CoC (SEER RC?), AJCC ID != XX
Grade Path	All cases, all standard setters	CoC (SEER RC?), AJCC ID != XX
Grade Post	Tx Clin All cases, all standard setters	CoC (SEER RC?), AJCC ID != XX
Grade Post	Tx Path All cases, all standard setters	CoC (SEER RC?), AJCC ID != XX
Grade 2025	Not collected	All cases, all standard setters

So while for any given field I can say any one of these things:

• Grade Clin Year start = 2018, Year end = 2024, Standard setters = SEER, CoC, NPCR, Canada
• Grade Clin Year start = 2025, Year end = , Standard setters = SEER, CoC, NPCR, Canada
• Grade Clin Year start = 2025, Year end = , Standard setters = CoC, SEER (RC)

What I can’t do currently is

• Grade Clin Year start = 2018, Year end = 2024, Standard setters = SEER, CoC, NPCR, Canada with 2018-2024 lookup
• Grade Clin Year start = 2025, Year end = , Standard setters = CoC, SEER (RC) with 2025+ lookup

Because these are the same field.

I don’t want to make an entirely new EOD library (Where 2018-2024 is in EOD 2018; and the 2025+ is in EOD 2018b) and I don’t want to make a second set of schemas for all schemas so these 4 fields can have 2 different lookups.

As you can see, we have a year before we need to be able to do this, if they do in fact choose to NOT convert the old values into the new coding system.

I think they should convert it, personally, but I’m not sure I’ll be able to convince them. So I want to think about the best way to do this now so if we need to make changes to the Helios, API, etc; we have a plan.

[ctmay4]

As a frame of reference, here is what grade_clin looks like now. Not that we do support start and end years for metadata entities.

{
  "key": "grade_clin",
  "name": "Grade Clinical",
  "naaccr_item": 3843,
  "naaccr_xml_id": "gradeClinical",
  "default": "9",
  "table": "grade_clinical_standard_non_ajcc_32473",
  "used_for_staging": false,
  "metadata": [
    { "name": "COC_REQUIRED" },
    { "name": "NPCR_REQUIRED" },
    { "name": "SSDI" },
    { "name": "CCCR_REQUIRED" },
    { "name": "SEER_REQUIRED" }
  ]
}

It makes a big difference whether any of these fields that will require multiple tables based on year are ever used in the staging process. This one is an SSDI that is not used in staging.

If we can assume that the fields are only non-staging fields, we could add a table field to the metadata entity. We would supply a "default" lookup in the existing table field. It can be overridden in the new metadata table field.

{
  "key": "grade_clin",
  "name": "Grade Clinical",
  "naaccr_item": 3843,
  "naaccr_xml_id": "gradeClinical",
  "default": "9",
  "table": "grade_clinical_standard_lookup_2018_2024",
  "used_for_staging": false,
  "metadata": [
    { 
      "name": "COC_REQUIRED",
      "start": 2018,
      "end": 2024 
    },
    { 
      "name": "COC_REQUIRED",
      "start": 2025,
      "table": "grade_clinical_standard_lookup_2025"
    },
    { "name": "NPCR_REQUIRED" },
    { "name": "SSDI" },
    { "name": "CCCR_REQUIRED" },
    { 
      "name": "SEER_REQUIRED",
      "start": 2018,
      "end": 2024 
    },
    { 
      "name": "SEER_REQUIRED",
      "start": 2025,
      "table": "grade_clinical_standard_lookup_2025"
    }    
  ]
}

@schusslern thoughts?

[schusslern]

yes, grade is sometimes used in staging.
there is a table that summarizes to pick the best 'grade' value - that would continue to be used for 2018-2024. in 2025, we would just use the Grade 2025 by itself. i can control that using ydx in the algorithm.

The summary table uses the superset of grade values, and since they aren't adding new potential values, it should not have any issues with the possible values vs values in the calculation table.

regarding your potential solution, i could live with that. will that mean we would have to adjust all the existing meta data?

since they are still voting on what to do, i wouldn't do anything right now, but if you could give me an idea if this will be time consuming?

[ctmay4]

The good thing about this solution is that it would only require a single change to the JSON structure. I would add getTable() to the Metadata interface. Metadata is not used in the calculation at this point (and hopefully never will) so nothing else would need to change in the library. If you say that you can handle the differences in the tables during calculation by using DX year, then I do think this is the solution that will have the least effort in the library. Sound good?

Of course applications, like SEER*DMS will need to be adjusted to show the correct lookup to the users. However I do not think that will be a lot of work.

Helios would need to support the new field as well but I assume that would not be much work.

[schusslern]

that sounds like a good working theory. i'll let you know once they start making some decisions.