IAMC-to-CMIP7: emissions mapping and harmonization for SSP/ScenarioMIP

[jkikstra]

For the ScenarioMIP process, many new variables are being added.
Now the question is, how to map these emissions for harmonization?

There are a bunch of things here to take into account.
Here, I list the ones I am now thinking of.
More can be added to this description later. (last updated: 26.09.2024)

Note: for this repository, only the "mapping" tables are important, such that we have a complete & clear list.
However, for lack of a better place, and to try to keep the discussion in one place, I provide more background information and more on current issues as well.

Introduction to Harmonization processes

There are two harmonization processes:

1. harmonizing for input for simple climate models (SCMs). This is global only. This is operational in the way it was done for IPCC AR6: iiasa/climate-assessment. This is used for IAMs to get a quick estimate of temperatures, in a methodology that aims to get the highest comparability between scenarios.
2. harmonizing for input for Earth System Models (ESMs). (a) This process doesn't only harmonize pathways, but also downscales the emissions to a grid. In the previous round (CMIP6), this was done using iiasa/aneris and iiasa/emissions_downscaling (see Gidden et al. 2019). This time, we are (very likely) going to use iamconsortium/concordia, a tool developed in the RESCUE project by @coroa and @gidden. (b) In addition, atmospheric concentrations should be provided.

I will focus here on part 2., because it is not yet operational.
However, I will also write bits about part 1., where changes may be required, too.

Current timeline

• 2024
  • October-December:
    • updating historical emissions; https://github.com/iiasa/emissions_harmonization_historical (/updates by Steve Smith)
    • introducing IAM teams to concordia workflow (regional, part 2)
    • settling on v1.0 for variable mapping (CEDSplus-IAM-concordia)
    • experimenting with harmonization methods (global, part 1)
  • October 25: submit emissions variable needs to template for SSP/ScenarioMIP reporting.
  • October 28-31: CMIP7 Forcings Task Team meeting in Reading
  • October-November: running sensitivity experiments in FaIR and MAGICC
  • End of November: running some harmonization (global, part 1) experiments
  • November 14: reporting fixes for v1.0 (becomes v1.1) of IAM scenarios
  • Mid-December: first round of default harmonization pathways (based on Nov14 reporting)
  • December: new round of IAM scenarios (v2.0) - with immediate AR6 climate assessment
• 2025
  • Jan 14 2025: updated climate-assessment (with FaIR and MAGICC updates, historical emissions updates, harmonization updates, infilling updates) ready
    • Soon after Jan 14: updated climate assessment (can be compared to previous climate assessment)
  • End of Jan / begin Feb: in-person ScenarioMIP meeting
  • March-May 2025: final round of IAM scenarios (v3.0)
  • June 2025: final harmonization (part 2) & updated emulator workflow (part 1)

Throughout: (bi)weekly meetings on especially part 2, once the workflow is operational.

General issues

Additionality

Unless otherwise states, there should not be any overlap between emissions categories. E.g. if there is International Shipping and Transport, than Transport should not include emissions from International Shipping.

Completeness

Unless otherwise stated, summing up all variables should account for the total emissions. No emissions should be 'missed'.

Land-use emissions reporting. (IAM v Inventories)

There's the wish to have two separate reportings: (i) one following standard previous IAM practice, and (ii) one aligned with national inventories.

Two outstanding questions here:

1. Can all models provide the information for this separation? Or would there need to be some post-processing work (like Gidden et al. 2023)
2. Does the template allow for reporting the information for this separation, for models to report it natively? See also another issue: Allowing emissions reporting both for IAM definitions and National Inventories definitions (CO2|AFOLU) #146

Historical emissions data sources

Harmonization happens against some historical emissions dataset. This dictates, to a large extent, what kind of harmonization is possible.

1. For AR6, the climate-assessment pipeline used RCMIP emissions (Nicholls et al.; for documentation see rcmip.org ).
2. For CMIP7, we will use a combination of CEDS (for most emissions) + GFED (which adds all necessary burning emissions, and replaces the burning where covered in CEDS).

Current questions about CMIP7 harmonization database:

1. Where are the scripts for processing original data sources?
   i. currently, work-in-progress is here: https://github.com/iiasa/emissions_harmonization_historical
2. What are the latest updates that can be incorporated at different points in the CMIP7 timeline?
   i. tbd.
3. What extensions can/should be done for harmonization?
   i. tbd. (until 2025, or not?)

Timing of harmonization

It is current practice to harmonize to a specific year.

1. For AR6, the climate-assessment workflow harmonizes to 2015.
2. For CMIP7, we will likely use 2023 (but still some discussion to be had here, regarding using extensions e.g. until 2025, the starting point of scenarios)

Harmonization methods / rules

The harmonization rules applied affect future pathways, and thus affects how closely an emissions pathway reflects the original scenario.

1. For AR6, the applied harmonization rules are documented in Kikstra et al. 2022, Table 1
2. For CMIP7, the harmonization rules are still to be determined. And IAM teams will likely want to make their own choices here.

Part 2: harmonization for ESMs

Current variable mapping (incomplete, to be improved upon)

My starting point is a description of the variables used in the RESCUE project was provided by @coroa, here:

This includes, for each expected emissions species:

• expected sectors
• expected level of reporting (global or regional)

There is also a used mapping of CEDS variables onto these RESCUE variables:

What comprises these sectors can also be seen in the attached CEDS mapping (ceds_mapping.xlsx). The * Burning variables are as defined in GFED.

CO2

Harmonization Sector (CEDS)	IAMC variable
Deforestation and other LUC (only positive emissions; global)	Emissions|CO2|AFOLU|Positive
Agriculture	Emissions|CO2|AFOLU|Agriculture     OR Emissions|CO2|AFOLU
Energy Sector	Emissions|CO2|Energy - Residential Commercial Other - International Shipping - Transportation Sector
Industrial Sector	Emissions|CO2|Industrial Processes
Residential Commercial Other	Emissions|CO2|Energy|Demand|Residential and Commercial and AFOFI
Solvents Production and Application	Emissions|CO2|Product Use|Solvents
Waste	Emissions|CO2|Waste
International Shipping (global)	Emissions|CO2|Energy|Demand|Transportation|Shipping|International
Aircraft (global)	Emissions|CO2|Energy|Demand|Transportation|Aviation
Transportation Sector	Emissions||Energy|Demand|Transportation|Road, Rail and Domestic Shipping + Emissions||Energy|Demand|Transportation|Other Sector     OR Emissions|*|Energy|Demand|Transportation - International Shipping- Aviation
OAE Calcination Emissions (only negative emissions)	Carbon Removal|Ocean Alkalinity Enhancement
CDR Afforestation (only negative emissions)	Carbon Removal|Forestry
CDR BECCS (only negative emissions)	Carbon Removal|Bioenergy with CCS
CDR DACCS (only negative emissions)	Carbon Removal|Direct Air Capture with CCS
CDR EW (only negative emissions)	Carbon Removal|Enhanced Weathering
CDR Industry (only negative emissions)	Carbon Removal|Durable Wood Products + Carbon Removal|Mineral Products
CDR OAE Uptake Ocean (only negative emissions)	Carbon Removal|Ocean Alkalinity Enhancement

Known issues:

• at the moment, Emissions|CO2|AFOLU|Positive is not yet part of the scenario template
• this mapping is unlikely to be complete (e.g. multiple new Carbon Removal variables in the template are missing)
• it is not clear how to adjust the mapping for IAM v National Inventories

BC, CH4, CO, NH3, N2O, NOx, OC, Sulfur, VOC

Harmonization Sector (CEDS+GFED)	IAMC variable
Agriculture	Emissions||AFOLU|Agriculture     OR Emissions||AFOLU - Agricultural Waste Burning - Forest Burning - Grassland Burning - Peat Burning
Energy Sector	Emissions|*|Energy - Residential Commercial Other  - International Shipping - Transportation Sector
Industrial Sector	Emissions|*|Industrial Processes
Residential Commercial Other	Emissions|*|Energy|Demand|Residential and Commercial and AFOFI
Solvents Production and Application	Emissions|*|Product Use|Solvents
Waste	Emissions|*|Waste
International Shipping (global)	Emissions|*|Energy|Demand|Transportation|Shipping|International
Aircraft (global; incl. Domestic and international)	Emissions|*|Energy|Demand|Transportation|Aviation
Transportation Sector	Emissions||Energy|Demand|Transportation|Road, Rail and Domestic Shipping + Emissions||Energy|Demand|Transportation|Other Sector     OR Emissions|*|Energy|Demand|Transportation - International Shipping – Aviation
Agricultural Waste Burning	Emissions|*|AFOLU|Biomass Burning
Forest Burning	Emissions|*|AFOLU|Land|Forest Burning
Grassland Burning	Emissions|*|AFOLU|Land|Grassland Burning
Peat Burning	Emissions|*|AFOLU|Land|Wetlands

Known issues:

• Not entirely clear where peatland burning is to be classified under the current template ( Allowing emissions reporting both for IAM definitions and National Inventories definitions (CO2|AFOLU) #146 )

HFC (total), C2F6, CF4, SF6

Harmonization Sector (CEDS+GFED)	IAMC variable
(total)	Emissions|*

Known issues:

• HFC currently is reported as CO2-eq, but it is needs to be checked which GWP100 version to use (aligning IAMs with CEDS would be good)
• Unclear to me what historical emissions dataset will be used for this.

Part 1: harmonization for SCMs

Current variable mapping (incomplete, to be improved upon)

No mapping is required, it works on IAMC variables. All the required variables are currently included in the template.
The workflow takes in a list of variables, documented here.

-> TODO: add table for climate emulator pipeline

Should 1. be updated to be as close as possible to 2.?

If method 1. follows all updates that are in method 2., you can get pretty close.
So one can think that then method 1. would be the way to provide concentrations, too.
However, pretty close, is not exactly the same, which is what we need.

If we want to allow IAM modelling teams to make their own, separate, harmonization decisions, on a regional level, it is not technically possible (or at least, lots of potentially unnecessary work) to reproduce this in the global-level-only climate-assessment workflow.
Therefore, I assume those responsible for the concentrations (@znicholls) would probably make just take the emissions as harmonized following 2., and feed that into an SCM, outside of climate-assessment.

At the same time, the IAM teams need quick feedback on the climate outcomes of their emissions pathways, in a way that is as close as possible to what we expect to come out from the final climate assessment/concentrations.
This means that climate-assessment would still need to be updated, for: (1) historical emissions, (2) climate model (FAIR/MAGICC/...) calibration, (3) harmonization methods.

---

People: @gidden @znicholls @coroa @phackstock (and others from IAM modelling teams will need to be included in this discussion too, with e.g. Nico Bauer already having brought this up in the last ScenarioMIP meeting).

[znicholls]

Thanks @jkikstra, super nice summary.

Therefore, I assume those responsible for the concentrations (@znicholls) would probably make just take the emissions as harmonized following 2., and feed that into an SCM, outside of climate-assessment.

Yep this is spot on. For the final CMIP7 ScenarioMIP, the SCMs (MAGICC, FaIR etc.) will only use the results based on the full regional, sectoral harmonisation (i.e. 2.). (It is this full regional, sectoral harmonisation which sits underneath the RCMIP emissions set, thanks again to @gidden for the years pulling that together took out of his life.)

At the same time, the IAM teams need quick feedback on the climate outcomes of their emissions pathways, in a way that is as close as possible to what we expect to come out from the final climate assessment/concentrations

Yes, definitely a challenge! My feeling is this: the AR6 workflow will put us in the right ballpark so groups can be ~ +/- 0.15C to whatever target they're trying to hit. If the IAM teams want to end up closer than that, say +/- 0.03C, once all is said and done, then we're going to need to work out our process and timeline a bit more precisely.

I think the steps are basically:

1. freeze historical emissions
   • this is trivial for historical emissions up to say, 2022. For 2022 to 2025, it's much trickier because we don't have historical data yet and the historical data we do have is more likely to be revised!
2. do initial harmonisation implementation
   • required for next step
3. freeze the SCMs (MAGICC/FaIR/CICERO/...)
   • in an ideal world, we'd re-calibrate the emulators to the updated historical emissions. If we don't have time for that step, I think it's still ok. The emulators are always 'wrong' in hindsight, because they don't perfectly match the (not yet run) CMIP models (as the CMIP models remind us every cycle), so I'd be ok running with something that isn't perfectly 100% consistent/re-calibrated through the entire stack in order to make sure we have data available quickly and a stable tool for the IAM teams to work with.
4. IAM teams iterate on their scenarios and harmonisation, using the frozen SCMs
   • IAM teams can obviously tweak their emissions until they get as close to whatever target as they want
   • I guess they could also request tweaks to the harmonisation (like in CMIP6?)
5. freeze harmonisation and scenarios
   • I guess this requires sign off by all IAM teams at the regional, sectoral level

In terms of timeline:

1. freeze historical emissions - now for emissions up to 2022, no idea for picking the number for 2023/2025
2. do initial harmonisation implementation - already done thanks to RESCUE?
3. freeze the SCMs (MAGICC/FaIR/CICERO/...) - MAGICC could have something frozen by Feb 1 2025 (high confidence)
4. IAM teams iterate on their scenarios and harmonisation, using the frozen SCMs - I don't know what the envisaged timeline for this is
5. freeze harmonisation and scenarios - I don't know what the envisaged timeline for this is. As I understand it, to have a chance of meeting GST timelines, we'd ideally be freezing these in ~March-May 2025 (assuming that the downstream processing is turnkey so they can be processed and then put into the ESMs in a matter of weeks after freezing)

[jkikstra]

Thanks Zeb. I will incorporate these points above and update later.

Yes, definitely a challenge! My feeling is this: the AR6 workflow will put us in the right ballpark so groups can be ~ +/- 0.15C to whatever target they're trying to hit. If the IAM teams want to end up closer than that, say +/- 0.03C, once all is said and done, then we're going to need to work out our process and timeline a bit more precisely.

Tomorrow, we have the climate & harmonization discussion in the ScenarioMIP in-person meeting.
I will share this with them.
My feeling is that 0.15K is very much unacceptable for the community towards the next round of submissions; therefore asking if a recent FaIR calibration can be used. I will give an overview of the constraints.
But let's stay in touch.

[znicholls]

My feeling is that 0.15K is very much unacceptable for the community towards the next round of submissions; therefore asking if a recent FaIR calibration can be used. I will give an overview of the constraints.

Ok. I would say the community is still not really understanding the limits of climate projections then (and what is going to happen once they're ultimately sent through ESMs anyway). A simple TCRE calculation puts you in the +/- 0.2C ballpark already, so modelling groups shouldn't have that many surprises and, any surprises that are there should all be basically related to non-CO2 stuff.

Nonetheless, if you can tell me when the ideal deadline for a frozen calibration for CMIP7 scenario runs would be and why, I can see what is possible from the MAGICC side.

[znicholls]

towards the next round of submissions

When is this planned for?

[znicholls]

therefore asking if a recent FaIR calibration can be used

Just as a note, this may not be any closer to the final CMIP7 calibration than the AR6 setup (depending on what new science we have to include in the projections, e.g. updated wetland methane emissions parameterisations).

[jkikstra]

Hey @znicholls, didn't mean to rock the boat / startle you.
Just relaying first impressions - but nothing decided yet.

I just spoke also to a few more people, some also think sticking with AR6 setup is anyway fine.

towards the next round of submissions

When is this planned for?

December

I would say the community is still not really understanding the limits of climate projections then

From the IAM side, it's about designing the pathways in such a way that they are as much as possible in line with a "best science best estimate". 0.15K is a lot, for best estimate differences. It makes a big difference, for the Very Low scenarios, on how much/quick you can/need to go down in the first decade(s) to limit peak temperatures enough.

Therefore, having an update before the December submissions would be very helpful, such that the IAMs can make some 'best guess' tweaks still after that submission still (towards the final submission), where they need to align more with the scenario design.

But if the best estimate is that it's not 'closer' to the final calibration later, then it would of course not be worth the work!

updated wetland methane emissions parameterisations

~ +/- 0.15C

They would like to know why you estimate this at 0.15; what are the updates in climate science that you expect could lead to the major part of this potential change?

[znicholls]

December

Tight, but good to know there is some time to do some thinking.

But if the best estimate is that it's not 'closer' to the final calibration later, then it would of course not be worth the work!

Yep, and this is the bit that's impossible to predict. We've got to pick which best-estimate we want to match. I think there's lots of good arguments for matching AR6 best-estimate, rather than whatever we have in December, but let's see what we learn over the coming months before deciding. I guess the key point is this: it's tricky and there's no obvious/objective answer.

They would like to know why you estimate this at 0.15; what are the updates in climate science that you expect could lead to the major part of this potential change?

Expert judgement :) In more detail, it's knowing the rough magnitude of what ESMs might serve up as they improve their carbon cycles, how much uncertainty there is around the best-estimate for non-CO2 stuff (particularly methane and aerosols), seeing the difference between MAGICC and FaIR (which could both be argued to be 'best-estimate', but differ by roughly this much in some cases), the uncertainty in observational timeseries. The change could be zero of course, but if we have a change bigger than ~0.15C, I would go looking for a bug in my code first. If the change is smaller than this, then I would not be surprised if we can explain it all due to science updates. Chris might have more thoughts on what, if any, new science has come out since AR6 and what impact this would have, which would give a more evidence based estimate.

[znicholls]

Alrighty, my notes from discussion today. @jkikstra may add his own too:

Understanding: we're targeting a scenario team meeting at end of Jan. We want to have a frozen emulator set by then to allow IAM teams to then iterate on their scenarios with a known target.

A suggestion: don't think of harmonisation as something you do at the end. Think of it like you think of climate emulators. i.e. if you think climate emulators should be frozen by mid-Jan, then you also want harmonisation frozen by mid-Jan. Reasoning: these two elements can be equally painful for getting a 1.5C scenario to be 1.5C +/- 0.03C and not 1.5C +/- 0.10C. Infilling should also be included in this basket, i.e. think of climate emulators, harmonisation and infilling altogether, not as separate steps. (I am 100% certain that, if you freeze your scenarios, then only start fiddling with harmonisation in May, a massive headache and delays in delivery will be the result.)

A further suggestion: to do this well, numerous iterations will be required. Get teams prepared for the idea that they should be talking regularly and iterating as fast as possible, rather than meeting once every two months, and not speaking in the meantime (that'll only give you ~4 iterations between now and final scenarios, which probably won't be enough). The more times we can get scenarios to run through the entire machinery, and be checked by multiple people, the better the outputs will be.

Figuring out sensitivity to historical emissions

Want to do 3 experiments:

1. AR6 emulators with AR6 historical emissions (already done, technically, but good to double check our workflows)
2. AR6 emulators with updated historical emissions (tells us whether there is a tight or loose coupling between emulator calibration and historical emissions)
3. Updated emulators with updated historical emissions (tells us the extent to which updated calibrations propagate changes into the future)

Steps:

1. Chris rolls his 'fast-track' RCMIP emissions compilation stuff using the latest version of CEDS (using PRIMAP trends to extend CH4 back in time)
2. Zeb does 1) and 2) above using MAGICC
3. Chris does 1), 2) and 3) above using FaIR
4. Jump back on zoom to discuss results

Expected timeline: aiming to have this all done by mid-November

Expected science updates for CMIP7

Chris is already involved in indicators, so, as far as we know, the only major update is this wetland methane stuff, and even that only has limited lines of evidence so far.

So, the summary is probably, our known knowns are probably only going to give small revisions compared to AR6. Those known knowns will push warming up, but in low emissions scenarios probably only marginally (hundredths of a degree, not tenths). Known unknowns remain at +/-0.15C, we'll have to just live with that.

Steps:

1. Zeb (via Trevor) to do runs to quantify impact of wetland methane emissions on scenarios (impact is higher for high warming scenarios, could be very small for lower scenarios i.e. the ~1.5C scenarios)
2. Update summary above

Expected timeline: aiming to have this all done by mid-November

Processing historical emissions

1. Jarmo will set up a repository for us to work on using https://gitlab.com/znicholls/copier-basic-python-repository as the template
   • we'll use poetry for our environment management
   • use jupytext to avoid notebook-git fun
2. convention: notebooks get run in order. Give notebooks a four digit number at the start so we can separate different 'streams' if we need to
   • this convention wouldn't scale, but for something as simple as this it will be fine

[jkikstra]

Looks good to me, @znicholls.
I spoke to Keywan about the following timeline (now also added to the first post). He thinks that this would work really well:

Current timeline

* 2024
  
  * October-December:
    
    * updating historical emissions
    * introducing IAM teams to concordia workflow (regional, part 2)
    * settling on v1.0 for variable mapping (CEDSplus-IAM-concordia)
    * experimenting with harmonization methods (global, part 1)
  * October 28-31: CMIP7 Forcings Task Team meeting in Reading
  * October-November: running sensitivity experiments in FaIR and MAGICC
  * November 14: reporting fixes for v1.0 (becomes v1.1) of IAM scenarios
  * December: new round of IAM scenarios (v2.0) - with immediate AR6 climate assessment

* 2025
  
  * Jan 14 2025: updated climate-assessment (with FaIR and MAGICC updates, historical emissions updates, harmonization updates, infilling updates) ready
  * Soon after Jan 14: updated climate assessment (can be compared to previous climate assessment)
  * March-May 2025: new round of IAM scenarios (v2.0)
  * June 2025: final harmonization (part 2) & updated emulator workflow (part 1)

Throughout: (bi)weekly meetings on especially part 2, once the workflow is operational.

[znicholls]

Super, thanks. Looks really good to me too!