Core configuration

CLI 

Infilling database 

climate_assessment.cli.create_infiller_database(*args: t.Any, **kwargs: t.Any) → t.Any

Creates infiller database by harmonizing data in input_emissions_file which is followed by some common-sense vetting and then saved in outdir

input_emissions_file should be a path to a file of emissions

outdir should be a path a directory which already exists

Infilling 

climate_assessment.cli.infill(*args: t.Any, **kwargs: t.Any) → t.Any

Infill harmonized data in input_emissions_file, saving output in outdir

input_emissions_file should be a path to a file of harmonized emissions

outdir should be a path a directory which already exists

Harmonisation 

climate_assessment.cli.harmonize(*args: t.Any, **kwargs: t.Any) → t.Any

Harmonise data in input_emissions_file, saving output in outdir

input_emissions_file should be a path to a file of emissions

outdir should be a path a directory which already exists

Harmonisation and infilling 

climate_assessment.cli._harmonize_and_infill(input_df, inputcheck, key_string, outdir, infilling_database, harmonize, prefix, harmonization_instance)

Thin wrapper function for running both harmonization and infilling.

Returns True if there are scenarios that can be run by a climate emulator. Those scenarios are not returned by this function, but rather written to the outdir. Returns False if there are no complete scenarios to be run.

For more information, see the code description under climate_assessment.harmonization_and_infilling.harmonization_and_infilling().

climate_assessment.cli.harmonize_and_infill(*args: t.Any, **kwargs: t.Any) → t.Any

Harmonise and infill data in input_emissions_file, saving output in outdir

input_emissions_file should be a path to a file of emissions

outdir should be a path a directory which already exists

climate_assessment.cli.harmonization_and_infilling(df, key_string, infilling_database, prefix='AR6 climate diagnostics', instance='ar6', outdir='output', do_harmonization=True)

Parameters:

input_df (pyam.IamDataFrame) – Input native emissions to harmonize and infill.
inputcheck (bool) – Perform checks to remove unsuitable emissisons pathways. [Default: True]
key_string (str) – Identifier string for writing out results. By default derived from the input_emissions_file string in a CLI command.
outdir (str) – Path to output folder.
infilling_database (str) – Path to the infiller database emissions file.
harmonize (bool) – Perform harmonization (with False, only infilling is run). [Default: True]
prefix (str) – Prefix used to identify the new variable names of results produced by the workflow.
harmonization_instance (str) – Config string required by aneris.

Returns:

Returns True if there are scenarios that can be run by a climate emulator. Those scenarios are not returned by this function, but rather written to the outdir. Returns False if there are no complete scenarios to be run.

Return type:

bool

Workflow 

climate_assessment.cli.workflow(*args: t.Any, **kwargs: t.Any) → t.Any

Run the full IPCC AR6 climate asessment workflow.

Example usage: python scripts/run_workflow.py tests/test-data/ex2.csv output --model "FaIR" --model-version "1.6.2" --num-cfgs 2237 --probabilistic-file tests/test-data/fair-1.6.2-wg3-params-slim.json --fair-extra-config tests/test-data/fair-1.6.2-wg3-params-common.json --infilling-database src/climate_assessment/infilling/cmip6-ssps-workflow-emissions.csv

See further documentation at Description of the workflow.

Climate 

climate_assessment.cli.clim_cli(*args: t.Any, **kwargs: t.Any) → t.Any

Run the climate emulator step of the IPCC AR6 climate asessment workflow.

Example usage: python scripts/run_clim.py tests/test-data/workflow-fair/ex2_harmonized_infilled.csv output --model "fair" --model-version "1.6.2" --num-cfgs 2237 --probabilistic-file tests/test-data/fair-1.6.2-wg3-params-slim.json --fair-extra-config tests/test-data/fair-1.6.2-wg3-params-common.json --scenario-batch-size 4

For more information, see the code description under climate_assessment.climate.climate_assessment().

Postprocess 

climate_assessment.cli._postprocess_worker(fname, outdir, **kwargs): Helper function which takes a file that ends with “_rawoutput.xlsx” and the output location, and then calls the function do_postprocess(output, outdir, key_string, prefix).

climate_assessment.cli.postprocess(*args: t.Any, **kwargs: t.Any) → t.Any

Merge and postprocess a collection of rawoutput into a single set of output files

This script should be run after run_workflow.py if –no-postprocess was supplied. This is particularly useful for a clusted run where a run_workflow is run multiple times with batches of scenario data.

Each file is processed individually, but the metadata from all the runs is merged into a single output file.

Infilling 

climate_assessment.infilling.run_infilling(harmonised_df, prefix, database_filepath=None, start_year=2015, end_year=2100)

Run infilling

Parameters:

harmonised_df (pyam.IamDataFrame) – Input harmonised emissions to infill
prefix (str) – Prefix used for the variable names
database_filepath (str) – Path to the file which contains the infilling database
start_year (int) – First year which should be reported in output
end_ear (int) – Last year which should be reported in output

Returns:

Infilled emissions. Output also includes the input emissions. The output is interpolated onto an annual timestep.

Return type:

pyam.IamDataFrame

climate_assessment.infilling._infill_variables(cruncher, variables, to_infill, db, lead, output_timesteps, old_prefix)

Start core infilling run, using silicone.multiple_infillers.infill_all_required_variables

Parameters:

cruncher (str) – Infilling method
variables (list) – List of required variables to infill
to_infill (pyam.IamDataFrame) – Emissions data to be infilled
db (pyam.IamDataFrame) – The infilling database
lead (str) – Lead infiller variable
output_timesteps (list) – List of required output timesteps
old_prefix (str) – The prefix of harmonized emissions, which will be replaced by a prefix for the infilled emissions variables.

Returns:

Infilled emissions.

Return type:

pyam.IamDataFrame

climate_assessment.infilling._add_to_infilled(infilled, infilled_variables)

Helper function which takes two dataframes, and binds them together.

Parameters:

infilled (pyam.IamDataFrame) – An IamDataFrame.
infilled_variables (str, None) – An IamDataFrame.

Returns:

An IamDataFrame.

Return type:

pyam.IamDataFrame

climate_assessment.infilling.load_csv_or_xlsx_for_one_region(file, sheet)

This loads data from either a csv or an xls(x) file and checks that it has only a single region.

Parameters:

file (str, None) – The (relative or absolute) file address of the file to load
sheet (str, None) – If file is a csv, use None. Otherwise, the sheet name to read from the file.

Returns:

The information in the file.

Return type:

pyam.IamDataFrame

climate_assessment.infilling.postprocess_infilled_for_climate(df_infilled, prefix, start_year=2015)

Helper function that takes a set of infilled emissions data, adds a provided prefix, and filters out the scenarios that are appropriately infilled by checking both for a minimum set of years of data and checking that there are no non-co2 negative emissions caused by the infilling method.

Parameters:

df_infilled (pyam.IamDataFrame) – Infilled data.
prefix (str) – Prefix used for the variable names

Returns:

Checked and reformatted infilled data.

Return type:

pyam.IamDataFrame

Harmonization 

climate_assessment.harmonization.postprocessing(harmonized_results, prefix): Helper function that makes sure that the harmonization prefix is added.

climate_assessment.harmonization.add_year_historical_percentage_offset(df, dfhist, yr=2015, low_yr=2010): In the case that the chosen harmonization year (yr, default=2015) is not reported, this function uses an earlier year (low_yr, default=2010) to derive the historical offset for harmonization. It takes the relative difference in low_yr, and assumes that this relative difference stays the same until yr.

climate_assessment.harmonization.run_harmonization(df, instance, prefix)

Run harmonization. Hamronization method overrides by specific species are set within this function too.

Parameters:

df (pyam.IamDataFrame) – Input native emisisons in IAMC format.
instance (str) – String used to choose what historical data to use.
prefix (str) – Prefix used for the variable names

Returns:

Harmonized scenarios.

Return type:

pyam.IamDataFrame

Harmonisation and infilling 

climate_assessment.harmonization_and_infilling.harmonization_and_infilling(df, key_string, infilling_database, prefix='AR6 climate diagnostics', instance='ar6', outdir='output', do_harmonization=True)

Parameters:

input_df (pyam.IamDataFrame) – Input native emissions to harmonize and infill.
inputcheck (bool) – Perform checks to remove unsuitable emissisons pathways. [Default: True]
key_string (str) – Identifier string for writing out results. By default derived from the input_emissions_file string in a CLI command.
outdir (str) – Path to output folder.
infilling_database (str) – Path to the infiller database emissions file.
harmonize (bool) – Perform harmonization (with False, only infilling is run). [Default: True]
prefix (str) – Prefix used to identify the new variable names of results produced by the workflow.
harmonization_instance (str) – Config string required by aneris.

Returns:

Returns True if there are scenarios that can be run by a climate emulator. Those scenarios are not returned by this function, but rather written to the outdir. Returns False if there are no complete scenarios to be run.

Return type:

bool

Climate 

climate_assessment.climate.climate_assessment(df, key_string, outdir, model='magicc', model_version=None, num_cfgs=600, historical_warming=0.85, historical_warming_reference_period='1850-1900', historical_warming_evaluation_period='1995-2014', test_run=False, scenario_batch_size=20, save_raw_output=False, probabilistic_file='data/magicc/0fd0f62-derived-metrics-id-f023edb-drawnset.json', magicc_extra_config=None, fair_extra_config=None, co2_and_non_co2_warming=False, prefix='AR6 climate diagnostics')

Run the climate assessment

Parameters:

df (pyam.IamDataFrame) – pyam.IamDataFrame with data to assess
key_string (str) – String to use to identify output files and find input files
outdir (str) – Directory in which to save the output
model (str) – Reduced-complexity climate model to run assessment with
model_version (str, None) – Version of the climate model. If None, use default
num_cfgs (int) – Number of model configs to run. Multiply this number by the number of scenarios to get total runs. The full drawnset is 600 so if you’re just testing it’s probably worth using a smaller number unless you want to wait for 58 000 runs (of course by using a smaller subset the temperature numbers won’t make sense but if all you’re checking is that things run you also don’t really care).
historical_warming (float) – Historical warming to match the climate model output to
historical_warming_reference_period (str) – Reference period to use for the historical warming (e.g. “1850-1900”)
historical_warming_evaluation_period (str) – Evaluation period to use for the historical warming (e.g. “1995-2014”)
test_run (bool) – Is this a test run? If it is, we won’t raise an error if the historical temperatures don’t match the assessment perfectly.
scenario_batch_size (int) – How many scenarios do you want to run at once? This should be adjusted to balance the amount of RAM and number of cores available on your system. The limit depends mostly on the RAM available.
save_raw_output (bool) – Should we also save the raw climate output (i.e. every ensemble member) to disk?
probabilistic_file (str) –

Path to file containing parameters to run climate model with (will be passed to
openscm-runner). Currently, only json files are supported.
co2_and_non_co2_warming (bool) – Include assessment of CO2 and non-CO2 warming (requires 3x as many runs)?
prefix (str) – Prefix for all variable names

Returns:

pyam.IamDataFrame containing climate assessment

Return type:

pyam.IamDataFrame

climate_assessment.climate.run_and_post_process(scenarios, climate_models_cfgs, climate_models_out_config, historical_warming, historical_warming_reference_period, historical_warming_evaluation_period, outdir, test_run, save_raw_output, co2_and_non_co2_warming)

Run the climate models probabilistically

Uses openscm-runner to parallise the model runs. The results are then post processed to calculate exceedence probabilities

Parameters:

scenarios ([scmdata.ScmRun, pyam.IamDataFrame]) – Emissions for the scenarios of interest
climate_models_cfgs (dict) – Configuration as expected by openscm-runner
climate_models_out_config (dict) – Climate models output config as expected by OpenSCM-Runner
historical_warming (float) – Historical warming to match the climate model output to
historical_warming_reference_period (str) – Reference period to use for the historical warming (e.g. “1850-1900”)
historical_warming_evaluation_period (str) – Evaluation period to use for the historical warming (e.g. “1995-2014”)
outdir (str) – Output directory for the raw output
test_run (bool) – If true, check model output
save_raw_output (bool) – If True, save all the raw climate model output for later analysis.
co2_and_non_co2_warming (bool) – Include assessment of CO2 and non-CO2 warming?

climate_assessment.climate._get_model_configs_and_out_configs(model, model_version, probabilistic_file, magicc_extra_config, fair_extra_config, num_cfgs, co2_and_non_co2_warming)

Postprocess 

climate_assessment.climate.post_process.check_hist_warming_period(period)

Check period for historical warming calculations

Parameters:: period (str) – Input period, must be formatted as “YYYY-YYYY” where the first year is less than or equal to the last year. The last year is included in the period.
Returns:: All years in the period
Return type:: range
Raises:: ValueError – period is formatted incorrectly

climate_assessment.climate.post_process.calculate_exceedance_probability_timeseries(res, exceedance_probability_calculation_var, test_run=False, historical_warming=0.85, historical_warming_reference_period='1850-1900', historical_warming_evaluation_period='1995-2014')

Calculate the timeseries with which we should determine exceedance probabilities.

Please note that calculating the statistical properties like the exceedance probability is done on the count of the raw, full ensemble data. It cannot be derived from from the time series of the final statistical indicator afterwards.

climate_assessment.climate.post_process.calculate_co2_and_nonco2_warming_and_remove_extras(res): Calculate non-CO2 warming. Currently only implemented for MAGICC, which can be run twice to allow for this calculation, using (the CLI option) co2_and_non_co2_warming.

climate_assessment.climate.post_process.post_process(res, outdir, test_run=False, save_raw_output=False, co2_and_non_co2_warming=False, temp_thresholds=(1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0), peak_percentiles=(5, 10, 17, 25, 33, 50, 66, 67, 75, 83, 90, 95), percentiles=(5, 10, 16.666666666666664, 17, 25, 33, 50, 66, 67, 75, 83, 83.33333333333334, 90, 95), historical_warming=0.85, historical_warming_reference_period='1850-1900', historical_warming_evaluation_period='1995-2014')

Postprocess 

climate_assessment.postprocess.do_postprocess(output, outdir, key_string, prefix, model_version='v7.5.3', categorisation=True, reporting_completeness_categorisation=True, gwp=True, model='magicc')

Runs any required postprocessing steps

Namely:

Categorisation
Completeness checks
Add GWP100 Kyoto Gases sums
Adding additional metadata
Plots

Checks on input and output scenario data 

climate_assessment.checks.add_completeness_category(df, filename=None, output_csv=False, outdir='output', prefix='AR6 climate diagnostics|Harmonized|'): Add a meta column that specified the reporting completeness based on qualitative categories.

climate_assessment.checks.check_against_historical(df, filename, instance, output_csv=False, outdir='output'): Check against historical data. Currently requires 2015 to already be in the dataframe.

climate_assessment.checks.check_negatives(df, filename=None, negativethreshold=-0.1, outdir='output', prefix=''): Check for negative emissions and remove any timeseries which has negative non-CO2 values.

climate_assessment.checks.check_reported_co2(df, filename, output_csv=False, outdir='output'): Check CO2 reporting.

climate_assessment.checks.co2_energyandindustrialprocesses(df): Auxiliary function for climate_assessment.checks.check_reported_co2(). Check if either CO2|Energy and Industrial are reported. Add the aggregate Energy and Industrial, if only subcomponents reported

climate_assessment.checks.infiller_vetting(df, prefix='AR6 climate diagnostics'): Filters out a set of harmonized emissions pathways df to then return an infiller_database that does not have emissions pathways that might skew infilling due to what is likely a model reporting error.

climate_assessment.checks.perform_input_checks(df, output_csv_files=True, output_filename='checks', lead_variable_check=False, historical_check=False, reporting_completeness_check=False, outdir='output')

Perform several selected checks on the native emissions input to the climate assessment workflow.

Parameters:

input_df (pyam.IamDataFrame) – Input native emissions before checks.
inputcheck (bool) – Perform checks to remove unsuitable emissisons pathways. [Default: True]
key_string (str) – Identifier string for writing out results. By default derived from the input_emissions_file string in a CLI command.
outdir (str) – Path to output folder.

Returns:

IAM dataframe with only model-scenario that will be without emissions. Output also includes the input emissions. The output is interpolated onto an annual timestep.

Return type:

pyam.IamDataFrame

climate_assessment.checks.reclassify_waste_and_other_co2_ar6(df: IamDataFrame) → IamDataFrame

Reclassify waste and other CO2 into the energy and industrial processes category

Reclassify CO2 emissions reported under Emissions|CO2|Other, Emissions|CO2|Waste, Emissions|CO2|Other Capture and Removal, and Emissions|CO2|Product Use, instead putting them under Emissions|CO2|Energy and Industrial Processes. This function should be aligned with the version of common-definitions, or a similar template that is used for the data ingested. See: https://github.com/IAMconsortium/common-definitions/blob/main/definitions/variable/emissions/emissions.yaml

Parameters:: df (pyam.IamDataFrame) – The original set of reported emissions
Returns:: Reclassified set of emissions.
Return type:: pyam.IamDataFrame

climate_assessment.checks.remove_rows_with_only_zero(df, filename=None, outdir='output'): Check and remove rows with zero emissions.

climate_assessment.checks.remove_rows_with_zero_in_harmonization_year(df, filename=None, harmonization_year=2015, outdir='output'): Check for zeros in harmonization year emissions and remove rows.

climate_assessment.checks.require_allyears(df, filename='test', output_csv=False, outdir='output', required_years=[2020, 2030, 2040, 2050, 2060, 2070, 2080, 2090, 2100], base_yr=2015, low_yr=2010): Check if some variables are reported for the required years (drops per variable)

climate_assessment.checks.sanity_check_bounds_kyoto_emissions(output_postprocess, out_kyoto_infilled): Check that the calculated Kyoto gases of the infilled emissions data are within certain bounds

climate_assessment.checks.sanity_check_comparison_kyoto_gases(output_postprocess, out_kyoto_harmonized, out_kyoto_infilled): Check that the calculated Kyoto gases of the infilled emissions data is in every year smaller than the calculated Kyoto gases of the harmonized emission data

climate_assessment.checks.sanity_check_hierarchy(co2_inf_db, harmonized, infilled, out_afolu, out_fossil): Check that hierarchy of variables is internally consistent (in this case check that Emissions|CO2 is the sum of AFOLU and Energy and Industrial Processes emissions)