Examples for how to accomplish things that are not covered by cabinetry API

[alexander-held]

This is going to serve as a central place to gather solutions to questions about how to accomplish tasks that are currently not covered by cabinetry API and other resources that might be useful.

Customizing data/MC plots: mplhep, experiment labels, normalized signal overlay

via #425

see https://gist.github.com/alexander-held/2ca63e4c4c3de2114bf8d903bf28bb4a

Removing specific parameters from fit result visualizations

via #441 (reply in thread):

import numpy as np

# assumes you have some object fit_results with your fit results
# remove all parameters with "staterror" in the name
delete_indices = [idx for idx, label in enumerate(fit_results.labels) if "staterror" in label]

fit_results_pruned = cabinetry.fit.FitResults(
    np.delete(fit_results.bestfit, delete_indices),
    np.delete(fit_results.uncertainty, delete_indices),
    np.delete(fit_results.labels, delete_indices),
    np.delete(np.delete(fit_results.corr_mat, delete_indices, axis=1), delete_indices, axis=0),
    fit_results.best_twice_nll,
    fit_results.goodness_of_fit,
    fit_results.minos_uncertainty,
)

cabinetry.visualize.correlation_matrix(fit_results_pruned)

Creating shape-only systematic variations

via #445 (comment)

import cabinetry

ws = cabinetry.workspace.load("workspaces/example_workspace.json")

MOD_NAMES_TO_PRUNE = ["Modeling"]  # modifiers name to prune
MOD_TYPE_TO_PRUNE = ["normsys"]  # modifier types to prune

# the ws object is just a dictionary, iterate through that and drop pieces as needed
# only prune modifiers matching all requirements (name and type)
for channel in ws["channels"]:
    for sample in channel["samples"]:
        sample["modifiers"] = [
            mod
            for mod in sample["modifiers"]
            if not (
                mod["name"] in MOD_NAMES_TO_PRUNE and mod["type"] in MOD_TYPE_TO_PRUNE
            )
        ]

model, data = cabinetry.model_utils.model_and_data(ws)

Extrapolating fit results to validation regions

see #431

Debugging NaN results in fits

via scikit-hep/pyhf#1695 (comment)

see https://gist.github.com/alexander-held/2f6727df4cc8f65d75368e293ca83b5a

The best method to find the culprit causing NaN likelihoods might depend a bit on the specific case, the approach outlined in this comment linked is quite generic.

2d likelihood scan

see https://gist.github.com/alexander-held/26724f1aa641632270a41079a8beac9c

Decorrelate statistical uncertainties across samples

via #450

import cabinetry

spec = cabinetry.workspace.load("workspaces/example_workspace.json")

# decorrelate MC statistical uncertainty of "Signal" samples
for channel in spec["channels"]:
    for sample in channel["samples"]:
        if sample["name"] != "Signal":  # pick the sample you need
            continue

        for modifier in sample["modifiers"]:
            if modifier["type"] == "staterror":
                modifier["type"] = "shapesys"
                modifier["name"] = f"{modifier['name'].replace('staterror', 'shapesys')}_{sample['name']}"

cabinetry.workspace.save(spec, "ws_modified.json")  # optionally save again

model, data = cabinetry.model_utils.model_and_data(spec)  # continue as usual
_ = cabinetry.fit.fit(model, data)

Decorrelate systematic uncertainty across bins

via #453
see #453 (comment)

POI uncertainty contributions from categories of parameters via quadrature subtraction

via #455
see #455 (reply in thread)