Skip to content

Commit

Permalink
Avoiding use of "####" where unecessary
Browse files Browse the repository at this point in the history
  • Loading branch information
nucleosynthesis committed Oct 19, 2023
1 parent 3e89502 commit 9420afe
Show file tree
Hide file tree
Showing 6 changed files with 16 additions and 15 deletions.
2 changes: 1 addition & 1 deletion docs/part2/settinguptheanalysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,7 @@ name rateParam bin process rootfile:workspacename
The name should correspond to the name of the object which is being picked up inside the RooWorkspace. A simple example using the SM XS and BR splines available in HiggsAnalysis/CombinedLimit can be found under [data/tutorials/rate_params/simple_sm_datacard.txt](https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/blob/main/data/tutorials/rate_params/simple_sm_datacard.txt)


#### Extra arguments
### Extra arguments

If a parameter is intended to be used and it is *not* a user defined `param` or `rateParam`, it can be picked up by first issuing an `extArgs` directive before this line in the datacard. The syntax for `extArgs` is

Expand Down
6 changes: 3 additions & 3 deletions docs/part3/commonstatsmethods.md
Original file line number Diff line number Diff line change
Expand Up @@ -930,7 +930,7 @@ If you suspect your fits/uncertainties are not stable, you may also try to run c
For a full list of options use `combine -M MultiDimFit --help`
#### Fitting only some parameters
### Fitting only some parameters
If your model contains more than one parameter of interest, you can still decide to fit a smaller number of them, using the option `--parameters` (or `-P`), with a syntax like this:
Expand Down Expand Up @@ -1023,7 +1023,7 @@ This can sometimes be an issue as Minuit may not know if has successfully conver
As in general for `HybridNew`, you can split the task into multiple tasks (grid and/or batch) and then merge the outputs, as described in the [combineTool for job submission](http://cms-analysis.github.io/HiggsAnalysis-CombinedLimit/part3/runningthetool/#combinetool-for-job-submission) section.


### Extracting contours
### Extracting contours from results files

As in general for `HybridNew`, you can split the task into multiple tasks (grid and/or batch) and then merge the outputs with `hadd`. You can also refer to the [combineTool for job submission](http://cms-analysis.github.io/HiggsAnalysis-CombinedLimit/part3/runningthetool/#combinetool-for-job-submission) section for submitting the jobs to the grid/batch or if you have more than one parameter of interest, see the instructions for running `HybridNew` on a grid of parameter points on the [CombineHarvest - HybridNewGrid](http://cms-analysis.github.io/CombineHarvester/md_docs__hybrid_new_grid.html) documentation.

Expand All @@ -1039,7 +1039,7 @@ The output tree will contain the values of the POI which crosses the critical va

You can produce a plot of the value of $p_{x}$ vs the parameter of interest $x$ by adding the option `--plot <plotname>`.

#### 2D contours
#### Extracting 2D contours

There is a tool for extracting *2D contours* from the output of `HybridNew` located in `test/makeFCcontour.py` provided the option `--saveHybridResult` was included when running `HybridNew`. It can be run with the usual combine output files (or several of them) as input,

Expand Down
2 changes: 1 addition & 1 deletion docs/part3/nonstandard.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ The `combine` tool has a number of features for diagnostics and plotting results

This can in some cases be achieved by removing a specific datacard when running `combineCards.py` however, when doing so the information of particular nuisances and pdfs in that region will be lost. Instead, it is possible to ***mask*** that channel from the likelihood! This is acheived at the `text2Workspace` step using the option `--channel-masks`.

#### Example: removing constraints from the signal region
### Example: removing constraints from the signal region

We will take the control region example from the rate parameters tutorial from [data/tutorials/rate_params/](https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/tree/81x-root606/data/tutorials/rate_params).

Expand Down
16 changes: 8 additions & 8 deletions docs/part3/runningthetool.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ There are a number of useful command line options which can be used to alter the

By default, the dataset used by combine will be the one pointed to in the datacard. You can tell combine to use a different dataset (for example a toy one that you generated) by using the option `--dataset`. The argument should be `rootfile.root:workspace:location` or `rootfile.root:location`. In order to use this option, you must first convert your datacard to a binary workspace and use this binary workspace as the input to the command line.

#### Generic Minimizer Options
### Generic Minimizer Options

Combine uses its own minimizer class which is used to steer Minuit (via RooMinimizer) named the `CascadeMinimizer`. This allows for sequential minimization which can help in case a particular setting/algo fails. Also, the `CascadeMinimizer` knows about extra features of Combine such as *discrete* nuisance parameters.

Expand Down Expand Up @@ -105,7 +105,7 @@ You can find details about these in the Minuit2 documentation [here](https://roo
More of these options can be found in the **Cascade Minimizer options** section when running `--help`.


#### Output from combine
### Output from combine

Most methods will print the results of the computation to the screen, however, in addition, combine will also produce a root file containing a tree called **limit** with these results. The name of this file will be of the format,

Expand Down Expand Up @@ -156,7 +156,7 @@ The output file will contain the toys (as `RooDataSets` for the observables, inc
The branches that are created by methods like `MultiDimFit` *will not* show the values used to generate the toy. If you also want the TTree to show the values of the POIs used to generate to toy, you should add additional branches using the `--trackParameters` option as described in the [common command line options](#common-command-line-options) section above. These branches will behave as expected when adding the option `--saveToys`.


#### Asimov datasets
### Asimov datasets

If you are using wither `-t -1` or using `AsymptoticLimits`, combine will calculate results based on an Asimov dataset.

Expand Down Expand Up @@ -184,7 +184,7 @@ You can turn off the internal logic by setting `--X-rtd TMCSO_AdaptivePseudoAsim
If you set `--X-rtd TMCSO_PseudoAsimov=X` with `X>0` and also turn on `--X-rtd TMCSO_AdaptivePseudoAsimov=`$\beta$, with $\beta>0$, the internal logic will be used but this time the default will be to generate Pseudo-Asimov datasets, rather than the normal Asimov ones.


#### Nuisance parameter generation
### Nuisance parameter generation

The default method of dealing with systematics is to generate random values (around their nominal values, see above) for the nuisance parameters, according to their prior pdfs centred around their default values, *before* generating the data. The *unconstrained* nuisance parameters (eg `flatParam` or `rateParam`) or those with *flat* priors are **not** randomised before the data generation. If you wish to also randomise these parameters, you **must** declare these as `flatParam` in your datacard and when running text2workspace you must add the option `--X-assign-flatParam-prior` in the command line.

Expand All @@ -199,14 +199,14 @@ If you are using `toysFrequentist`, be aware that the values set by `--setParame
!!! warning
The methods such as `AsymptoticLimits` and `HybridNew --LHCmode LHC-limits`, the "nominal" nuisance parameter values are taken from fits to the data and are therefore not "blind" to the observed data by default (following the fully frequentist paradigm). See the detailed documentation on these methods for avoiding this and running in a completely "blind" mode.

#### Generate only
### Generate only

It is also possible to generate the toys first and then feed them to the Methods in combine. This can be done using `-M GenerateOnly --saveToys`. The toys can then be read and used with the other methods by specifying `--toysFile=higgsCombineTest.GenerateOnly...` and using the same options for the toy generation.

!!! warning
Some Methods also use toys within the method itself (eg `AsymptoticLimits` and `HybridNew`). For these, you should **not** specify the toy generation with `-t` or the options above and instead follow the specific instructions.

#### Loading snapshots
### Loading snapshots

Snapshots from workspaces can be loaded and used in order to generate toys using the option `--snapshotName <name of snapshot>`. This will first set the parameters to the values in the snapshot *before* any other parameter options are set and toys are generated.

Expand Down Expand Up @@ -265,7 +265,7 @@ Remember, any usual options (such as redefining POIs or freezing parameters) are
The option `-n NAME` should be included to avoid overwriting output files as the jobs will be run inside the directory from which the command is issued.


### Running combine jobs on the Grid
### Grid submission with combineTool

For more CPU-intensive tasks, for example determining limits for complex models using toys, it is generally not feasible to compute all the results interactively. Instead, these jobs can be submitted to the Grid.

Expand Down Expand Up @@ -314,7 +314,7 @@ combineTool.py -M HybridNew -d htt_mt.root --LHCmode LHC-limits --singlePoint 0.

When the `--dry-run` option is removed each command will be run in sequence.

#### Grid submission
### Grid submission with crab3

Submission to the grid with `crab3` works in a similar way. Before doing so ensure that the `crab3` environment has been sourced in addition to the CMSSW environment. We will use the example of generating a grid of test-statistic distributions for limits.

Expand Down
2 changes: 1 addition & 1 deletion docs/part5/longexercise.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,8 @@ This produces a new ROOT file `cls_qmu_distributions.root` containing the plots,
```shell
python3 printTestStatPlots.py cls_qmu_distributions.root
```
### Advanced section: B: Asymptotic approximation limitations

#### Advanced exercises
These distributions can be useful in understanding features in the CLs limits, especially in the low statistics regime. To explore this, try reducing the observed and expected yields in the datacard by a factor of 10, and rerun the above steps to compare the observed and expected limits with the asymptotic approach, and plot the test statistic distributions.

**Tasks and questions:**
Expand Down
3 changes: 2 additions & 1 deletion docs/part5/longexerciseanswers.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@

</details>

#### Advanced exercises
### Advanced section: B: Asymptotic approximation limitations

**Tasks and questions:**

- Is the asymptotic limit still a good approximation?
Expand Down

0 comments on commit 9420afe

Please sign in to comment.