PyCBC Live O3 development

PyCBC Live Development

O3 Review page

• Template Bank in used for ER14/EarlyO3 is here

To Do Table for O3/ER14

Issue	description	Assigned to	Status
Reduce memory usage on master node		Alex/Tito	Done
Verify FAR + pvalue combination	To see if satisfies expected distribution as shown here: pvalue combining tests	Tom, Alex, Tito	Seems to work, deployed
Verify triggers from simulated data	To check triple, double coincs and single trigger
Verify triggers from O2 replay	To check triple, double coincs and single trigger
Verify stability of multidetector analysis	Check lag, duty factor and memory usage over time	Tito, Khun Sang (?)	Stable over O(hours), lag still often too high
GraceDB upload	To check the structure of the coinc xml file and check uploads	Bhooshan	Works
Follow-up early trigger https://pycbc.slack.com/archives/lvc-low-latency/p1550510201048400	Error while assigning p-value to trigger at the beginning of the run	Alex, Bhooshan
Phase-time-amp files for all HLV combinations		Tito	Done

To Do Table for O3b

Issue	description	Assigned to	Status
P_astro improvement	Calculate relative probabilities of source types based on template chirp mass - optionally also incorporate some redshift/distance measure. Question : whether code belongs in pycbc live or gwcelery (gracedb alert content generator package)?	Veronica V./TD (initial codes by A. Curiel)	Initial commit of area calc module #2951, need to calibrate distance/redshift estimate & integrate into PLive
Single-ifo trigger alerts	Estimate single trigger FAR based on fitting existing O3a triggers (relevant region of the bank TBD) plus 'conservative' trials factor. Followup for such triggers?	Veronica V., Gareth D., TD, ..	Done (link)

Additional things

Task	Description	Assigned to	Relevance + Status
Automated testing	Add a basic unit test to Travis	Tito	Done
Automated testing	Create a set of scripts to verify the correct behavior of the full search	Tito/Max	Done
Data fetching and processing at main process	Data fetching and preprocessing to head node of MPI and then distribute to filtering nodes	Bhooshan?	Is it feasible?
Autogating	Applying autogating to strain	Liting Xiao	Done
SG Chisq	Use SG chisq and relevant statistics	Ian	Done
Save more metadata in output	add channel names of strain, status and DQ, status and DQ flags, PyCBC version, values of thresholds, maybe the entire command line to output hdf5 file	Tito	Done
Save lag and duty factor to HDF files	Removes the need to parse log files
Data-fetching after template generation	Start reading frames at a time after the template waveforms have been generated. This should reduce the amount of lookback frames that need to be present. Sometimes this results in a file not found error if the bank takes a long time to generate.	Alex	Done
Trigger followup	Upload plots of trigger SNR time series and PSD	Ian	Done
Trigger followup	Upload plots of background
Run on Python 3		Tito	Done
p_astro	PyCBC Live should become compatible with the code for generating p_astro in low latency		Sometimes in O3
Better handling of single-detector triggers	Signals loud in one detector but not others should generate triggers with an appropriate FAR		Sometimes in O3
iDQ integration	Can low-latency iDQ info be used to improve VT?	Max?
Better state/DQ settings	More flexible per-detector settings and support for Virgo's veto streams	Tito	Done
Report why detectors are not used	When a detector is not used, report the reason in the output file (state vector, bad PSD, late frame files etc)

Multi IFO Plan Outline

• Calculate all possible double coinc combinations
• Pick "best" one
• For all remaining IFOS calculate a p-value for the on-source time with the same template
• combine double coinc FAR with pvalues from other ifos

Methods and options for improvements

• handling of singles

  • improvement: handle case where one detector is non-vetoed by not the only "active" one.

• Preparation of strain

  • improvement: add in autogating

• Calculation of the pvalue

  • improvement: Better check of DQ in the background time

• How to pick the "best" double coinc?

  • Simple implementation: best far, largest SNR
    • Downside: Larger trials factor
  • Possibly better: Choose based on best instrument combination, past noise, sensitivity

• How to combine FAR with pvalues from additional detectors?

  • Simple implementation: Fisher's method, using fiducial "foreground time" (.01 * IFAR).
  • Possibly better: Take trials factor if additional ifos will reduce significance (might expect low snr)
  • Possibly better: Do a weighted combinations of p-values based on additional information (detector sensitivity, background).

Write methods paper

1. Where? here's a repo .. https://github.com/gwastro/pycbc_live_o3
2. By when? (Before first publication on O3 data would be nice!)
3. Paper with 5 detector network and for BNS and NSBH (q < 5) (may be)

Possibility of improving stability for O4

• Keep running even if one of the workers fail
  • Easy fix: Neglect those templates?
  • Start another booked but unused worker with the same set of templates
• Do coinc on more than 1 node: speed, stable (?), easier to process delayed queue
• Can we survive the crash of head node in mpi?

Add better injection infra

• Injections on fly
• InjFilterRejLive: Filter strain with and/or without injections

(Don't need answers here yet, but this needs to be on the radar)

Older page for reference: PyCBC-Live-O3-development-Old