Filter days shown in dropdown menu

[nritsche]

From @jrs65 :

1. Exclude a day if more than 80% of the total day is flagged.
2. Exclude a day if more than 50% of the cross talk estimation region is flagged.

base = Path("/project/rpp-krs/chime/chime_processed/daily/rev_02/")available_lsds = np.array(sorted([int(path.stem.split("_")[-1]) for path in base.glob("*/sstream_lsd_*.h5")]))dates = ctime.unix_to_datetime(ephemeris.csd_to_unix(np.array(available_lsds)))
quarter_ind = np.array([(d.month - 1) // 3 for d in dates])crosstalk_ra = np.array([[165, 180],[240, 255],[315, 330],[45, 60]])core.connect()bad_flags = [
    'bad_calibration_fpga_restart',
    #'globalflag',
    #'acjump',
    'acjump_sd',
    #'rain',
    #'rain_sd',
    'bad_calibration_acquisition_restart',
    #'misc',
    #'rain1mm',
    'rain1mm_sd',
    'srs/bad_ringmap_broadband',
    'bad_calibration_gains',
    'snow',
    'decorrelated_cylinder',
]#           - acjump
#           - bad_calibration_acquisition_restart
#           - bad_calibration_fpga_restart
#           - bad_calibration_gains
#           - decorrelated_cylinder
#           - globalflag
#           - rain1mmflags = df.DataFlag.select().join(df.DataFlagType).where(df.DataFlagType.name << bad_flags)flag_time_spans = [(f.type.name, f.start_time, f.finish_time) for f in flags]xtalk_lsd_range = available_lsds[:, np.newaxis] + crosstalk_ra[quarter_ind] / 360.0xtalk_time_range = ephemeris.csd_to_unix(xtalk_lsd_range)lsd_time_range = ephemeris.csd_to_unix(available_lsds[:, np.newaxis] + np.array([[0, 1]]))def frac_masked(time_ranges, flags):    from skyfield.positionlib import _to_altaz
    eph = ctime.skyfield_wrapper.ephemeris    x = np.linspace(0, 1, 1024)[np.newaxis, :]    grid = (1 - x) * time_ranges[:, np.newaxis, 0] + time_ranges[:, np.newaxis, 1] * x    # Initially flag out all day time data
    obs = ephemeris.chime.skyfield_obs()
    pos = obs.at(ctime.unix_to_skyfield_time(grid.ravel())).observe(eph["sun"])
    alt = _to_altaz(pos, None, None)[0].degrees
    mask = (alt > 0).reshape(grid.shape)    # Then loop over all known flags and apply them to data
    for _, ta, tb in flags:
        mask[(grid > ta) & (grid < tb)] = True    return mask.mean(axis=1)df_frac = frac_masked(lsd_time_range, flag_time_spans)xt_frac = frac_masked(xtalk_time_range, flag_time_spans)df_flag = df_frac > 0.8
xt_flag = xt_frac > 0.5bad_days = (df_flag | xt_flag)
good_days = (~(df_flag | xt_flag))

frac_masked is a bit of a hack. Rather than trying to do a fancy job calculating all the unions of flags to see what fraction of the day is masked, it just makes a grid of 1024 points across the day and applies each flag and then calculates the fraction from those finite samples.

Either we set a default and have people start with that, or don't let people adjust it

One thing to note is that the cross talk estimation regions might change at some point. It's really my first pass at picking ones that work for each quarter, and that scheme might change, so it's possible that configuration should be considered specific to a revision.