traceApertures loses a trace on a fairly strong source

[KathleenLabrie]

Version: release/3.1.x

This is GMOS LS N&S. The apertures are found and defined nicely. The trace works really well for a while, nice and tight, and then there's a sharp drop. The 2D image does not show anything particular happening where the drop and lost of the trace happens.

To reproduce:
reduce -r traceApertures N20080830S0261_aperturesFound.fits

The aperture file can be found at: https://drive.google.com/drive/folders/16Juhlho1ALWTSuoiH9-4cJ8DLwKZMicP?usp=share_link

Ignore the red fit, it's the big discontinuity in the black dot that is the problem. The 2D image clearly shows the source at 1016 in that region, not 1014.

BTW, since it was discussed a couple weeks ago for a different case, I tried with and without CR flagging, no difference. No need to go down that road.

[KathleenLabrie]

Chris correctly points out that trace is doing exactly the right thing. Upon close inspection, there is a jump in the stacked data.

Suspecting a problem with the mosaic solution, I looked at a standard from the same time and the mosaic looks fine. Then I looked a the 261, 262, and 265 frames, pre-alignment, pre-stack, just after the beams have been combined. There are no jump at the chip gap. So it can't be the mosaic solution.

The jump appears during the alignment/resampling/stack phase.

The 3 frames are dithered in wavelength, 10nm between each. There's a slight (~4 pixels) spatial shift in 162 relative to the other two.

[KathleenLabrie]

Correction, the jump does NOT appear during the alignment/resampling/stack. It is there in the beamCombined frames, just not at the chip gap.

So, NOT mosaic. NOT alignment. It is starting to look like the distortion correction is not quite correct, maybe?

[KathleenLabrie]

Or a N&S feature for that part of the detector? Or a grating feature at that wavelength?

[KathleenLabrie]

I'm unable to find data to prove one hypothesis or the other. The discontinuity is in the data itself and not an artifact of the reduction, I'm almost 100% sure of that. The possibilities are: 1) intrinsic to the source (seems unlikely), 2) some glitch in the shuffling of the charges, 3) some weird effect in the grating.

[chris-simpson]

distortionCorrect() doesn't change the rows of the pixels, it just shuffles them along each row.

A glitch in the charge shuffling sounds reasonable but I don't think it's actually a sharp discontinuity. If the centroid shifted by 1 pixel suddenly then the trace wouldn't be able to catch it for values of "Max shifted" less than 0.017. The step size is 10 and it's allowed to miss at most 5 steps before giving up so it has only 60 pixels to achieve that 1 pixel jump, but "Max shifted" can be as low as 0.012 and it will keep the trace. (Note that the value printed above the slide bar is truncated to 2 decimal places, when it should really be at least 3, and the slider bar seems to be too coarse and effectively not work; you might want to ask somebody to look into this?) The shift happens where the source is unusually faint and...

Ah. Second order light. The source goes faint and then gets bright again around 1030nm. And the blocking filter is the OG515, which cuts on at 515nm so second order light will appear at 1030nm.

[KathleenLabrie]

okay, now that makes a lot sense. Do we have code in place to limit the range of the trace? Not a showstopper, that's why we have the interactive tools. Something to consider though for future improvements.

[chris-simpson]

Right, so the interactive tool makes it apparent that something is going wrong and the user can either reduce the "max shifted" parameter to prevent the trace from hopping the gap fast enough (in which case it will end when the second order light comes in) or can simply reject the contaminated points.

In terms of having some sort of automated way to say "Ah, it's the OG515 filter, so that cuts on at 515nm, which means second order light will be a problem from 1030nm, so we should stop the trace there", I think we have better things to focus on and we don't want to pollute the core traceApertures() primitive with Gemini-specific code, so the right place to put a "stop at this wavelength" value isn't clear to me. Bear in mind that these are not our data, so the fact that it wasn't clear to us that there would be a problem with second order light doesn't mean that the PI was unaware. It's also not a massive problem since even the messed up trace is within half a pixel of the correct location so there won't be much of a flux error if the aperture is a decent size.

[chris-simpson]

I've relabelled this as "under discussion" and "urgency-low" and removed "bug" because we understand it and nothing bad is happening. As per my last reply, I don't actually think there's anything to do here. We have the interactive tools to handle these sorts of cases and the default behaviour, although not perfect, isn't terrible.