Releases: cadet/CADET-Match
Dextran fixes
Found and fixed the offset issue in DextranSSE and DextranShape
More minor fixes
DextranShape and DextranSSE have improvements to calculating the SSE and applying smoothing to the system. Fixed a few other minor issues.
There is a small tradeoff between smoothing and accuracy for purely synthetic data but necessary for experimental data and sometimes necessary for synthetic data.
Minor fixes
Some of the plotting for MCMC was not working correctly. This fixes those issues and makes a few adjustments to building the Kernel Density Estimators used in the error model and maximum likelihood calculations.
Continuous and deterministic is good
MCMC is working again. A bug was found related to async processing in MCMC.
AbsoluteTime and DextranShape are now continuous in time for finding the offset. This is important for MCMC and also improves the Dextran fits. The system now looks at 90% of peak max instead of peak max so that a spline can be fit and the exact offset found.
KDE bandwidth, time offsets and a few other things use powell's method as a local minimizer to give deterministic and faster results while also improves quality of fit and MCMC performance.
NSGA3 reference points and chromatogram offset
This version picks the reference points for NSGA3 more dynamically so that it reduces the points per dimension as the dimensions increase. A problem was found when using 54 goals that the memory usage grew so long the system would run out of memory. This should be fixed now. A scaling method for reference points was used from the original NSGA3 paper.
Chromatogram offsets are also found now using resampling to 100 points/second and the cross-correlation to find the offset. This method is deterministic and about 300x faster than the previous search method while also being more robust.
Updated smoothing and more stable time offset
This release mostly has some small updates to the smoothing code and some other things. The smoothing should work for more types of data now. The code for finding the time offset between two signals has also been upgraded to make it more robust.
emcee 3 and removal of scoop
Scoop has been removed from CADETMatch and the system now uses the python multiprocessing model. This removes the ability to run on multiple nodes (which was not used) but adds VASTLY better debuggability. The code is much easier to debug now including debugging while running in parallel.
Logging has been upgraded as part of this.
Emcee 3.x upgrade is complete. The code uses a combination of differential evolution moves by default and converges faster than before. The system runs a 3 phase MCMC process. The first phase just samples for a while until the 5 and 95 confidence regions stabilize and then resets the upper and lower bounds to center on this region. The second phase does an automatic burn in and adjusts some step size values to get a good acceptance rate and finally the actual run is performed.
MCMC and smoothing
MCMC has been upgraded to emcee 3.x and the moves have been changed from stretch to a combination of DE moves from https://emcee.readthedocs.io/en/stable/user/moves/
0.1 DE-snooker 0.81 DE 0.09 DE(gamma=1.0)
The smoothing system has been completely redone. A butter lowpass filter is first applied to remove most noise and a smoothing spline is applied to get a clean derivative.
Fixed smoothing spline
The new smoothing spline code failed when used without data normalization and the values where small (<1e-4 or so) which broke some of the searching. This has been fixed.
Better splines and multistart
This version has a new data smoothing system for dealing with experimental data. It also has an improved Multistart which checks the top points instead of all of them.