Solver has started to apparently run forever

[mikepictor]

Yesterday I went from getting a perfectly reasonable solution from my solver, to no solution at all. By this I mean that the solver seems to all of sudden be running forever, or at least I can see no sign of it stopping. My spent-limit is set to 4s (I'll give it more time in production, I just want it coming back to me quickly while I work on it), and that was very reliable until recently.

val solverJob: SolverJob<FillingSchedule, Long> = solverManager.solve(problemId, problem)
val solution: FillingSchedule = try { solverJob.finalBestSolution } catch(...){...}

it's not throwing exceptions, and it's no longer getting past this line.

I even rewound my code to an earlier iteration where I was getting results, and now again, it just seems to run forever. I can't even imagine what may have changed. My problem looks fine. I can even comment out all my constraints, and it STILL seems to run forever and/or it isn't returning a solution.

This just started late yesterday...one moment I was getting solves, then I wasn't. My colleague is running the same code, and it also never returns for him.

I experimented with this

val solution: FillingSchedule = try {
            thread {
                sleep(6000)
                solverJob.terminateEarly()
            }
            solverJob.finalBestSolution
        }

and I do see in the logs that it is ending after 6 seconds (remember it should end after 4)...but I still get nothing back.

I know I didn't provide any reproducible code base, but I guess I'm just seeking ideas of what to look for? What changes can I try as an experiment?

My yaml config is as follows..

timefold:
  solver:
    environmentMode: TRACKED_FULL_ASSERT
    termination:
      spent-limit: 4s
      unimproved-spent-limit: 4s

I tried changing the environment mode to REPRODUCIBLE, but it didn't help. All I get in the log is

2024-02-13 14:53:24.597 [pool-6-thread-1] INFO  ai.timefold.solver.core.impl.solver.DefaultSolver [] - Solving started: time spent (48), best score (-7init/0hard/-1medium/0soft), environment mode (TRACKED_FULL_ASSERT), move thread count (NONE), random (JDK with seed 0).
2024-02-13 14:53:30.551 [Thread-16] INFO  ai.timefold.solver.core.impl.solver.DefaultSolver [] - Terminating solver early.

I'd appreciate any advice that might make me look in the right direction.

[triceo]

Hello @mikepictor. Just so that I understand correctly - are you saying that your code, without any change, suddenly started behaving differently? No version upgrade of anything (incl. the JDK), no data set changes, no config changes?

[triceo]

@mikepictor Can you give an example of what changed? We certainly didn't intend to change anything that already worked. (I noticed you are somehow using YAML; we do not actually test that. But the format of property files remains unchanged.)

[mikepictor]

yeah, I configure it like so

timefold:
  solver:
    environmentMode: TRACKED_FULL_ASSERT
    termination:
      spent-limit: 10s
      unimproved-spent-limit: 4s

If I upgrade to 1.7, I get the following error starting up

Failed to bind properties under 'timefold' to ai.timefold.solver.spring.boot.autoconfigure.config.TimefoldProperties:

    Property: timefold.solver.termination.unimproved-spent-limit
    Value: "4s"
    Origin: class path resource [application.yml] - 118:31
    Reason: java.lang.ClassCastException: class java.lang.String cannot be cast to class java.util.Map (java.lang.String and java.util.Map are in module java.base of loader 'bootstrap')

Action:

Update your application's configuration

Your TimefoldProperties class has changed. solver is now a map..
private Map<String, SolverProperties> solver;

I need to group solver under a key...I am guessing simply default? I need to change something though. I just haven't spent time figuring out what (you only released it a few hours ago :) )

[triceo]

This is unfortunate. Until your question, I wasn't even aware that Quarkus supports YAML as config format. Had I known, this wouldn't have happened - we take backwards compatibility very seriously. Apologies for this.

[mikepictor]

No worries. I am using Springboot to be clear, I am not very familiar with Quarkus.

If you have insight into how I am meant to structure the config now, I'd be happy to hear it. I'll do some experimenting, and check your documentation more.

[mikepictor]

I will add, I think I may have solved my original issue (on 1.6) by using forEachIncludingNullVars. Since I allowed nulls, my constraints I guess saw no planning entities, and so never got penalized. Somewhere in there it may have hit an error...but since including the null var condition, it is resolving. I do have score corruption, but I am trying to get to the bottom of that.

[zepfred]

Hello, @mikepictor. About the upgrade to version 1.7.0, we are now enforcing the use of the kebab-case format to match our documentation. I noticed that the properties you shared have an invalid key environmentMode. These properties should unblock you:

timefold:
  solver:
    environment-mode: TRACKED_FULL_ASSERT
    termination:
      spent-limit: 10s
      unimproved-spent-limit: 4s

Furthermore, the PR #637 will enhance the error message by making it more consistent and easier to understand.

Thank you for sharing the issue!

[triceo]

A little clarification from my end: until your report, @mikepictor, we didn't know that YAML was a possibility, and that Spring allowed users to use pretty much any case they want. The changes that @zepfred mentions will come into effect in 1.8.0 and should prevent this issue from ever happening again - besides the better error message, we have also added a whole bunch of extra tests, now that we know that YAML is a possibility.

[mikepictor]

regrettably that did not work

I also tried

timefold:
  solver:
    default:
      environment-mode: TRACKED_FULL_ASSERT
      termination:
        spent-limit: 10s
        unimproved-spent-limit: 4s

as a way of initializing the map key, but that didn't seem to work either.

I am not very stressed..I can stay with 1.6 for now, but I am still curious what is missing. Glad I helped elaborate your use cases though

[triceo]

I see two options here:

• You can try to somehow minimize it, remove it from your codebase, and give us code that reproduces it. We want to fix it, we just need to be able to see it.
• Or you can use standard properties file instead of YAML, as I think that - in this case - the complexity of YAML doesn't bring anything to the table.