Add QP solver through OSQP #225

cdiener · 2020-12-11T21:26:57Z

This is a first working version of a QP solver via OSQP. OSQP expects the problem to be sparse and static so I added a thin compatibility layer that manages the OSQP problem via dictionaries. All operations on that layer are at worst O(nv + nc) but it makes problem setup about 10-20% of the entire solve time for most models. Could maybe be optimized more in the future but should be sufficient for now. Still need to add tests but I added some more complicated models to the test data for now.

It now supports all features such as reduced costs etc. This version is also faster than the previous PR since it gets primals and duals via the C interface instead of evaluating expressions. We used it in the recent MICOM course and it worked well.

Demo:

Python 3.9.1 (default, Dec  8 2020, 07:51:42) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from optlang import osqp_interface as o

In [2]: import json

In [3]: qp = json.load(open("src/optlang/tests/data/QPLIB_8785.json"))

In [4]: mod = o.Model.from_json(qp)

In [6]: len(mod.variables), len(mod.constraints)
Out[6]: (10399, 11362)

In [7]: mod.configuration.verbosity = 1

In [8]: mod.configuration.tolerances.optimality = 1e-6

In [9]: mod.optimize()
-----------------------------------------------------------------
           OSQP v0.6.0  -  Operator Splitting QP Solver
              (c) Bartolomeo Stellato,  Goran Banjac
        University of Oxford  -  Stanford University 2019
-----------------------------------------------------------------
problem:  variables n = 10399, constraints m = 21761
          nnz(P) + nnz(A) = 73422
settings: linear system solver = qdldl,
          eps_abs = 1.0e-06, eps_rel = 1.0e-06,
          eps_prim_inf = 1.0e-06, eps_dual_inf = 1.0e-06,
          rho = 1.00e+00 (adaptive),
          sigma = 1.00e-06, alpha = 1.60, max_iter = 100000
          check_termination: on (interval 25),
          scaling: off, scaled_termination: off
          warm start: on, polish: on, time_limit: off

iter   objective    pri res    dua res    rho        time
   1   0.0000e+00   9.90e+01   9.90e+01   1.00e+00   5.97e-01s
 200   7.8676e+03   3.39e-02   2.37e-05   9.64e-04   3.62e+00s
 300   7.8675e+03   1.17e-04   2.22e-07   6.19e-03   4.87e+00s
plsh   7.8675e+03   8.53e-14   6.68e-10   --------   5.60e+00s

status:               solved
solution polish:      successful
number of iterations: 300
optimal objective:    7867.4911
run time:             5.60e+00s
optimal rho estimate: 5.22e-03

Out[9]: 'optimal'

In [10]: mod.objective.value  # reference solution: 7867.4911490000004051
Out[10]: 7867.491148760153

One should note that this is a direct solver so it performs pretty bad for LPs or at very low tolerances. I added some optimizations for LP so it works ok-ish but worse than GLPK. However it is great for QPs so I think having a specialized solver for that is still helpful.

TODO

add tests
add documentation
update deps?

… costs)

…onstraints

cdiener · 2021-02-13T05:57:21Z

@Midnighter @KristianJensen this is done for now. There may be small fixes we can make in the future but it's pretty complete.

KristianJensen · 2021-02-23T08:06:06Z

@cdiener I will try to have a look at it as soon as possible

carrascomj

A couple of lines about OSQP should be added to installation.rst. Also, I left some comments regarding cuosqp. Other than that, it looks good to me.

carrascomj · 2021-03-04T16:40:02Z

src/optlang/util.py

+    # OSQP can be provided by the base or CUDA package
+    solvers['OSQP'] = False
+    try:
+        import cuosqp


cuosqp does not provide a linear system solver, so it does not cover all the functionality of osqp (see osqp/osqp#30 and osqp/cuosqp#2). I guess the user will know that, but it would be nice to document that (and also document osqp at installation.rst).

I think that just means that the direct linear solver does not run on the GPU for now, but it still uses a linear solver.

carrascomj · 2021-03-04T16:42:58Z

src/optlang/tests/test_osqp_interface.py

+import unittest
+
+try:  # noqa: C901
+    import osqp


Tested this with cuosqp and it does not work for some of the classes of tests (as we would expect). Not that it matters for CI, but would it be fine to have some tests that should pass for cuosqp in case it is installed?

Yeah, it fixes some of the parameters like lambda. What tests fail? I would suspect the LP problems since those have a lambda adjustment in the code that is probably ignored. I'll add a warning when using cuosqp.

The tests that fail with cuosqp are those under ConstraintTestCase and ModelTestCase.test_binary_variables. There could be more. The thing is that the tests don't fail, they error out:

test_binary_variables (optlang.tests.test_osqp_interface.ModelTestCase) ... /home/georg/.virtualenvs/opt/lib/python3.9/site-packages/cuosqp/utils.py:20: UserWarning: Linear system solver not recognized. Using default solver CUDA PCG. warn("Linear system solver not recognized. " + terminate called after throwing an instance of 'thrust::system::system_error' what(): parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device [1] 19842 abort (core dumped) python -m nose src/optlang/tests -v

That's why I thought that they don't have a linear solver. Does it work for you? It might be a problem with my cuda version (using 11.2), but they say v11 is supported (other libraries which use cuda work for me and some of the OSQP tests pass).

This looks more like an issue with the compute capability of the used card. I think I tried it on Google Colab before and it kind of worked.

I could reproduce the error on colab https://gist.github.com/carrascomj/a4d91a926814646e36592a853516c7ac
Is there something that I am not installing maybe?

I think you just did not change the Runtime to one with a GPU. If you do that it does run but it is slow due to the setting issue and some tests fail because it can't hit the needed accuracy for LPs.

RIght! I had changed it but when I restarted the runtime it went back to CPU for some reason. It works (with some caveats as you said), so this PR looks done to me.

I will add a warning ng message for cuosqp on load to make clear that is an experimental feature. Since you have already worked through it, would be great if you could add a review to the PR, thanks!

I don't have write access either (although you may have it now?). First thing on Monday, I will ask Niko to approve it so it can be finally merged. Also, please remember the docs! Something along the lines of

OSQP: osqp | cuosqp
OSQP is an efficient open source solver for Quadratic Programs. It is self-contained and can be installed via pip. Alternatively, cuOSQP provides a cuda-enabled implementation.

under installation.rst would suffice, I guess.

OK, I have write access now so feel free to merge it.

carrascomj

LGTM.

cdiener · 2021-03-09T01:07:51Z

Failing now due to missing packages for swiglpk 5.0.0 for Mac and some security issue with pyyaml which is not a dependency so no idea where that comes from.

cdiener · 2021-03-12T18:18:23Z

Okay nice, at that point the only thing missing is to remove Python 3.5 from the CI. I'll send a separate PR for that.

KristianJensen

Looks good to me. I agree that it is a useful addition to the solver set.

cdiener · 2021-03-16T18:18:01Z

Ups, that was a bit too early. This needed additional fixes to make the CI work. I'll send another PR.

cdiener and others added 22 commits October 12, 2020 12:24

add osqp interface

1d64600

first functional version

206a2c3

complete functionality

54fe5fb

add tests

f1ea437

make compatible with sympy

77f6287

try to up numpy version to avoid problems

b286adb

remove skip for fixed issue

d4165b9

additional fixes

39afef2

make pickling superfast

613e0fd

change parameter defaults for better stability

c740e58

update default params

caed9bd

using the dual solution to populate shadow prices (instead of reduced…

6e633bc

… costs)

fix: adding the _objective_offset to the objective value

0ca13da

fix regressions

10ccd72

enable more agressive solution recycling

b39d9a7

support cuosqp as well

fd02b71

faster access to constraint primals, return dual values of variable c…

9b80b0b

…onstraints

remove merge statements

a48046e

remove merge statements

0f40532

enable missing features

1494a94

add osqp to tox

92bf883

add osqp to tox

4d5364d

cdiener requested review from Midnighter and KristianJensen January 11, 2021 17:33

carrascomj reviewed Mar 4, 2021

View reviewed changes

carrascomj approved these changes Mar 8, 2021

View reviewed changes

cdiener added 2 commits March 8, 2021 16:36

mention OSQP and the docs, show warning for cuosqp and pep8 fixes

e773852

choose osqp compatible with Python 2.7{

77b92be

cdiener force-pushed the feature/osqp branch from 1fcf4e4 to 77b92be Compare March 9, 2021 01:12

KristianJensen approved these changes Mar 16, 2021

View reviewed changes

carrascomj merged commit 217d98f into opencobra:devel Mar 16, 2021

carrascomj mentioned this pull request Mar 16, 2021

Feature/coin or cbc interface #213

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add QP solver through OSQP #225

Add QP solver through OSQP #225

cdiener commented Dec 11, 2020

cdiener commented Feb 13, 2021

KristianJensen commented Feb 23, 2021

carrascomj left a comment

carrascomj Mar 4, 2021

cdiener Mar 4, 2021

carrascomj Mar 4, 2021

cdiener Mar 4, 2021

carrascomj Mar 4, 2021

cdiener Mar 4, 2021

carrascomj Mar 4, 2021 •

edited

Loading

cdiener Mar 4, 2021 •

edited

Loading

carrascomj Mar 5, 2021 •

edited

Loading

cdiener Mar 5, 2021

carrascomj Mar 5, 2021

carrascomj Mar 8, 2021

carrascomj left a comment

cdiener commented Mar 9, 2021

cdiener commented Mar 12, 2021

KristianJensen left a comment

cdiener commented Mar 16, 2021

Add QP solver through OSQP #225

Add QP solver through OSQP #225

Conversation

cdiener commented Dec 11, 2020

TODO

cdiener commented Feb 13, 2021

KristianJensen commented Feb 23, 2021

carrascomj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carrascomj Mar 4, 2021 • edited Loading

Choose a reason for hiding this comment

cdiener Mar 4, 2021 • edited Loading

Choose a reason for hiding this comment

carrascomj Mar 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carrascomj left a comment

Choose a reason for hiding this comment

cdiener commented Mar 9, 2021

cdiener commented Mar 12, 2021

KristianJensen left a comment

Choose a reason for hiding this comment

cdiener commented Mar 16, 2021

carrascomj Mar 4, 2021 •

edited

Loading

cdiener Mar 4, 2021 •

edited

Loading

carrascomj Mar 5, 2021 •

edited

Loading