Skip to content

Commit

Permalink
Merge pull request #240 from Gurobi/gurobi11
Browse files Browse the repository at this point in the history 
Updates for Gurobi 11
  • Loading branch information
@pobonomo
pobonomo authored Oct 25, 2023
2 parents 266395e + 1363e5e commit d9921bd
Show file tree
Hide file tree
Showing 12 changed files with 130 additions and 195 deletions.
147 changes: 42 additions & 105 deletions docs/notebooks/ipynb/student_admission.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "7eba6234",
"id": "6ce1fcdb",
"metadata": {},
"source": [
"# Student Enrollment\n",
Expand Down Expand Up @@ -80,7 +80,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "c9961b42",
"id": "8d8067c7",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -102,7 +102,7 @@
},
{
"cell_type": "markdown",
"id": "81a96d64",
"id": "03a40109",
"metadata": {},
"source": [
"We now retrieve the historical data used to build the regression from Janos\n",
Expand All @@ -115,7 +115,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "24154c54",
"id": "c29d537f",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -133,7 +133,7 @@
},
{
"cell_type": "markdown",
"id": "9a87740e",
"id": "dcdb4ba6",
"metadata": {},
"source": [
"## Fit the logistic regression\n",
Expand All @@ -145,7 +145,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "bd8a43ce",
"id": "d6524441",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -158,7 +158,7 @@
},
{
"cell_type": "markdown",
"id": "4236c46e",
"id": "85b6498d",
"metadata": {},
"source": [
"### Optimization Model\n",
Expand All @@ -172,7 +172,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "4830ff33",
"id": "86f0976c",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -183,19 +183,19 @@
{
"cell_type": "code",
"execution_count": null,
"id": "47f1ab07",
"id": "7de4d8e6",
"metadata": {},
"outputs": [],
"source": [
"nstudents = 250\n",
"nstudents = 25\n",
"\n",
"# Select randomly nstudents in the data\n",
"studentsdata = studentsdata.sample(nstudents)"
"studentsdata = studentsdata.sample(nstudents, random_state=1)"
]
},
{
"cell_type": "markdown",
"id": "861392fc",
"id": "b037eac7",
"metadata": {},
"source": [
"We can now create the our model.\n",
Expand All @@ -206,7 +206,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "b6222f9f",
"id": "37a642f0",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -216,25 +216,28 @@
"# The y variables are modeling the probability of enrollment of each student. They are indexed by students data\n",
"y = gppd.add_vars(m, studentsdata, name='enroll_probability')\n",
"\n",
"# We add to studentsdata a column of variables to model the \"merit\" feature. Those variable are between 0 and 2.5.\n",
"# They are added directly to the data frame using the gppd extension.\n",
"studentsdata = studentsdata.gppd.add_vars(m, lb=0.0, ub=2.5, name='merit')\n",
"\n",
"# We want to complete studentsdata with a column of decision variables to model the \"merit\" feature.\n",
"# Those variable are between 0 and 2.5.\n",
"# They are added using the gppd extension and the resulting dataframe is stored in\n",
"# students_opt_data.\n",
"students_opt_data = studentsdata.gppd.add_vars(m, lb=0.0, ub=2.5, name='merit')\n",
"\n",
"# We denote by x the (variable) \"merit\" feature\n",
"x = studentsdata.loc[:, \"merit\"]\n",
"x = students_opt_data.loc[:, \"merit\"]\n",
"\n",
"# Make sure that studentsdata contains only the features column and in the right order\n",
"studentsdata = studentsdata.loc[:, features]\n",
"students_opt_data = students_opt_data.loc[:, features]\n",
"\n",
"m.update()\n",
"\n",
"# Let's look at our features dataframe for the optimization\n",
"studentsdata[:10]"
"students_opt_data[:10]"
]
},
{
"cell_type": "markdown",
"id": "83a83662",
"id": "9b91307e",
"metadata": {},
"source": [
"We add the objective and the budget constraint:"
Expand All @@ -243,7 +246,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "1a21833e",
"id": "1df88255",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -255,7 +258,7 @@
},
{
"cell_type": "markdown",
"id": "aba0719c",
"id": "418664ec",
"metadata": {},
"source": [
"Finally, we insert the constraints from the regression. In this model we want to\n",
Expand All @@ -270,29 +273,34 @@
{
"cell_type": "code",
"execution_count": null,
"id": "5868b7ef",
"id": "c5354d86",
"metadata": {},
"outputs": [],
"source": [
"pred_constr = add_predictor_constr(\n",
" m, pipe, studentsdata, y, output_type=\"probability_1\"\n",
" m, pipe, students_opt_data, y, output_type=\"probability_1\"\n",
")\n",
"\n",
"pred_constr.print_stats()"
]
},
{
"cell_type": "markdown",
"id": "1eb51817",
"id": "0c7a14f1",
"metadata": {},
"source": [
"We can now optimize the problem."
"We can now optimize the problem.\n",
"With Gurobi ≥ 11.0, the attribute `FuncNonLinear` is automatically set to 1 by Gurobi machine learning on the nonlinear constraints it adds\n",
"in order to deal algorithmically with the logistic function.\n",
"\n",
"Older versions of Gurobi would make a piece-wise linear approximation of the logistic function. You can refer to [older versions\n",
"of this documentation](https://gurobi-machinelearning.readthedocs.io/en/v1.3.0/mlm-examples/student_admission.html) for dealing with those approximations."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "efffd1f0",
"id": "f141099d",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -301,88 +309,17 @@
},
{
"cell_type": "markdown",
"id": "1f3d5c78",
"id": "cb742bf5",
"metadata": {},
"source": [
"Remember that for the logistic regression, Gurobi does a piecewise-linear\n",
"approximation of the logistic function. We can therefore get some significant\n",
"errors when comparing the results of the Gurobi model with what is predicted by\n",
"the regression.\n",
"\n",
"We print the error using [get_error](../api/AbstractPredictorConstr.rst#gurobi_ml.modeling.base_predictor_constr.AbstractPredictorConstr.get_error) (note that we take the maximal error\n",
"over all input vectors)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af9a2820",
"metadata": {},
"outputs": [],
"source": [
"print(\n",
" \"Maximum error in approximating the regression {:.6}\".format(\n",
" np.max(pred_constr.get_error())\n",
" )\n",
")"
]
},
{
"cell_type": "markdown",
"id": "de8fa4d3",
"metadata": {},
"source": [
"The error we get might be considered too large, but we can use Gurobi parameters\n",
"to tune the piecewise-linear approximation made by Gurobi (at the expense of a\n",
"harder models).\n",
"\n",
"The specific parameters are explained in the documentation of [Functions\n",
"Constraints](https://www.gurobi.com/documentation/9.1/refman/constraints.html#subsubsection:GenConstrFunction)\n",
"in Gurobi's manual.\n",
"\n",
"We can pass those parameters to the\n",
"[add_predictor_constr](../api/AbstractPredictorConstr.rst#gurobi_ml.add_predictor_constr)\n",
"function in the form of a dictionary with the keyword parameter\n",
"`pwd_attributes`.\n",
"\n",
"Now we want a more precise solution, so we remove the current constraint, add a\n",
"new one that does a tighter approximation and resolve the model."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a8706d13",
"metadata": {},
"outputs": [],
"source": [
"pred_constr.remove()\n",
"\n",
"pwl_attributes = {\n",
" \"FuncPieces\": -1,\n",
" \"FuncPieceLength\": 0.01,\n",
" \"FuncPieceError\": 1e-5,\n",
" \"FuncPieceRatio\": -1.0,\n",
"}\n",
"pred_constr = add_predictor_constr(\n",
" m, pipe, studentsdata, y, output_type=\"probability_1\", pwl_attributes=pwl_attributes\n",
")\n",
"\n",
"m.optimize()"
]
},
{
"cell_type": "markdown",
"id": "4b1c7700",
"metadata": {},
"source": [
"We can see that the error has been reduced."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "606d2ba8",
"id": "ac8c7e09",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -395,16 +332,16 @@
},
{
"cell_type": "markdown",
"id": "e802a077",
"id": "22c9c454",
"metadata": {},
"source": [
"Finally note that we can directly get the input values for the regression in a solution as a pandas dataframe using input_values."
"Finally, note that we can directly get the input values for the regression in a solution as a pandas dataframe using input_values."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6b5c180",
"id": "aa7a19dc",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -413,7 +350,7 @@
},
{
"cell_type": "markdown",
"id": "766c54d0",
"id": "a7548099",
"metadata": {
"nbsphinx": "hidden"
},
Expand Down Expand Up @@ -441,7 +378,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
"version": "3.10.0"
},
"license": {
"full_text": "# Copyright © 2023 Gurobi Optimization, LLC\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n# =============================================================================="
Expand Down
Loading

0 comments on commit d9921bd

Please sign in to comment.