How to add indicator constraints when using gurobipy pandas
Answered
I find gurobipy pandas very efficient in model building especially when the input data is very large and takes the form of a pandas data frame. Is there a way to write indicator constraints with gurobipy pandas, similar to what we have "gppd.add_constrs()" ? At present, I am converting indicator constraints into big M formulation and then using "gppd.add_constrs()".
Thank you,
Bhartendu
-
Hi Bhartendu,
It is certainly is possible. Did you perhaps want to provide a concrete example with code or a math formulation? Or we can just make one up to demonstrate.
- Riley
0 -
Thanks Riley for the prompt response.
model = gp.Model("")
ind = pd.Series(range(500))
bin_varb = gppd.add_vars(model, ind, vtype="B")
int_varb = gppd.add_vars(model, ind, vtype="I")
# I want to express the below relationship
# if bin_varb[i] == 0 => int_varb[i] <= 0 for i in 0 .. 499
# I am doing like below
big_M = 1000
constr = gppd.add_constrs(
model, int_varb - (big_M * bin_varb), GRB.LESS_EQUAL, 0
)# with gurobipy, I would do like below, but would have to loop through all indices
for i in range(500):
model.addGenConstrIndicator(bin_varb[i], 0, int_varb[i] <= 0)I am looking to express the above via indicator constraints with gurobipy pandas.
Thanks
0 -
Hi Bhartendu,
There is no built-in method in gurobipy-pandas which can add multiple indicator constraints at once. What this means is that the approach will be more "pandas with gurobipy" rather than "gurobipy-pandas".
So, we have to add each indicator constraint one by one, but there are many ways we could do this. Purely for academic reasons I set about trying as many as I could think of and collecting timing results using Python's timeit module.This is the setup code:
import timeit
import pandas as pd
setup = """
import gurobipy as gp
from gurobipy import GRB
import gurobipy_pandas as gppd
import pandas as pd
model = gp.Model("")
ind = pd.Series(range(500))
bin_varb = gppd.add_vars(model, ind, vtype="B")
int_varb = gppd.add_vars(model, ind, vtype="I")
# combining series into a single dataframe
df = pd.concat([bin_varb, int_varb], axis=1)
"""
def min_runtime(stmt):
return min(timeit.repeat(
stmt,
setup=setup,
repeat=100,
number=1,
))The min_runtime function is defined so that we run each possible approach 100 times and take the minimum (minimum runtimes are important when comparing approaches to remove variability by other processes on the machine).
These are all the different ways we can add indicator constraints, with the final result being a pandas.Series of the constraints. The code is expressed in strings as this is what timeit requires.zip_addConstr_code = """
pd.Series(
[model.addConstr((b == 0) >> (i <= 0))
for b,i in zip(bin_varb, int_varb)],
index = ind,
)
"""
zip_addGenConstrIndicator_code = """
pd.Series(
[model.addGenConstrIndicator(b, 0, i, GRB.LESS_EQUAL, 0)
for b,i in zip(bin_varb, int_varb)],
index = ind,
)
"""
df_apply_expr_apply_addConstr = """
df.apply(lambda r: (r[0] == 0) >> (r[1] <= 0), axis=1).apply(model.addConstr)
"""
df_apply_addConstr = """
df.apply(lambda r: model.addConstr((r[0] == 0) >> (r[1] <= 0)), axis=1)
"""
df_apply_addGenConstrIndicator = """
df.apply(lambda r: model.addGenConstrIndicator(r[0], 0, r[1], GRB.LESS_EQUAL, 0), axis=1)
"""
df_itertuples_addConstr = """
pd.Series(
(model.addConstr((r._1 == 0) >> (r._2 <= 0))
for r in df.itertuples()),
index=df.index,
)
"""
df_itertuples_addGenConstrIndicator = """
pd.Series(
[model.addGenConstrIndicator(r._1, 0, r._2, GRB.LESS_EQUAL, 0)
for r in df.itertuples()],
index=df.index,
)
"""
df_iterrows_addConstr = """
pd.Series(
(model.addConstr((r[0] == 0) >> (r[1] <= 0))
for _, r in df.iterrows()),
index=df.index,
)
"""
df_iterrows_addGenConstrIndicator = """
pd.Series(
(model.addGenConstrIndicator(r[0], 0, r[1], GRB.LESS_EQUAL, 0)
for _, r in df.iterrows()),
index=df.index,
)
"""
Then a pandas.Series can be created by running each of these approaches through the min_runtime function, and then dividing by the time of the fastest approach:results = pd.Series(
{
string_name: min_runtime(eval(string_name))
for string_name in (
"zip_addGenConstrIndicator_code",
"zip_addConstr_code",
"df_apply_expr_apply_addConstr",
"df_apply_addConstr",
"df_apply_addGenConstrIndicator",
"df_itertuples_addConstr",
"df_itertuples_addGenConstrIndicator",
"df_iterrows_addConstr",
"df_iterrows_addGenConstrIndicator",
)
}
)
results_normalized = (results/results.min()).sort_values()
Then, results_normalized looks like this, and represent the performance of each approach as a multiple of the fastest approach found (zip_addGenConstrIndicator_code):zip_addGenConstrIndicator_code 1.000000
df_itertuples_addGenConstrIndicator 1.062170
zip_addConstr_code 1.663099
df_itertuples_addConstr 1.742310
df_apply_expr_apply_addConstr 2.001572
df_apply_addGenConstrIndicator 2.001766
df_apply_addConstr 2.852161
df_iterrows_addGenConstrIndicator 5.110845
df_iterrows_addConstr 6.337511
dtype: float64
So what can we say about this?- Using addGenConstrIndicator is faster than addConstr. This is not surprising since addConstr handles multiple types of constraints and this incurs an overhead to facilitate this.
- Using itertuples is much faster than iterrows. This is also not surprising since itertuples is well known to be much faster than iterrows (in general, not specifically to this code).
- The fastest approach is to just pair the variables up with the zip function, but the approach with itertuples is not too far behind, so "the best" approach might be a matter of taste in this instance.- Riley
1 -
Part 2: A more complicated example
Say that we have binary variables \(x_{i,k}\) and \(z_{k,n}\), which share a common index \(k\), a vector of constants \(C_k\), and we want the following indicator constraints:\[x_{i,k} = 0 \rightarrow \sum_{n}z_{k,n} \leq C_k, \quad\quad \forall i,k\]
Setup:
import gurobipy as gp
from gurobipy import GRB
import gurobipy_pandas as gppd
import numpy as np
import pandas as pd
i = (1,2)
k = (1,2,3)
n = ("a","b")
model = gp.Model()
x = gppd.add_vars(model, pd.MultiIndex.from_product((i,k), names=("i", "k")), vtype=GRB.BINARY, name="x")
z = gppd.add_vars(model, pd.MultiIndex.from_product((k,n), names=("k", "n")), vtype=GRB.BINARY, name="z")
C = pd.Series(
np.random.randint(1,10, size=len(k)),
index=pd.Index(k, name="k"),
)Note that
z.groupby("k").sum()will give us linear expressions which sum up z variables for each k indice:
k
1 z[1,a] + z[1,b]
2 z[2,a] + z[2,b]
3 z[3,a] + z[3,b]
Name: z, dtype: objectWe can then join this with x and C:
df = x.to_frame().join(z.groupby("k").sum()).join(C.to_frame("C"))to give
x z C
i k
1 1 <gurobi.Var x[1,1]> z[1,a] + z[1,b] 3
2 <gurobi.Var x[1,2]> z[2,a] + z[2,b] 2
3 <gurobi.Var x[1,3]> z[3,a] + z[3,b] 4
2 1 <gurobi.Var x[2,1]> z[1,a] + z[1,b] 3
2 <gurobi.Var x[2,2]> z[2,a] + z[2,b] 2
3 <gurobi.Var x[2,3]> z[3,a] + z[3,b] 4
We can then add the indicator constraints using the itertuples & addGenConstrIndicator approach:pd.Series(
[model.addGenConstrIndicator(r.x, 0, r.z, GRB.LESS_EQUAL, r.C)
for r in df.itertuples()],
index=df.index,
)
- Riley1 -
WOW ! Thanks Riley for such a detailed explanation. Everything is crystal clear now.
I will definitely use the above patterns in my live projects.Thanks again,
Bhartendu0
Please sign in to leave a comment.
Comments
5 comments