Add Linear Regression Constraint Constr with some constant features and some optimization variable features
Answered
Hi. I'm testing out this functionality for the first time - https://gurobi-machinelearning.readthedocs.io/en/stable/api/LinearRegressionConstr.html#module-gurobi_ml.sklearn.linear_regression
To try to simplify the problem, my LR model is trained on 3 features: [torque, water, days].
Water is a real optimization variable that I want to manipulate. Torque and Days are actually constant values I'm grabbing from a data sheet and plugging into the model. To do this, I defined the following variables and constraints.
water = model.addVar(vtype=GRB.INTEGER, name="Water", lb=0)
torque = model.addVar(vtype=GRB.INTEGER, name="Torque", lb=0)
days = model.addVar(vtype=GRB.INTEGER, name="Days", lb=0)
define_days= model.addConstr(days == 30)
define_torque = model.addConstr(torque == 1943.079429314926)
input_gurobi_var_feats = [torque, water, days]
output_gurobi_var = [pct_yield_ta]
lr_pred_constr = add_linear_regression_constr(gp_model= model,
linear_regression=regressions['Linear Regression'],
input_vars=input_gurobi_var_feats,
output_vars=output_gurobi_var)
My LR model is trained properly with the same number of features. My model failed with the ilp as follows. I have also attached the ilp, mlp, log, and sol files. Help on how to set this up would be greatly appreciated! I am on a bit of a time crunch to figure this out as well.
-
Sorry - the support website isn't letting me attach the ilp, log, sol, mps files?
0 -
Darn ok, putting in the LR model directly with its coefficients gives me the same exact error, so something else must be wrong in my problem setup, but I don't know how to interpret the output of the IIS file. I have a dictionary for optimization constants with floats as the values. Note that water = gp.Var here.
lr_yield = model.addConstr(pct_yield_ta == 1.01e-3 * optimization_constants['Torque'] + 1.54e1 * water + 1.22e-3 *optimization_constants['Days'] + 86.7)
model.update()0 -
In the ILP file you have that the variable `Torque_Mill_1_TA` should take an integer value but it is equal to 1943.07, a contradiction. That is why the model is infeasible.
As in your initial description you say that Torque and days are fixed values they shouldn't have integer requirements when you declare them in Gurobi.
0 -
Hi, yes I realized my mistake later. Thank you so much for your help. I actually have a more challenging follow-up now. I am now trying to compare outputs of my model when I plug in Linear Regression vs. Random Forest Regression. My RF model has much lower error, and I was hoping to plug it into my Gurobi model. However, it seems to be giving results that we "don't expect" as opposed to LR, which gives more interpretable outputs.
For example, in my problem, I have 2 water flows (variables Imbibition Water TA and Imbibition Water TB) and I expect my optimizer to be maximizing both because that should be maximizing the objective as well. My optimizer with LR models does indeed maximize both these variables, but my optimizer with RF models does not and is perhaps getting stuck in a local optima. I tried warm starting Imbibition Water TA and Imbibition Water TB at their max values, but that did not help me find the global optima either. I assume it has to do with the nonlinear model having nondifferentiable points that mess up the gradient search from the optimizer? Would appreciate some advice here if possible!
0 -
Hi,
To be sure, when you talk about the results for your optimizer with RF model, you are talking after the RF has been computed with sciki-learn.
Is your problem solved to an optimal solution by Gurobi? Gurobi is not a local optimizer and is not doing gradient search so there shouldn't be any local optima issue in Gurobi. It should be a property of your RF estimator that you get a better output with input that is not maximizing both variables (is the output of your RF estimator the objective value?)
0
Please sign in to leave a comment.
Comments
5 comments