Fast creation of Gurobi models

Answered

Ahmed Abbas

February 02, 2023 15:10 Edited

• 
• 
• 

Hi, 

I have many ILP instances which I need to load in Python repeatedly(each instance more than a 100 times) as Gurobi models. Is there some way that I can speed-up the loading process to Gurobi? A possible way I can think of:

1. Extract the ILP description out of Gurobi as python objects (e.g. as numpy arrays).
2. Save the numpy arrays to disk.
3. Load numpy arrays in parallel by multiprocessing.
4. Build Gurobi model efficiently from the numpy arrays. 

Step 1, 2 will be done only once and can be slow. Step 3 and 4 need to be done repeatedly thus should be efficient. Is there some functionality to do Step 1 and 4 efficiently?

Alternative solution: Allow gurobi models to be pickled (ref: Support for pickle in the python API (google.com)

Ahmed

Related to

• Model Files
• Matrix API

0

• 

  Jaromił Najman

  • Gurobi Staff

  February 03, 2023 08:51

  Hi Ahmed,

  I have a couple of questions regarding your issue.

  I have many ILP instances which I need to load in Python repeatedly(each instance more than a 100 times)

  Does this mean that you have to solve the exactly same instance multiple times? If this is the case, instead of solving the same instance again, I would propose to save the fingerprint of the instance together with all other attributes and values that you need and then check via the fingerprint attribute whether you solved this exact instance before already. In order to avoid always having to reload the instance to get the fingerprint, you could have a mapping saying that "if you have to read this ILP then check vs this fingerprint first".

  If the above is not applicable, you might want to try the following.

  Are the models very big or would it be possible to store all of the models in memory? If memory is not an issue, you could just keep all models in memory and check against the fingerprint as described above.

  If memory is an issue then extracting the models to specific data structures makes sense. You could extract all necessary information as Python objects. For example, you could use the getA() method to directly get get a sparse matrix representation of the constraint matrix. Then, you would also need to extract all variable attributes and save those in some data structures which can be read in quickly. You can then rebuild the model by adding all constraints via the addMConstr() method. Please note that this does not work for quadratic and general constraints.

  Alternative solution: Allow gurobi models to be pickled

  We will consider allowing to pickle a given Gurobi model. Now, instead of pickling the whole gurobi model, you might try pickling the data structure you extracted from the model.

  Best regards, 
  Jaromił

  0
• 

  Ahmed Abbas

  February 21, 2023 11:34 Edited

  • 
  • 
  • 

  Hi Jaromił,
  
  Thanks for your reply. Sorry for late reply, I was having some licensing issues which are fixed now.
  
  I need to solve same ILP but with different objective vector. I think your suggestion of converting to numpy is possible for me. To this end I have created a small example to first export the model to numpy and then create the gurobi model from it. I have created the following MVP, could you please check if it looks ok? 
  
  

  import gurobipy as gp
  from gurobipy import GRB
  import numpy as np
  import pickle
  
  def solve_model(input_data):
      with gp.Env() as env, gp.Model(env=env) as model:
          (model_sense, c, A, rhs, con_sense, lb, ub, vtype, varnames) = input_data
          model.ModelSense = model_sense
          model.addMVar((len(varnames), ), lb=lb, ub=ub, obj=c, vtype=vtype, name=varnames)
          model.addMConstr(A, None, con_sense, rhs)
          model.optimize()
  
  ilp_path = 'some_example.lp'
  model = gp.read(ilp_path)
  A = model.getA()
  model_sense = model.ModelSense
  cons_sense = np.array(model.getAttr("Sense", model.getConstrs()))
  rhs = np.array(model.getAttr("rhs", model.getConstrs()))
  lb = np.array(model.getAttr("LB", model.getVars()))
  ub = np.array(model.getAttr("UB", model.getVars()))
  vtype = np.array(model.getAttr("Vtype", model.getVars()))
  c = np.array(model.getAttr("Obj", model.getVars()))
  varnames = model.getAttr("VarName", model.getVars())
  input_data = (model_sense, c, A, rhs, cons_sense, lb, ub, vtype, varnames)
  # Save to disk
  pickle.dump(input_data, open('temp_model.pkl', "wb"))
  # Load from disk
  input_data = pickle.load(open('temp_model.pkl', "rb"))
  solve_model(input_data)

  Thanks,

  Ahmed

  
  
  

  0
• 

  Jaromił Najman

  • Gurobi Staff

  February 27, 2023 08:53

  Hi Ahmed,

  The code looks fine.

  Best regards, 
  Jaromił

  0

Please sign in to leave a comment.