Multiprocessing with gurobi
回答済みI am using gurobi with python multiprocessing. But my code will stuck under fork method sometimes. I create the model and globalize it, then use model.copy() in each process. I wonder if it is an issue caused by copy? Is there still anything shared by the original model and copied model?
import multiprocessing as mp import gurobipy as gp
global model def solve_model(input_data): global model
sub_model = model.copy()
objVar = sub_model.getVarByName(...)
sub_model.addConstr(...)
sub_model.setObjective(objVar, grb.GRB.MINIMIZE)
sub_model.update()
sub_model.optimize() if __name__ == '__main__':
global model
model = gp.Model() with mp.Pool() as pool: pool.map(solve_model, [input_data1, input_data2, input_data3]
-
Hi Keyu,
Thanks for posting here! In your example, all copies of the model live in the same (default) Gurobi environment, which is not thread-safe. Each model should live in its own, dedicated Gurobi environment. That can be done by constructing a new environment in your solve_model function and passing that as an argument to model.copy().
However, my other concern is whether the approach with making the model a global variable works as expected. When using multiprocessing, Python already has to make its own copy of the model variable for each child process and I doubt if that copy operation is going to work on a Gurobi model.
If the changes to the various copies of the model are minimal, you might consider using the "Multiple scenario" feature of Gurobi instead. Also, if constructing the model takes a significant amount of time, we could look into making that faster which might remove the need to work on model copies?
Kind regards,
Ronald0
サインインしてコメントを残してください。
コメント
1件のコメント