Running time increase when running multiple models

Answered

Assia Kamal-idrissi

• May 03, 2024 17:01
• Edited

Hello,

I'm running the same model for different instances files in separated files independently using Jupyter. For each instance, I run the model 5 times with time limit 3600 for each iteration. So, in the worst case, it can be 5 houres to get the average runtime. The problem that the running time increase when I run multiple files. Here is the result for both scenario  for the same model and same instance. The running time passes from 197 to 1129.

Scenario 1: solo run

Explored 1416944 nodes (47643692 simplex iterations) in 196.46 seconds (339.24 work units)
Thread count was 32 (of 112 available processors)

Solution count 10: 12083.6 12083.6 12101.5 ... 12512.9

Optimal solution found (tolerance 1.00e-04)
Best objective 1.208360025332e+04, best bound 1.208360025332e+04, gap 0.0000%
Optimal Travel Time: 12083.600253319368
Running Time: 196.4777159690857
run time average CVRP:  197.4301966190338
Travel Time  12083.600253319368

Scenario 2: run multiple file at once

Explored 1416944 nodes (47643692 simplex iterations) in 988.68 seconds (339.24 work units)
Thread count was 32 (of 112 available processors)

Solution count 10: 12083.6 12083.6 12101.5 ... 12512.9

Optimal solution found (tolerance 1.00e-04)
Best objective 1.208360025332e+04, best bound 1.208360025332e+04, gap 0.0000%
Optimal Travel Time: 12083.600253319368
Running Time: 988.7430329322815
run time average CVRP:  1129.1102116584777
Travel Time  12083.600253319368

Related to

• VRP

0

• 

  Dan Steffy

  Gurobi Staff

  • May 03, 2024 17:22

  Hello Assia,

  It looks like when running multiple jobs in parallel, you are oversubscribing the usage of the CPU. You can see a clue when comparing the time and work done between the two runs:

  196.46 seconds (339.24 work units)

  988.68 seconds (339.24 work units)

  The second run did the same amount of work but simply took more time. This suggests that you had several processes running simultaneously and that the number of threads they were collectively using was larger than the number of cores on the machine. This situation causes some threads to sit idle, waiting for turns to be scheduled on one of the CPU cores.

  Thread count was 32 (of 112 available processors)

  I suggest setting the Gurobi Threads parameter to a lower value for your runs. This will restrict the number of threads used by each Gurobi process. You could experiment with different values, but a good starting point would be to set Threads to a value such that the number of Threads used by each Gurobi process, multiplied by the number of processes you will run concurrently, is no larger than the number of available CPU cores on the machine you are using.

  I hope this helps,

  -Dan

  0

Please sign in to leave a comment.