Understanding Gurobi's Runtime Distribution

Hello,

I am using Gurobi to solve the mathematical model contained in a .mps file (which is too long to add here as a code). The model is a linear program for L1 regression, consisting of 500 constraints and 1,005 continuous variables (5 unbounded, 1,000 constrained to ≥ 0). I solve it using GurobiPy in Python, with the solver set to Primal Simplex and presolve turned off, though I have observed similar behavior with Dual Simplex and the Barrier Method and presolve turned on.

When solving the same .mps file repeatedly (e.g., 200,000 times) on a supercomputer, I observe three distinct clusters of runtime data, which I believe are caused by variations in computational resources at the supercomputer center. However, within each cluster, the run times appear to follow a normal distribution, suggesting that Gurobi itself may solve the same linear program in a time that is normally distributed. I attached two pictures illustrating what have been described in this paragraph.

Could you help me understand what aspects of Gurobi’s solver behavior (e.g., memory allocation, warm starts, thread management) might contribute to this run time distribution? Additionally, do you recommend any solver settings or configurations that could help reduce variability?

I have attached the run time graphs to this post. At the end, I also add the code that I send to the mentioned supercomputer center.

Best regards,
Botan Citil, Ph.D.

try:
    import numpy as np
    import pandas as pd
    import math
    import copy
    import random
    from pandas import ExcelWriter
    from pandas import ExcelFile
    from gurobipy import *
    import gurobipy as gp
    from gurobipy import GRB
    import datetime
    from datetime import datetime
    import sys
    import os
    import time
    #from openpyxl import load_workbook
    import numpy as np
    from os import listdir
    from os.path import isfile, join
except Exception as e:
    print(e)

runTimeList=[]
#--------Manuel model parameters
modelName="TestModel500"
replicationSize=200000
replication=1
method=1
warmStart=0
preSolve=0
methodText=f"PrimalSimplex{warmStart}{preSolve}"
#--------
print("Method: ", method)
for i in range(replicationSize):
    L1Model = gp.read(f"{modelName}.mps")
    #-------- SET MODEL PARAMETERS -------
    L1Model.Params.LogToConsole=0
    L1Model.Params.LPWarmStart=warmStart
    L1Model.Params.Presolve=preSolve
    L1Model.setParam('Method', method)  # Setting LP solver method to Automatic See for more: https://www.gurobi.com/documentation/current/refman/method.html
    #print(f'Solver is set to {methodText}')
    #--------------------------------------
    L1Model.optimize()
    print("Model runtime: ", L1Model.RunTime)
    runTimeList.append(L1Model.RunTime)

        
dfToAppend = pd.DataFrame ({'runTimeList': runTimeList})

# Get today's date
date_yymmdd = datetime.now().strftime('%y%m%d')

writer = pd.ExcelWriter(f'{date_yymmdd}Experiments{modelName}Method{methodText}RepSize{replicationSize}Rep{replication}.xlsx' ,\
                        engine='xlsxwriter')
dfToAppend.to_excel(writer, index=True, sheet_name='ExperimentStatistics')
writer.close()

David Torres Sanchez

Gurobi Staff

February 13, 2025 19:17
Edited

Hi Botan,

These runtimes are so very small. Have you tried running the same experiment without solving the model? Also, what version of Gurobi are you using? We resolved a related issue in version 12.

Since you are using simplex and this problem solves very quickly, there's probably minimal memory allocation, no multi-threading

Cheers,
David

Botan Citil

February 17, 2025 20:06

Hi David,

Thank you for your message. I am using Gurobi version 11.0. I have not yet run the experiment without solving the model, but I can test that and compare the runtimes.

My main interest is understanding the variance in solving time for the same linear program. Specifically, I want to determine whether it is normal to observe some variation in solving time when running the same model on the same computers. I have also noticed that variances are more prevalent and chaotic as the model size increases. Below, I have attached a screenshot containing a graph that records the solve time of a linear program with 14,993 constraints and 29,991 continuous variables (5 are unbounded, and 29,986 are constrained to ≥ 0).

Best,
Botan

February 19, 2025 10:22
Edited

This could be the effects of performance variability: What is performance variability?

But a few ingredients are missing from the script:

Setting the Seed parameter for every iteration
Disposing of the model and environment: How do I manage Gurobi environments in gurobipy?

with gp.Env() as env, gp.read(filename, env=env) as model:
    # set seed, optimize, ...

Riley Clement

February 19, 2025 22:42

Some notes on variation in solve time under identical conditions:

At Gurobi we maintain several large clusters of machines for benchmarking and tuning purposes. We are not only interested in determinism of the solution paths, but minimal variance in runtime across identical machines running the same version of Gurobi and model. We went through a cluster upgrade last year and there was some difficulty in finding machines that could provide us the consistency we were looking for. With identical machines we could guarantee identical solution paths but even consistent timings were difficult to obtain with many candidate machines with factors such as thermal effects on CPUs to turbo boosting coming into play.

For very small, easy models, that require very consistent timing you end up needing to set all sorts of system parameters to minimize performance variations across machines and across time - using numactl to pin threads to NUMA codes, running OS threads on separate cores to handle interrupts, p-state/c-state transmissions to maximize clock rate and more.

- Riley

Understanding Gurobi's Runtime Distribution

Comments

Didn't find what you were looking for?