Gurobi v11 outperforms Gurobi v12 for some models

Hi everyone,

I've recently been very impressed by how fast Gurobi v12 is for non-convex MIQCP problems. Out of curiosity, I wanted to compare its performance with Gurobi v11 on some models. Here is one of my test cases in Python:

import numpy as np
import time
from scipy.linalg import circulant
import gurobipy as gp
from gurobipy import GRB

def slackngon(n):
    X  = circulant(np.cos(np.pi/n)-np.cos(np.pi/n + 2*np.pi*np.arange(0,n,1)/n))
    for i in range(n):
        for j in range(n):
            if X[i,j]>1e-6: X[i,j]=1.0
            else:X[i,j]=0.0
    return X

def rc(X,r):
    start_time=time.time()
    m, n  = X.shape
    model = gp.Model()

    model.params.nonconvex = 2
    W = model.addMVar((m, r), vtype=GRB.BINARY)
    H = model.addMVar((r, n), vtype=GRB.BINARY)
    Z = model.addMVar((m ,n), vtype=GRB.BINARY)
 
    model.addConstr((W @ H)/r <= Z)
    model.addConstr(Z <= W @ H)
    model.addConstr(Z == X)
    
    model.optimize()
    end_time=time.time()

    if model.status==2:
        print(f"OPTIMAL FOUND - ({m}x{n}), r={r} - time: {end_time-start_time} seconds")
        return W.X, H.X, Z.X
    if model.status==3:
        print(f"OPTIMAL NOT FOUND - ({m}x{n}), r={r} - time: {end_time-start_time} seconds")
        return [], [], []

rc(slackngon(15),7)
rc(slackngon(15),6)

With Gurobi 11, I got

OPTIMAL FOUND - (15x15), r=7 - time: 2.185544967651367 seconds
OPTIMAL NOT FOUND - (15x15), r=6 - time: 85.52518367767334 seconds

With Gurobi 12, I got

OPTIMAL FOUND - (15x15), r=7 - time: 9.460731744766235

And for r=6, I had to kill the kernel after 2 hours without any result.

After testing a few other models, I noticed that for feasibility problems with binary variables (especially when the problem is infeasible), Gurobi 11 can sometimes be significantly faster than Gurobi 12.

Any thoughts on why this might be happening?

Thanks!

| Version | mean | median | min | max | std |
|:----------|--------:|---------:|---------:|--------:|--------:|
| 11.0.3 | 2.83528 | 2.29737 | 0.518183 | 8.32261 | 2.22336 |
| 12.0.1 | 2.88264 | 2.69979 | 0.113792 | 8.18384 | 1.98267 |

Riley Clement

Gurobi Staff

March 12, 2025 05:40

Hi Timothy,

For the r=7 I ran 30 values of Seed (see How can I make accurate comparisons? if you are unfamiliar with why we do this). Summary stats for runtime are below:

I don't see a difference between v11 and v12. I do see a large difference across Seeds though, with the max runtime being 16-70x slower than the fastest. My guess is you got unlucky with the default value of Seed, and if you were to change it you may see v12 much faster than v11.

For r=6 the infeasibility is determined in the tree, not during presolve and this does open the door for substantial differences. It does seem like v11 is faster to determine infeasibility than v12. It may be a consequence of tweaking internal tolerances between versions to reduce false positives. I'll send the model over to our development team and see what they say. I have tried a few parameter sets to try and help but so far no magic bullet.

- Riley

March 18, 2025 08:11

Regarding the r=6 infeasible model, our development team has looked into it and we narrowed it down to a particular change in our code. There doesn't look to be a bug causing this, and the change was in general an improvement for our extensive benchmark set, meaning this model is one of the unlucky ones to experience worse performance under v12.

Any questions, just let me know.

Gurobi v11 outperforms Gurobi v12 for some models

Comments

Didn't find what you were looking for?