Skip to main content

Delayed Result Printing in Gurobi Optimization (Python)

Answered

Comments

6 comments

  • Mario Ruthmair
    Gurobi Staff Gurobi Staff

    Hi Amir,

    Do you get additional output after the log snippet you showed in your message?

    It seems you are solving a continuous problem, probably a linear problem. When the barrier algorithm finishes, it may have found a (near-)optimal objective value, but the corresponding solution may have some violations left and even may be slightly sub-optimal. Additionally, the solution is usually not a basic solution.

    Therefore, the so-called crossover phase follows the barrier to clean up the solution and moves it to a corner of the polytope, i.e., transforms it to a basic solution. This can take a considerable amount of time in some cases.

    If you are already satisfied with the solution you get immediately after barrier, then you can switch off the crossover phase with parameter Crossover=0.

    Best regards,
    Mario

    0
  • Amir Fayaz Heidari
    Conversationalist
    First Question

    Thanks for your response. My model is quadratic, continuous, and convex. I use the barrier method to solve the optimization problem.

    I set the parameters below for solving the optimization problem:

        model.Params.Crossover = 0
        model.Params.Method = 2
        model.Params.BarHomogeneous = 0
        model.Params.BarConvTol = 1e-6
        model.optimize()

    As you see, I set the crossover parameter at 0. Although I got the message that says "the barrier method solved model" and also the optimal objective is shown after 9 hours, it takes an additional 8 hours to print the optimal values of the variables I need. 

    Here is the output I got:

     

    Set parameter TokenServer to value "lic.hpc.unimelb.edu.au"
    Set parameter Crossover to value 0
    Set parameter Method to value 2
    Set parameter BarHomogeneous to value 0
    Set parameter BarConvTol to value 1e-06
    Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)

    CPU model: Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz, instruction set [SSE2|AVX|AVX2|AVX512]
    Thread count: 72 physical cores, 72 logical processors, using up to 32 threads

    Optimize a model with 66129016 rows, 41131802 columns and 186920724 nonzeros
    Model fingerprint: 0xb4489054
    Model has 12902080 quadratic objective terms
    Coefficient statistics:
      Matrix range     [7e-07, 1e+03]
      Objective range  [6e-02, 1e+12]
      QObjective range [1e-05, 3e+01]
      Bounds range     [0e+00, 0e+00]
      RHS range        [1e-01, 9e+11]
    Warning: Model contains large objective coefficients
    Warning: Model contains large rhs
             Consider reformulating model or setting NumericFocus parameter
             to avoid numerical issues.
    Presolve removed 0 rows and 0 columns (presolve time = 8s) ...
    Presolve removed 9056872 rows and 4702406 columns (presolve time = 10s) ...
    Presolve removed 22813842 rows and 7872700 columns (presolve time = 22s) ...
    Presolve removed 22813842 rows and 7872700 columns (presolve time = 25s) ...
    Presolve removed 22813842 rows and 7872705 columns (presolve time = 31s) ...
    Presolve removed 22813842 rows and 7872705 columns (presolve time = 40s) ...
    Presolve removed 22821137 rows and 7880000 columns (presolve time = 43s) ...
    Presolve removed 30838479 rows and 15897342 columns (presolve time = 47s) ...
    Presolve removed 30883153 rows and 15942016 columns (presolve time = 51s) ...
    Presolve removed 30883153 rows and 15942016 columns (presolve time = 58s) ...
    Presolve removed 30931549 rows and 15942016 columns (presolve time = 64s) ...
    Presolve removed 30931549 rows and 15942016 columns (presolve time = 66s) ...
    Presolve removed 30931549 rows and 15942016 columns (presolve time = 71s) ...
    Presolve removed 30979933 rows and 15942016 columns (presolve time = 80s) ...
    Presolve removed 30979933 rows and 15942016 columns (presolve time = 81s) ...
    Presolve removed 30979933 rows and 15942016 columns (presolve time = 88s) ...
    Presolve removed 30979933 rows and 15942016 columns (presolve time = 90s) ...
    Presolve removed 31463856 rows and 15942016 columns (presolve time = 98s) ...
    Presolve removed 31463856 rows and 15942016 columns
    Presolve time: 100.82s
    Presolved: 34665160 rows, 25189786 columns, 113041249 nonzeros
    Presolved model has 9232849 quadratic objective terms
    Elapsed ordering time = 5s
    Elapsed ordering time = 10s
    Elapsed ordering time = 15s
    Ordering time: 19.79s

    Barrier statistics:
     Dense cols : 1065
     AA' NZ     : 1.945e+08
     Factor NZ  : 1.067e+09 (roughly 30.0 GB of memory)
     Factor Ops : 1.158e+11 (roughly 3 seconds per iteration)
     Threads    : 32

                      Objective                Residual
    Iter       Primal          Dual         Primal    Dual     Compl     Time
       0  -1.62067448e+22  1.62063773e+22  1.43e+12 8.06e+07  1.73e+14   488s
       1  -1.29996564e+22  1.29993172e+22  1.28e+12 7.19e+07  1.54e+14   539s
       2  -1.20395165e+22  1.20391872e+22  1.23e+12 6.92e+07  1.48e+14   594s
       3  -1.05710291e+22  1.05707162e+22  1.15e+12 6.47e+07  1.39e+14   669s
       4  -9.66351561e+21  9.66321353e+21  1.10e+12 6.17e+07  1.33e+14   726s

    ...

    ...

    ...

     387   6.40622057e+11  8.81885950e+11  1.31e-03 1.54e-08  6.79e-15  32695s
     388   6.40622115e+11  8.81885950e+11  1.32e-03 1.49e-08  6.97e-15  32809s

    Barrier solved model in 388 iterations and 32809.52 seconds (9049.73 work units)
    Optimal objective 6.40622115e+11

    /data/gpfs/projects/punim1865/Code/Output_Market_Cournot_planning_Energy_inertia_with_BESS_2.py:2628: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
      fig = plt.subplots(figsize =(3.5 , 3))

     

    Also, I attempted to retrieve the variable values without generating their corresponding figures. Nevertheless, this process incurred some time delay.

    0
  • Amir Fayaz Heidari
    Conversationalist
    First Question

    Mario, could you please address my problem?

     

    Thanks in advance

    0
  • Mario Ruthmair
    Gurobi Staff Gurobi Staff

    Please do not create support tickets and further threads in the community forum if we do not answer immediately to your forum questions. This just leads to more work on our side. We give the license for free to academics, but individual support is not included. As you can see, we still answer most of the questions in the community forum, but probably with a delay.

    To your case:

    • Your model has 41M variables, so retrieving the values of those variables might take some time. Of course, 8 hours is far too long.
    • How do you retrieve the solution values? Could you post the corresponding code snippets? Maybe there is a more efficient way of doing it.
    0
  • Amir Fayaz Heidari
    Conversationalist
    First Question

    M=6

    R=5

    Y=20

     

    Variable_1 = model.addVars(M , R , Y , vtype=GRB.CONTINUOUS)

     

    Output_1 = np.zeros((M,R,Y))

    for m in range(M):

          for r in range(R):

                for y in range(Y):

                      Output_1[m,r,y] = Variable_1[m,r,y].x

    0
  • Mario Ruthmair
    Gurobi Staff Gurobi Staff

    Instead of retrieving the value for each single variable in for loops, you could try the following:

    `Output_1 = model.getAttr('X', Variable_1)`

    This should be more efficient.

    Are you retrieving any further results than the variable values?
    If the delay of 8h happens after optimize() finishes, you can analyze where exactly the time is spent in your code by printing timestamps between the lines etc.

    0

Please sign in to leave a comment.