Memory Issue During Optimization with Gurobi

Answered

Matteo Catania

• June 13, 2024 15:14

I am currently encountering a memory-related issue while using Gurobi for optimization in Python. Specifically, I have observed that a seemingly less computationally intensive model is running out of memory during the matrix reordering process, whereas a more complex problem with higher variables does not encounter any memory problems.

Here are the details:

1. Problem Description:

   • I am working on an optimization problem using Gurobi.
   • The less complex model (with fewer variables) unexpectedly exhausts memory during the matrix reordering phase.
   • Conversely, a more complex problem (with more variables) does not exhibit this issue.
   • Both versions of the problem have similar constraints.

2. Key Difference:

   • The primary difference between the two versions lies in how they represent time.
   • The less complex model uses a year represented by 12 days, while the more complex problem uses a year represented by 24 days.

3. Environment:

   • Gurobi version: 10.0.0
   • Operating system: Windows (64-bit)
   • CPU: Intel Core i9-10980XE @ 3.00GHz (18 physical cores, 36 logical processors)

Here the images of the logs:

the code with 24 days

the code with 12 days which exhausted available memory

Any insights, suggestions, or guidance would be greatly appreciated. Thank you in advance for your assistance!

Related to

• Memory

0

• 

  Maliheh Aramon

  Gurobi Staff

  • June 14, 2024 02:47

  Hi Matteo,

  • Do you get an explicit out-of-memory error code with error number 10001 for the 12-day model? The log only shows that the ordering took longer for the 12-day model.  
  • What is the memory capacity of your machine? 
  • Do you run both models in parallel?

  The BarOrder parameter controls the fill-reducing algorithm used for the constraint matrix A. There are two settings with 0 being computationally cheaper and 1 being computationally expensive. You can try BarOrder=0 for the 12-day model to potentially reduce the ordering time. 

  Another general recommendation for managing memory is to experiment with lower values for the Threads parameter. 

  Best regards,

  Maliheh

  0
• 

  Matteo Catania

  • June 24, 2024 14:07

  

  

  Dear Malineh,

  Thank you for your support. Setting BarOrder to 0 resolved the issue within 12 days. However, I have not understood why the optimization stops if I do not set BarOrder to 0, as shown in the image below, which is the error taken directly from Python. I do not believe this is due to a lack of available memory since I have 192 GB of RAM available, and the problem size is less than 2 GB.

  

   

  Best regards,

  Matteo

  0
• 

  Marika Karbstein

  Gurobi Staff

  • June 25, 2024 06:17

  Hi Matteo,

  With Gurobi 10, the memory requirements for the barrier were massively overestimated in some cases. Setting the BarOrder parameter helped here to not run in this overestimation. This issue is fixed with Gurobi 11. If possible, please upgrade.

  Best regards,
  Marika

  0

Please sign in to leave a comment.