Logging Memory Consumption

Answered

Mahnoor Naseer

January 24, 2025 13:52 Edited

• 
• 
• 

Hello,

So, I am trying to do this performance comparison between two MILP models for the same problem, the parameters I want to work with are time and memory (runtime) consumption for both the models.

For time, that data is easily accessible using the gurobi log files however, for memory I am a bit confused. I know I can use htop on my system to get the data but I really want to know if there is a way to add memory column in the gurobi log file itself? just like we have a column for time in seconds?

I am using gurobi v12 and I do my optimizations using gurobi_cl. But even if there is a way to do it using gurobipy, it will be helpful.

Thanks in advance.

Related to

• Memory

0

• 

  David Torres Sanchez

  • Gurobi Staff

  January 24, 2025 17:14 Edited

  You can measure memory usage using some other tool: e.g. https://github.com/astrofrog/psrecord.

  With this you can even get a nice plot with memory consumption as your python script runs:

  psrecord "gurobi_cl model.mps" --include-child --interval 0.1 --log activity.txt --plot plot.png

  Cheers, 
  David

  0
• 

  Mahnoor Naseer

  January 24, 2025 17:50

  • 
  • 
  • 

  Hey David, thank you for your response, this is really helpful.

  0
• 

  David Torres Sanchez

  • Gurobi Staff

  January 24, 2025 19:25 Edited

  HI Mahnoor,

  Here's another way using callbacks (see also Callback Codes)

  import sys
  from functools import partial
  
  import gurobipy as gp
  from gurobipy import GRB
  
  
  def mycallback(model, where, *, cbdata):
      if where != GRB.Callback.POLLING:
          runtime = model.cbGet(GRB.Callback.RUNTIME)
          memused = model.cbGet(GRB.Callback.MEMUSED)
          cbdata[runtime] = memused
  
  
  memusage = {}
  with gp.read(sys.argv[1]) as m:
      callback_func = partial(mycallback, cbdata=memusage)
      m.optimize(callback_func)
  
  print(memusage)

  Cheers, 
  David

  2
• 

  Mahnoor Naseer

  January 27, 2025 19:16

  • 
  • 
  • 

  Thank you so much for the updated response David. Both the methods are very useful for my current application. Cheers!

   

   

  0
• 

  Mahnoor Naseer

  February 07, 2025 13:12

  • 
  • 
  • 

  Hello,

  Just an addition to this thread which I think others might find helpful too.

  So based on the two methods suggested by David above, I ran my simulations. Both of them do a great job, however, for someone who is looking to compare the performance of different MILP models based on the internal memory usage during a MILP optimization (MIP iterations), the callback method is more efficient.

  psrecord does the job too, but it externally records the memory usage of a running Gurobi optimization process, so the events are system-defined rather than solver-defined. This means that callback logging aligns with solver events (e.g., MIP iterations, node exploration, simplex steps), while psrecord logs at fixed time intervals regardless of what the solver is doing.

  I hope my conclusion is correct, will appreciate any additions/ corrections.

   

  Thank you. 
  
  
  

  1

Please sign in to leave a comment.