Logging Memory Consumption
回答済みHello,
So, I am trying to do this performance comparison between two MILP models for the same problem, the parameters I want to work with are time and memory (runtime) consumption for both the models.
For time, that data is easily accessible using the gurobi log files however, for memory I am a bit confused. I know I can use htop on my system to get the data but I really want to know if there is a way to add memory column in the gurobi log file itself? just like we have a column for time in seconds?
I am using gurobi v12 and I do my optimizations using gurobi_cl. But even if there is a way to do it using gurobipy, it will be helpful.
Thanks in advance.
-
You can measure memory usage using some other tool: e.g. https://github.com/astrofrog/psrecord.
With this you can even get a nice plot with memory consumption as your python script runs:
psrecord "gurobi_cl model.mps" --include-child --interval 0.1 --log activity.txt --plot plot.png
Cheers,
David0 -
Hey David, thank you for your response, this is really helpful.
0 -
HI Mahnoor,
Here's another way using callbacks (see also Callback Codes)
import sys
from functools import partial
import gurobipy as gp
from gurobipy import GRB
def mycallback(model, where, *, cbdata):
if where != GRB.Callback.POLLING:
runtime = model.cbGet(GRB.Callback.RUNTIME)
memused = model.cbGet(GRB.Callback.MEMUSED)
cbdata[runtime] = memused
memusage = {}
with gp.read(sys.argv[1]) as m:
callback_func = partial(mycallback, cbdata=memusage)
m.optimize(callback_func)
print(memusage)Cheers,
David1 -
Thank you so much for the updated response David. Both the methods are very useful for my current application. Cheers!
0 -
Hello,
Just an addition to this thread which I think others might find helpful too.
So based on the two methods suggested by David above, I ran my simulations. Both of them do a great job, however, for someone who is looking to compare the performance of different MILP models based on the internal memory usage during a MILP optimization (MIP iterations), the callback method is more efficient.
psrecord
does the job too, but it externally records the memory usage of a running Gurobi optimization process, so the events are system-defined rather than solver-defined. This means that callback logging aligns with solver events (e.g., MIP iterations, node exploration, simplex steps), whilepsrecord
logs at fixed time intervals regardless of what the solver is doing.I hope my conclusion is correct, will appreciate any additions/ corrections.
Thank you.
1
サインインしてコメントを残してください。
コメント
5件のコメント