Parameter for defining a memory limit (as a stopping criteria, like a time limit)
Answered
I just want to confirm if it really does not exist a memory limit parameter like CPLEX (tree memory limit) that makes the optimizer stop with special status when reached (akin defining a time limit, which is possible in Gurobi).
My use case is a experiment for a scientific paper. I am not buying more memory, as I am a PhD student ( (so is not my responsibility buying for a single computer), and if I bought memory for one computer of the cluster I am using then I would need to buy for them all. I am also not enabling NodeFiles, because the disk should not interfere in the experiment (no other paper in the literature does this). I cannot change the model to use less memory because I am reproducing a model from the literature, and it needs to be faithful to its description. I do not care if I do not get any solution at all, I just want a memory limit, so Gurobi does not end up killing the whole process or other processes that are running, because they tried allocating memory when Gurobi left basically no free memory.
For now, it is not viable to use Gurobi for my experiment because there any many instances in which the model I am reproducing allocates all memory and this kills the process in a way that I cannot check the status nor capture an exception (if I could capture the exception at least I could work around it). The whole experiment script is just halted and I need to check on it constantly, and re-run it from the next instance, what is not trivial.
-
Hi Henrique,
I can confirm Gurobi does not have such a parameter. Also please note that normally Gurobi does not kill the process but your operating system does. If Gurobi runs out of memory, it will throw an out-of-memory exception with the corresponding error code. If the operating system kills the process because of high memory usage and because the restrictions you or your admins have set in your system, there is not much Gurobi can do about it.
If you are rebuilding something from a paper, what kind of architecture did they use?
Best,
Sonja
0 -
Hi Henrique,
I looked for this feature, but could not find something similar to CPLEX TreLim (tree memory limit). As this post backs 2 years ago, I am wondering has Gurobi added this feature to the newer versions?
Thanks,
Masoud
0 -
Hi Masoud and Henrique,
Gurobi versions >=9.5.0 supports a new parameter called MemLimit which allows the user to limit the total amount of memory available to Gurobi. If the Gurobi Optimizer needs more memory, it will fail with the error message \(\texttt{OUT_OF_MEMORY}\).
Best regards,
Maliheh
0 -
Thanks Maliheh, for the response.
I think MemLim (limits the total amount of memory available to Gurobi) from Gurobi is the equivalent for WorkMem in CPLEX (specifies an upper limit on the amount of central memory, in megabytes, that CPLEX is permitted to use for working files). On the other hand, TreLim sets an absolute upper limit on the size (in megabytes, uncompressed) of the branch-and-cut tree. Hence, I think Gurobi has not added a feature to control the memory used only by branch-and-cut tree.
What do you think?
Thanks,
Masoud
0 -
Hi Masoud,
Thank you for your comments.
The Gurobi MemLimit parameter is not just applicable to MIP problems. It is applicable to continuous problems too. For example, if the total amount of memory required by Gurobi exceeds the MemLimit parameter value during the Barrier or concurrent algorithms, Gurobi will fail with an out-of-memory error.
Gurobi by default uses as much memory as it needs. One reasonable use-case for setting the MemLimit parameter is to protect other processes on the same machine from being killed because of an out-of-memory error.
I looked at the definitions of WorkMem and MIP_Limits_TreeMemory in the CPLEX documentation. To me, a combination of CPLEX WorkMem and CPLEX MIP_Strategy_File parameters does the same thing that the Gurobi NodeFileStart parameter does. The CPLEX documentation says:
When tree storage size exceeds the limit defined by WorkMem, and if the tree-memory limit has not been exceeded, then what happens next is decided by the setting of the node storage file switch. If that parameter is set to zero, then optimization proceeds with the tree stored in memory until CPLEX reaches the tree memory limit. If the node file indicator is set to 1 (the default), then a fast compression algorithm is used on the nodes to try to conserve memory, without resorting to writing the node files to disk. If the parameter is set to 2, then node files are written to disk. If the parameter is set to 3, then nodes are both compressed (as in option 1) and written to disk (as in option 2).
Therefore, based on my understanding, both CPLEX WorkMem and MIP_Limits_TreeMemory parameters are not exactly the same as the Gurobi MEMLimit parameter. The CPLEX MIP_Limits_TreeMemory will stop the CPLEX as soon as the size of the tree (on disk and in memory) exceeds that limit and will report the best solution.
Sorry for the confusion. Please let me know if something does not make sense.
Best regards,
Maliheh
0 -
Thanks, Maliheh, for the explanation.
I believe NodeFileStart does what I am looking for.
Thanks,
Masoud
0
Please sign in to leave a comment.
Comments
6 comments