Setting the number of threads
Answered
Hi to all,
My question is a general one on setting the number of threads when solving a MIP which takes a very very long time to finish.
First I set the number of threads to 24, which ended up exhausting the memory. Then, I reduced the threads to 18, and now I am resolving the problem.
Is there a generally agreed "good" number of threads to solve a MIP problem?
If the new number of threads does not work, will it be a good idea to set NodefileStart to 0.5 and keeping the number of threads as 18? Or do I have to reduce the number of threads while setting NodefileStart?
I am using Gurobi 811.
Thank you very much.
Ebru
-
In general, it is not possible to say that more threads are going to lead to a faster solving process. There are too many factors influencing the solver's behavior. If you run into memory issues, it can help to use the NodeFile parameter.
And as stated in the documentation:
If you still exhaust memory after setting the
NodefileStartparameter to a small value, you should try limiting the thread count. Each thread in parallel MIP requires a copy of the model, as well as several other large data structures. Reducing the Threads parameter can sometimes significantly reduce memory usage.0 -
Thank you very much for the answer. I am now running the program with NodefileStart = 0.5 and Threads = 24, and waiting for the results. So far the memory usage has not exceeded 5%.
0 -
Hi Matthias,
Before starting my code, I fixed
m.Params.Threads = 24
m.Params.NodefileStart = 0.5.
There are only 120 binary variables and many continuous variables to be optimized.
After 700,000 seconds, I got an error message
GRBaddnodefile failed
error code 10019
What can be the reason of this failure? The mip gap is still 94.2%.
Thanks in advance
Ebru
0 -
Hi Ebru,
Did you verify that the path to the NodeFile is correct and that you have write permissions there (and enough storage)?
Cheers,
Matthias0 -
Thank you Matthias.
0 -
Hi Matthias,
Previously, I havent defined NodeFile, and the default directory grbnodes0 was created on the Desktop. I am the admin of the server, so I have the permission. It seems then that the storage wasnt enough.
My questions are:
1- Is it a better idea to create and define my own NodeFile rather than using the default one? It seems that it does not matter whether I create my own directory or use the default one.
2- If there is a storage problem, will setting NodefileStart to a value greater than 0.5 (e.g., 0.7) help?
3- What exactly setting NodefileStart to 0.5 does?
Cheers
Ebru
0 -
Hi Ebru,
In general, you want to avoid writing a NodeFile at all. You should try to utilize your available memory (RAM) as much as possible and only use the nodefile as a fallback.
It doesn't matter where you put this file in the file system, but be aware that there can be differences in IO performance if you use a "far away" network drive for example.
Please refer to the documentation for an explanation of the parameter:
"When the amount of memory used to store nodes (measured in GBytes) exceeds the specified parameter value, nodes are compressed and written to disk. We recommend a setting of
0.5, but you may wish to choose a different value, depending on the memory available in your machine."In any case, you should upgrade your hardware and also try Gurobi v9 if you want to solve such large problems.
Cheers,
Matthias0 -
Thanks very much Matthias.
0
Please sign in to leave a comment.
Comments
8 comments