Formulation problem time
Answered
Hello again!
The time from the start of the program to the appearance of optimization in the log is about 20 minutes. Does this mean that the problem formulation takes 20 minutes? If so, how can this process be accelerated? What things can be improved on my server?
Best regards,
Andrey
-
Hi Andrey,
From what you describe, it sounds like it is indeed the model generation that takes so long. You should profile your model generation to better investigate what is going on. A starting point would be to measure the time of model construction and print the needed time just before the optimization begins
import gurobipy as gp
from gurobipy import GRB
#[...]
import time
start_time = time.time()
# construct model
# ...
#
end_time = time.time()
print("=======\ntime needed to construct model: %f seconds\n=======\n"%(end_time-start_time))
model.optimize()Once you are sure that it is model construction that takes so long, you can start measuring particular parts of your code to find the culprit.
Best regards,
Jaromił0 -
Thank you!
It helped me to find a time-consuming block.
Do I understand correctly that the general scheme is as follows?
- Problem construction;
- Sending it to the Gurobi server;
- Solving on the Gurobi server;
- Sending solution back to the user's machine;
- Solution generation (meaning the possibility to extract solution via .X attribute);
0 -
Yes, the general scheme is correct.
The only detail that is different is that the solution is only send back to user's machine if it is actually requested by the user via, e.g., the X attribute.
Best regards,
Jaromił0 -
Since the solving of the problem occurs on the Gurobi server, then does it no matter which server the user has? Is it only important that the user's server allows quickly constructing the problem?
Best regards,
Andrey
0 -
Since the solving of the problem occurs on the Gurobi server, then does it no matter which server the user has? Is it only important that the user's server allows quickly constructing the problem?
If by "user server" you mean the user's machine, then yes. It is important to have a strong "solution server machine" to achieve best solving performance on this particular machine. Additionally, the internet connection can have an impact on communication speed, i.e., transfer of the model data.
For the user's machine it is often enough to be a notebook or similar, since only model construction has to be performed on it and it mostly depends on one's programming skills and not the underlying CPU (of course a good CPU does not hurt).
Best regards,
Jaromił0 -
Thanks.
Can you please answer one more question?
I noticed that the CPU usage was around 100% during the problem construction, which means that all CPUs were working to their fullest. But during the solving optimization, CPU usage reached almost 1000%. What could it mean?Best regards,
Andrey
0 -
Which OS are you using? Usually, e.g., on Linux or Mac, 100% CPU usage means that 1 thread is fully utilized and 1000% CPU usage would mean that 10 threads are currently fully utilized. This is the best fitting scenario for your case because model construction usually uses only 1 thread (unless you are explicitly using multiple threads during model construction) and the model solution process tries to utilize all available threads.
On Windows, you can see the utilization of each thread in the task manager.
Best regards,
Jaromił0 -
Thanks, the process became more transparent for me.
Yes, my OS is Linux. Where does the use of my machine's threads go during solving the problem, if solving the problem occurs on the Gurobi server?
Best regards,
Andrey
0 -
Where does the use of my machine's threads go during solving the problem, if solving the problem occurs on the Gurobi server?
Oh, I thought you were working on your local machine. After the problem is sent to the compute server, there should be only the common CPU load. Are you sure that the model is solved on the compute server and not on your local machine?
Best regards,
Jaromił0 -
The order of execution of commands is as follows:
- The virtual environment activation with gurobipy;
- Exporting file license;
- Running the program with the model.
In this case, is the problem solving on the local machine?
Best regards,
Andrey
0 -
It depends on the license you are using.
Do you have access to a floating site license or are you using an individual academic license?
Could you share the first 20-30 lines of the log output when you are solving a model?
Best regards,
Jaromił0 -
Yes, I'm using an individual academic license.
Best regards,
Andrey
0 -
Ok, this explains the behavior. With an individual academic license, the solution process is performed on your machine. In order to be able to deploy the solution process to a server, you need access to a floating site license and a server machine. You could ask your academic institute whether they have access to such a license already. If not, they can contact Gurobi Support to request one.
Best regards,
Jaromił0 -
Do I understand correctly there is no difference between an individual academic license and a floating site license, except that the user could use a Gurobi server machine with a floating site license?
Best regards,
Andrey
0 -
Yes, for academic use that is the difference. You can find more information at Gurobi's Academic Program and Licenses.
Best regards,
Jaromił0
Please sign in to leave a comment.
Comments
15 comments