Out of memory error while running a large MIP problem
進行中Hi,
I am trying to solve a MIP model in Python using the Gurobi package, but I am getting an Out of Memory error.
I have already set the parameters bellow:
Set parameter TimeLimit to value 3600
Set parameter NodefileStart to value 1
Set parameter PreSparsify to value 2
Set parameter Presolve to value 0
Set parameter MIPFocus to value 3
Set parameter Threads to value 15
Following is the log information:
Gurobi Optimizer version 11.0.1 build v11.0.1rc0 (linux64 - "CentOS Linux 7 (Core)")
CPU model: Intel(R) Xeon(R) CPU E7- 2870 @ 2.40GHz, instruction set [SSE2]
Thread count: 20 physical cores, 20 logical processors, using up to 15 threads
Optimize a model with 765630 rows, 702268 columns and 2347653656 nonzeros
Model fingerprint: 0xda7fb1ae
Variable types: 3261 continuous, 699007 integer (690581 binary)
Coefficient statistics:
Matrix range [1e+00, 3e+05]
Objective range [1e+00, 1e+00]
Bounds range [1e+00, 1e+00]
RHS range [1e+00, 5e+02]
Presolve removed 0 rows and 0 columns (presolve time = 253s) ...
Presolve removed 0 rows and 0 columns (presolve time = 289s) ...
Presolve removed 0 rows and 0 columns (presolve time = 589s) ...
Variable types: 2461 continuous, 699807 integer (691381 binary)
Explored 0 nodes (0 simplex iterations) in 1812.22 seconds (368.36 work units)
Thread count was 1 (of 20 available processors)
Solution count 0
Solve interrupted (error code 10001)
Best objective -, best bound -, gap -
Are there any tips or suggestions on how to deal with the Out of Memory error?
Thank you for your help!
0
-
正式なコメント
-
Gurobi Staff
This post is more than two years old. Some information may not be up to date. For current information, please check the Gurobi Documentation or Knowledge Base. If you need more help, please create a new post in the community forum, or try Gurobot, our chatbot interface offering instant, expert-level support. -
-
-
Hi Rainne,
It seems that your model is very dense, there are more than 2.3 billions non-zeroes in the matrix.
Your machine needs a substantial amount of memory to handle this.Here are a few comments on the solver parameters you set:
- NodeFileStart=1: This would only help when the solver reaches the branch-and-bound phase and only if many thousands of open branch-and-bound nodes can be written to disk while waiting to be handled.
- Presolve=0: For large models like this, I strongly recommend keeping presolving enabled to reduce the model. You could even consider more aggressive presolving with Presolve=2.
- Threads=15: The model needs to be copied for each thread, so you might first try with Threads=1 and see whether you can save memory with this. Then, increase to 2 or 4 to see if it still fits into memory.
Best regards,
Mario0 -
-
Hi Mario,
Thank you for your tips!
When I keep presolve enabled, it cannot find a feasible solution considering the time limit setted.
Do you have any other suggestions that I can try to apply to my model?
Set parameter TimeLimit to value 3600
Set parameter NodefileStart to value 0.5
Set parameter PreSparsify to value 2
Set parameter Presolve to value 2
Set parameter MIPFocus to value 3
Set parameter Threads to value 1
CPU model: Intel(R) Xeon(R) CPU E7- 2870 @ 2.40GHz, instruction set [SSE2]
Thread count: 20 physical cores, 20 logical processors, using up to 1 threads
Optimize a model with 760140 rows, 696802 columns and 2308170820 nonzeros
Model fingerprint: 0xc006377b
Variable types: 3261 continuous, 693541 integer (685181 binary)
Coefficient statistics:
Matrix range [1e+00, 3e+05]
Objective range [1e+00, 1e+00]
Bounds range [1e+00, 1e+00]
RHS range [1e+00, 5e+02]
Presolve removed 0 rows and 0 columns (presolve time = 248s) ...
Presolve removed 0 rows and 0 columns (presolve time = 282s) ...
Presolve removed 0 rows and 0 columns (presolve time = 312s) ...
Presolve removed 0 rows and 1 columns (presolve time = 394s) ...
Presolve removed 0 rows and 1 columns (presolve time = 424s) ...
Presolve removed 0 rows and 1 columns (presolve time = 467s) ...
Presolve removed 0 rows and 1 columns (presolve time = 497s) ...
Presolve removed 3540 rows and 1771 columns (presolve time = 743s) ...
Presolve removed 3540 rows and 1771 columns (presolve time = 827s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 857s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 899s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 929s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1061s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1084s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1114s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1156s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1186s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1241s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1323s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1724s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1883s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 1966s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2033s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2035s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2040s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2045s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2050s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2055s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2060s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2065s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2070s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2075s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2080s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2085s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2090s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2095s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2100s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2105s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2110s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2115s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2120s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2125s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2130s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2135s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2140s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2145s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2150s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2155s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2160s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2165s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2170s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2175s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2180s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2185s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2190s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2195s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2200s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2205s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2210s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2215s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2220s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2225s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2230s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2235s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2240s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2245s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2250s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2255s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2260s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2265s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2270s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2275s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2280s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2285s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2290s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2295s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2300s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2305s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2310s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2315s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2320s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2325s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2658s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2789s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 2902s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3015s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3124s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3125s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3130s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3135s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3140s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3145s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3150s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3155s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3160s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3165s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3170s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3175s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3180s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3185s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3190s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3195s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3200s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3205s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3210s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3215s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3220s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3225s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3230s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3235s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3240s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3245s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3250s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3255s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3260s) ...
Presolve removed 108522 rows and 1771 columns (presolve time = 3560s) ...
Presolve removed 108522 rows and 1771 columns
Presolve time: 3559.70s
Explored 0 nodes (0 simplex iterations) in 3892.70 seconds (2601.27 work units)
Thread count was 1 (of 20 available processors)
Solution count 00 -
-
-
Hi Rainne,
With parameters
Set parameter PreSparsify to value 2 Set parameter Presolve to value 2
you enabled aggressive presolving, which takes too much time for your model.
What happens if you keep presolving at the default, i.e., remove those two settings?Best regards,
Mario0 -
-
Hi Mario,
It still cannot find a feasible solution considering the time limit setted.
Set parameter TimeLimit to value 3600
Set parameter NodefileStart to value 0.5
Set parameter MIPFocus to value 3
Set parameter Threads to value 1
CPU model: Intel(R) Xeon(R) CPU E7- 2870 @ 2.40GHz, instruction set [SSE2]
Thread count: 20 physical cores, 20 logical processors, using up to 1 threads
Optimize a model with 765630 rows, 702268 columns and 2347653656 nonzeros
Model fingerprint: 0xda7fb1ae
Variable types: 3261 continuous, 699007 integer (690581 binary)
Coefficient statistics:
Matrix range [1e+00, 3e+05]
Objective range [1e+00, 1e+00]
Bounds range [1e+00, 1e+00]
RHS range [1e+00, 5e+02]
Presolve removed 0 rows and 0 columns (presolve time = 251s) ...
Presolve removed 0 rows and 0 columns (presolve time = 286s) ...
Presolve removed 0 rows and 0 columns (presolve time = 317s) ...
Presolve removed 0 rows and 1 columns (presolve time = 401s) ...
Presolve removed 0 rows and 1 columns (presolve time = 431s) ...
Presolve removed 0 rows and 1 columns (presolve time = 474s) ...
Presolve removed 0 rows and 1 columns (presolve time = 505s) ...
Presolve removed 3540 rows and 1771 columns (presolve time = 757s) ...
Presolve removed 3540 rows and 1771 columns (presolve time = 844s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 874s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 917s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 947s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1084s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1107s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1137s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1179s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1210s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1263s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1346s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1750s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1913s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 1997s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2066s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2070s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2075s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2080s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2085s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2090s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2095s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2100s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2105s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2110s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2115s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2120s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2125s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2130s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2135s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2140s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2145s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2150s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2155s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2160s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2165s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2170s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2175s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2180s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2185s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2190s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2195s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2200s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2205s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2210s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2215s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2220s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2225s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2230s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2235s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2240s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2245s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2250s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2255s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2260s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2265s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2270s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2275s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2280s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2285s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2290s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2295s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2300s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2305s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2310s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2315s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2320s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2325s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2330s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2335s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2340s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2345s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2350s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2355s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2360s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2365s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2370s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2703s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2837s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 2952s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3066s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3177s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3180s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3185s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3190s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3195s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3200s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3205s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3210s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3215s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3220s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3225s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3230s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3235s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3240s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3245s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3250s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3255s) ...
Presolve removed 103302 rows and 1771 columns (presolve time = 3260s) ...
Presolve removed 103302 rows and 1771 columns
Presolve time: 3260.30s
Explored 0 nodes (0 simplex iterations) in 3600.71 seconds (2248.80 work units)
Thread count was 1 (of 20 available processors)
Solution count 0
Time limit reached
Best objective -, best bound -, gap -0 -
-
-
Hi Rainne,
This model seems to be tough.
However, I also oversaw an important point: MIPFocus=3 is a meta-parameter and will automatically set Presolve=2. If you want to keep MIPFocus=3 (you should also try 1 and 2), then you could overrule it with Presolve=1 or PrePasses=1.0 -
投稿コメントは受け付けていません。
コメント
6件のコメント