Gurobi Post-Solve Memory Error
ユーザーの入力を待っています。Good afternoon.
I've been solving some iterations of a case study and have been having issues with MemoryErrors.
I've had memory issues before, but never in a situation where the error only occurs after the model reaches a 0.00% gap. It's not too clear for me why this is the case. Running the same code sometimes results in this happening, other times it solves without issue.
The log is the following:
Gurobi Optimizer version 10.0.3 build v10.0.3rc0 (win64)
CPU model: 13th Gen Intel(R) Core(TM) i9-13900KF, instruction set [SSE2|AVX|AVX2]
Thread count: 24 physical cores, 32 logical processors, using up to 32 threads
Optimize a model with 7274544 rows, 3516304 columns and 83609190 nonzeros
Model fingerprint: 0x5f202b5e
Variable types: 3126432 continuous, 389872 integer (195474 binary)
Coefficient statistics:
Matrix range [1e-05, 7e+03]
Objective range [1e-01, 2e+06]
Bounds range [1e+00, 1e+00]
RHS range [1e-04, 2e+02]
Presolve removed 3533079 rows and 97741 columns (presolve time = 9s) ...
Presolve removed 6812907 rows and 97741 columns (presolve time = 15s) ...
Presolve removed 6812907 rows and 97741 columns (presolve time = 21s) ...
Presolve removed 6830493 rows and 3276223 columns (presolve time = 25s) ...
Presolve removed 6830493 rows and 3276223 columns (presolve time = 30s) ...
Presolve removed 7020345 rows and 3466591 columns (presolve time = 37s) ...
Presolve removed 7020345 rows and 3466591 columns (presolve time = 41s) ...
Presolve removed 7228275 rows and 3476421 columns (presolve time = 48s) ...
Presolve removed 7233669 rows and 3481438 columns (presolve time = 50s) ...
Presolve removed 7233672 rows and 3481441 columns
Presolve time: 50.66s
Presolved: 40872 rows, 34863 columns, 132575 nonzeros
Variable types: 32514 continuous, 2349 integer (753 binary)
Deterministic concurrent LP optimizer: primal and dual simplex
Showing first log only...
Root simplex log...
Iteration Objective Primal Inf. Dual Inf. Time
0 3.8770904e+07 5.580136e+03 3.359651e+10 65s
18851 3.8818463e+07 1.250809e+03 3.738747e+09 65s
Concurrent spin time: 0.00s
Solved with dual simplex
Root simplex log...
Iteration Objective Primal Inf. Dual Inf. Time
31913 1.7364556e+07 0.000000e+00 0.000000e+00 65s
Root relaxation: objective 1.736456e+07, 31913 iterations, 0.58 seconds (0.79 work units)
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
0 0 1.7365e+07 0 500 - 1.7365e+07 - - 65s
0 0 1.7626e+07 0 813 - 1.7626e+07 - - 67s
0 0 1.7654e+07 0 827 - 1.7654e+07 - - 67s
0 0 1.7654e+07 0 778 - 1.7654e+07 - - 67s
0 0 1.7654e+07 0 787 - 1.7654e+07 - - 67s
0 0 1.7828e+07 0 998 - 1.7828e+07 - - 70s
0 0 1.8221e+07 0 970 - 1.8221e+07 - - 71s
0 0 1.8224e+07 0 1021 - 1.8224e+07 - - 71s
0 0 1.8224e+07 0 1035 - 1.8224e+07 - - 71s
0 0 1.8955e+07 0 1119 - 1.8955e+07 - - 73s
0 0 1.8971e+07 0 1123 - 1.8971e+07 - - 73s
0 0 1.8974e+07 0 1176 - 1.8974e+07 - - 74s
0 0 1.8975e+07 0 1183 - 1.8975e+07 - - 74s
0 0 1.9000e+07 0 1026 - 1.9000e+07 - - 76s
0 0 1.9701e+07 0 1121 - 1.9701e+07 - - 77s
0 0 1.9701e+07 0 1145 - 1.9701e+07 - - 78s
0 0 1.9701e+07 0 1193 - 1.9701e+07 - - 78s
0 0 1.9701e+07 0 1127 - 1.9701e+07 - - 79s
0 0 1.9703e+07 0 1134 - 1.9703e+07 - - 80s
0 0 1.9703e+07 0 1136 - 1.9703e+07 - - 81s
0 0 1.9703e+07 0 1116 - 1.9703e+07 - - 82s
0 0 1.9703e+07 0 1139 - 1.9703e+07 - - 83s
0 0 1.9703e+07 0 1139 - 1.9703e+07 - - 84s
0 2 1.9703e+07 0 1139 - 1.9703e+07 - - 87s
15 28 1.9703e+07 4 1028 - 1.9703e+07 - 932 95s
89 96 1.9703e+07 7 972 - 1.9703e+07 - 1279 100s
185 189 1.9824e+07 8 1087 - 1.9703e+07 - 1108 106s
267 259 1.9825e+07 10 1043 - 1.9788e+07 - 886 110s
403 395 infeasible 12 - 1.9788e+07 - 740 115s
511 504 1.9840e+07 14 986 - 1.9788e+07 - 661 120s
567 534 1.9849e+07 16 987 - 1.9788e+07 - 634 127s
668 677 1.9845e+07 17 976 - 1.9788e+07 - 596 131s
846 869 1.9849e+07 22 815 - 1.9788e+07 - 531 137s
1048 1036 infeasible 31 - 1.9788e+07 - 481 142s
1125 1137 1.9865e+07 31 675 - 1.9788e+07 - 471 145s
1344 1358 1.9869e+07 40 734 - 1.9788e+07 - 439 150s
1587 1627 1.9871e+07 45 704 - 1.9788e+07 - 411 156s
1916 1965 1.9903e+07 58 624 - 1.9788e+07 - 377 162s
2096 2120 1.9982e+07 60 660 - 1.9788e+07 - 362 166s
2257 2277 1.9985e+07 65 550 - 1.9788e+07 - 355 170s
2560 2642 1.9987e+07 72 543 - 1.9788e+07 - 352 178s
2789 2970 2.0023e+07 83 560 - 1.9788e+07 - 342 182s
3123 3353 2.0033e+07 96 477 - 1.9788e+07 - 323 186s
3608 3771 1.9943e+07 46 665 - 1.9788e+07 - 299 190s
4462 4498 1.9832e+07 17 786 - 1.9788e+07 - 269 199s
4877 4848 1.9849e+07 19 748 - 1.9788e+07 - 261 204s
5319 5132 1.9845e+07 19 747 - 1.9788e+07 - 252 208s
5721 5335 1.9863e+07 22 764 - 1.9788e+07 - 246 212s
5990 5521 1.9870e+07 29 750 - 1.9788e+07 - 246 216s
6217 5523 2.0112e+07 39 465 - 1.9788e+07 - 247 223s
6218 5524 2.0076e+07 75 908 - 1.9788e+07 - 247 225s
6221 5526 2.0161e+07 121 1008 - 1.9788e+07 - 247 230s
6225 5529 2.0132e+07 52 1192 - 1.9788e+07 - 247 240s
6229 5531 2.0181e+07 81 927 - 1.9788e+07 - 247 246s
6232 5533 1.9998e+07 109 1039 - 1.9788e+07 - 246 250s
6236 5541 1.9788e+07 13 983 - 1.9788e+07 - 12.0 256s
6260 5573 1.9809e+07 16 1120 - 1.9804e+07 - 16.7 261s
6284 5597 1.9811e+07 17 1102 - 1.9808e+07 - 23.6 265s
6380 5655 1.9821e+07 20 1002 - 1.9808e+07 - 31.6 270s
6489 5727 1.9818e+07 21 1057 - 1.9808e+07 - 40.2 276s
6603 5807 1.9837e+07 24 971 - 1.9808e+07 - 49.2 281s
6704 5866 1.9843e+07 25 940 - 1.9808e+07 - 54.6 285s
6850 5968 1.9843e+07 28 935 - 1.9808e+07 - 62.7 291s
6912 6005 1.9849e+07 29 911 - 1.9808e+07 - 66.4 297s
6973 6056 1.9851e+07 30 893 - 1.9808e+07 - 68.4 300s
7215 6202 1.9855e+07 32 905 - 1.9808e+07 - 78.3 311s
7399 6331 1.9860e+07 36 793 - 1.9808e+07 - 83.5 316s
7637 6502 1.9868e+07 44 703 - 1.9808e+07 - 90.2 322s
7785 6532 1.9876e+07 47 699 - 1.9808e+07 - 92.9 332s
7870 6627 1.9889e+07 48 717 - 1.9808e+07 - 95.7 336s
8185 6846 1.9888e+07 52 757 - 1.9808e+07 - 102 342s
8380 6989 1.9890e+07 58 706 - 1.9808e+07 - 105 346s
8628 7094 1.9904e+07 59 669 - 1.9808e+07 - 107 350s
8839 7168 1.9912e+07 62 662 - 1.9808e+07 - 110 377s
8994 7322 1.9967e+07 63 605 - 1.9808e+07 - 111 381s
9223 7485 1.9930e+07 67 560 - 1.9808e+07 - 114 385s
9834 7830 1.9954e+07 74 619 - 1.9808e+07 - 118 393s
10188 7906 1.9975e+07 79 591 - 1.9808e+07 - 121 398s
10438 8025 2.0003e+07 85 553 - 1.9808e+07 - 124 402s
10699 8174 1.9997e+07 92 575 - 1.9808e+07 - 126 406s
10999 8267 2.0049e+07 94 423 - 1.9808e+07 - 129 411s
11292 8359 2.0014e+07 101 527 - 1.9808e+07 - 131 416s
11561 8471 infeasible 106 - 1.9808e+07 - 133 423s
11855 8515 1.9822e+07 20 1069 - 1.9808e+07 - 136 428s
12093 8594 1.9826e+07 22 988 - 1.9808e+07 - 139 441s
12309 8720 1.9884e+07 25 922 - 1.9808e+07 - 142 446s
12571 8827 1.9829e+07 25 1042 - 1.9808e+07 - 144 452s
12812 8947 1.9856e+07 29 952 - 1.9808e+07 - 147 458s
13050 9093 1.9867e+07 31 859 - 1.9808e+07 - 150 465s
13351 9184 1.9873e+07 35 900 - 1.9808e+07 - 153 471s
13649 9192 1.9889e+07 41 862 - 1.9808e+07 - 155 564s
H13748 8882 1.085201e+08 1.9808e+07 81.7% 155 564s
H13782 7533 2.027969e+07 1.9808e+07 2.32% 156 633s
H13794 7019 2.020326e+07 1.9808e+07 1.95% 156 633s
H13811 6544 2.015376e+07 1.9808e+07 1.71% 156 633s
13819 6837 1.9899e+07 45 827 2.0154e+07 1.9808e+07 1.71% 156 642s
14239 6904 2.0030e+07 57 704 2.0154e+07 1.9808e+07 1.71% 159 650s
14563 6923 2.0132e+07 76 522 2.0154e+07 1.9811e+07 1.70% 162 701s
H14604 5529 2.004228e+07 1.9811e+07 1.16% 163 701s
H14737 4804 2.001912e+07 1.9811e+07 1.04% 166 701s
14773 4831 1.9826e+07 19 889 2.0019e+07 1.9811e+07 1.04% 166 710s
14972 4988 1.9842e+07 22 839 2.0019e+07 1.9811e+07 1.04% 171 718s
15243 5178 1.9848e+07 28 939 2.0019e+07 1.9811e+07 1.04% 174 727s
15615 5378 1.9872e+07 37 870 2.0019e+07 1.9811e+07 1.04% 176 736s
H15729 2762 1.990529e+07 1.9811e+07 0.48% 176 736s
15987 2388 cutoff 55 1.9905e+07 1.9814e+07 0.46% 178 745s
16231 2396 1.9830e+07 19 974 1.9905e+07 1.9827e+07 0.40% 182 757s
16532 2328 1.9841e+07 22 796 1.9905e+07 1.9827e+07 0.40% 186 767s
16902 2384 1.9847e+07 26 758 1.9905e+07 1.9827e+07 0.40% 189 779s
17276 2429 1.9902e+07 33 869 1.9905e+07 1.9829e+07 0.38% 193 790s
17664 2602 1.9867e+07 24 885 1.9905e+07 1.9829e+07 0.38% 196 803s
H18031 2039 1.985367e+07 1.9829e+07 0.12% 198 803s
H18050 2039 1.985366e+07 1.9829e+07 0.12% 198 803s
18160 1344 cutoff 46 1.9854e+07 1.9829e+07 0.12% 199 816s
19014 1161 1.9851e+07 22 622 1.9854e+07 1.9833e+07 0.10% 195 829s
19176 1161 1.9845e+07 26 740 1.9854e+07 1.9833e+07 0.10% 195 830s
*19421 1161 78 1.984499e+07 1.9833e+07 0.06% 194 830s
21080 6 1.9841e+07 26 976 1.9845e+07 1.9836e+07 0.05% 182 839s
21210 5 1.9836e+07 21 830 1.9845e+07 1.9836e+07 0.05% 181 840s
Cutting planes:
Learned: 2
Gomory: 16
Cover: 212
Implied bound: 657
Projected implied bound: 10
Clique: 29
MIR: 1199
StrongCG: 2
Flow cover: 12937
Flow path: 669
Inf proof: 14
Zero half: 56
Network: 116
RLT: 31
Relax-and-lift: 1000
PSD: 1
Explored 21238 nodes (5465367 simplex iterations) in 842.42 seconds (487.71 work units)
Thread count was 32 (of 32 available processors)
Solution count 10: 1.9845e+07 1.98537e+07 1.98537e+07 ... 1.0852e+08
Optimal solution found (tolerance 1.00e-04)
Best objective 1.984498735603e+07, best bound 1.984498735603e+07, gap 0.0000%
MemoryError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File C:\Program Files\Python39\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
exec(code, globals, locals)
File c:\scenarioreducer-main\new\scenarioreducer-main\mpinv_30_11_cvar_beta1_alpha99.py:2990
solver.solve(model, tee=True)
File ~\AppData\Roaming\Python\Python39\site-packages\pyomo\solvers\plugins\solvers\direct_solver.py:168 in solve
result = self._postsolve()
MemoryError
Any idea how to fix this? The issue is not background apps taking memory as I use the machine only to run the code. I've restarted both the IDE and the computer several times. Using Spyder and Pyomo. Previously I was saving the solution to a variable, and I deleted it as I thought that might be the issue, however, no luck.
Thanks in advance,
Fábio Castro
-
正式なコメント
This post is more than two years old. Some information may not be up to date. For current information, please check the Gurobi Documentation or Knowledge Base. If you need more help, please create a new post in the community forum, or try Gurobot, our chatbot interface offering instant, expert-level support. -
Hi Fábio,
This error is from Pyomo.
Could you try the following, using the latest version of Pyomo:with SolverFactory('gurobi', solver_io='python', manage_env=True) as opt:
opt.solve(model)from: Pyomo gurobi_direct.GurobiDirect
Cheers,
David0
投稿コメントは受け付けていません。
コメント
2件のコメント