Limit memory usage

Comments

5 comments

  • Official comment
    Greg Glockner

    Thanks. You should enable node files, and look into tuning to reduce the number of nodes.

    Comment actions Permalink
  • Greg Glockner

    Please post your logs so that we can understand the model.

    0
    Comment actions Permalink
  • Maël Madon

    Hi,

    Thanks for your answer. Here is the log (I removed the middle lines) of the problem I am solving. Do you also want the .lp file?

    Gurobi 8.1.1 (linux64) logging started Thu May 16 15:17:46 2019

    Changed value of parameter LogFile to courbes/2019-05-16 15:17:46.888624.log
    Prev: gurobi.log Default:
    Optimize a model with 5184 rows, 4032 columns and 11518 nonzeros
    Variable types: 2880 continuous, 1152 integer (1152 binary)
    Coefficient statistics:
    Matrix range [2e-05, 2e+04]
    Objective range [1e+00, 1e+00]
    Bounds range [1e+00, 3e+04]
    RHS range [3e-01, 1e+04]
    Presolve removed 2886 rows and 2308 columns
    Presolve time: 0.01s
    Presolved: 2298 rows, 1724 columns, 6321 nonzeros
    Variable types: 1148 continuous, 576 integer (576 binary)

    Root relaxation: objective 2.407602e+06, 2121 iterations, 0.01 seconds

    Nodes | Current Node | Objective Bounds | Work
    Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time

    0 0 2407602.17 0 570 - 2407602.17 - - 0s
    0 0 2408504.38 0 572 - 2408504.38 - - 0s
    0 0 2408524.39 0 569 - 2408524.39 - - 0s
    0 0 2408524.93 0 572 - 2408524.93 - - 0s
    H 0 0 3000985.6667 2408524.93 19.7% - 0s
    0 0 2409943.26 0 573 3000985.67 2409943.26 19.7% - 0s
    H 0 0 2843315.6667 2409943.26 15.2% - 0s
    0 0 2409966.22 0 573 2843315.67 2409966.22 15.2% - 0s
    0 0 2409970.38 0 573 2843315.67 2409970.38 15.2% - 0s
    0 0 2409972.19 0 573 2843315.67 2409972.19 15.2% - 0s
    H 0 0 2570021.0000 2409972.19 6.23% - 0s
    0 0 2410426.73 0 573 2570021.00 2410426.73 6.21% - 0s
    0 0 2410447.08 0 573 2570021.00 2410447.08 6.21% - 0s
    0 0 2410452.15 0 573 2570021.00 2410452.15 6.21% - 0s
    0 0 2410453.42 0 573 2570021.00 2410453.42 6.21% - 0s
    0 0 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    0 0 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    0 2 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    H 993 961 2475419.0000 2410487.02 2.62% 14.6 0s
    H 1077 1028 2470163.3333 2410487.02 2.42% 14.3 0s
    * 1862 1581 203 2464907.6667 2410487.02 2.21% 10.7 0s
    * 2016 1536 297 2459652.0000 2410499.68 2.00% 11.0 0s
    H 2141 1546 2454396.3333 2411223.05 1.76% 11.2 3s
    H 2141 1468 2449140.6667 2411223.05 1.55% 11.2 3s
    H 2142 1394 2443885.0000 2411223.05 1.34% 11.2 3s
    H 2142 1323 2438629.3333 2411223.05 1.12% 11.2 3s
    2178 1347 2438629.33 285 571 2438629.33 2411594.01 1.11% 11.0 5s
    2291 1424 2411962.70 40 571 2438629.33 2411962.70 1.09% 18.4 10s
    2364 1474 2438629.33 213 572 2438629.33 2412107.73 1.09% 21.0 15s
    10667 8112 2422387.58 294 277 2438629.33 2412144.27 1.09% 15.8 20s
    22838 19322 2412193.82 54 563 2438629.33 2412174.36 1.08% 13.8 25s
    30772 26722 2414442.84 167 441 2438629.33 2412185.83 1.08% 13.3 30s
    35950 31540 2413894.22 164 573 2438629.33 2412189.14 1.08% 13.1 39s
    35953 31542 2424401.72 354 568 2438629.33 2412189.14 1.08% 13.1 40s
    36008 31579 2420328.63 341 573 2438629.33 2412936.73 1.05% 13.0 45s
    36047 31605 2426410.72 372 573 2438629.33 2413202.26 1.04% 13.0 50s
    36090 31633 2419427.77 266 573 2438629.33 2413407.18 1.03% 13.0 55s
    H36100 30055 2433373.6667 2413439.64 0.82% 13.0 56s
    36117 30066 2421709.86 325 573 2433373.67 2413470.63 0.82% 13.0 60s
    36150 30090 2413894.22 164 573 2433373.67 2413487.10 0.82% 13.5 65s
    36170 30103 2416348.76 211 573 2433373.67 2413505.96 0.82% 13.5 70s
    36194 30119 2413528.80 93 573 2433373.67 2413528.80 0.82% 13.5 75s
    36218 30135 2417883.34 237 573 2433373.67 2413538.90 0.82% 13.5 80s
    36225 30143 2413540.79 76 571 2433373.67 2413540.79 0.82% 13.8 87s
    36227 30146 2413584.97 77 570 2433373.67 2413552.52 0.81% 13.8 92s
    36658 30433 2414007.22 95 535 2433373.67 2413574.47 0.81% 14.0 95s
    40983 33192 2418311.56 251 359 2433373.67 2413574.47 0.81% 14.6 100s
    45258 35973 2426240.97 424 191 2433373.67 2413574.47 0.81% 14.5 105s
    62347 45881 2413653.91 83 564 2433373.67 2413604.08 0.81% 15.2 110s
    77937 54924 2414236.39 125 519 2433373.67 2413617.65 0.81% 16.0 119s
    77949 54938 2414270.66 126 518 2433373.67 2413617.65 0.81% 16.0 120s
    91241 62419 2413781.35 99 548 2433373.67 2413627.62 0.81% 16.8 125s
    104019 69704 2413875.48 106 541 2433373.67 2413634.28 0.81% 17.4 130s
    112417 74285 2413770.13 87 560 2433373.67 2413639.50 0.81% 17.8 140s
    123250 80412 2413822.86 102 539 2433373.67 2413644.35 0.81% 18.2 145s
    137501 91964 2414354.13 125 520 2433373.67 2413649.60 0.81% 18.7 150s
    149184 102426 2414317.22 119 526 2433373.67 2413655.91 0.81% 18.9 155s
    152116 105100 2414051.44 111 531 2433373.67 2413655.96 0.81% 19.0 161s
    160898 112930 2422892.73 310 314 2433373.67 2413658.54 0.81% 19.1 165s
    174178 125159 2420076.84 275 357 2433373.67 2413661.12 0.81% 19.2 170s
    188151 137625 2419804.13 271 349 2433373.67 2413666.51 0.81% 19.4 175s
    191393 140622 2424391.03 413 188 2433373.67 2413666.51 0.81% 19.5 181s
    199152 147499 2414324.44 121 523 2433373.67 2413669.60 0.81% 19.6 185s
    214131 161242 2413727.80 95 551 2433373.67 2413673.95 0.81% 19.8 190s

    [...]

    15016285 13351891 2416129.56 158 480 2433373.67 2413855.56 0.80% 23.0 9040s
    15023515 13358361 2415076.96 144 495 2433373.67 2413855.58 0.80% 23.0 9045s
    15032529 13366375 2413957.77 109 535 2433373.67 2413855.59 0.80% 23.0 9050s
    15041192 13374030 2414049.59 113 531 2433373.67 2413855.62 0.80% 23.0 9055s
    15044648 13377144 cutoff 353 2433373.67 2413855.64 0.80% 23.0 9064s
    15044663 13377144 2414071.65 93 545 2433373.67 2413855.64 0.80% 23.0 9065s
    15053002 13384348 2419433.53 234 392 2433373.67 2413855.67 0.80% 23.0 9070s
    15061786 13392157 2423697.64 306 321 2433373.67 2413855.69 0.80% 23.0 9075s
    15072376 13401780 2414869.23 142 500 2433373.67 2413855.72 0.80% 23.0 9080s
    15081879 13410260 2414116.16 102 533 2433373.67 2413855.75 0.80% 23.0 9085s
    15090633 13418134 cutoff 377 2433373.67 2413855.75 0.80% 23.0 9090s
    15098557 13425186 2416071.70 172 463 2433373.67 2413855.81 0.80% 23.0 9095s
    15106074 13431914 2414476.58 121 514 2433373.67 2413855.82 0.80% 23.0 9100s
    15114962 13439735 2415197.08 146 491 2433373.67 2413855.87 0.80% 23.0 9105s

    This execution has been stopped by the kernel because the memory was full.

    0
    Comment actions Permalink
  • Maël Madon

    For a complete execution (with time limit) :

    Gurobi 8.1.1 (linux64) logging started Thu May 16 10:37:43 2019

    Changed value of parameter LogFile to courbes/2019-05-16 10:37:43.528653.log
    Prev: gurobi.log Default:
    Optimize a model with 5184 rows, 4032 columns and 11518 nonzeros
    Variable types: 2880 continuous, 1152 integer (1152 binary)
    Coefficient statistics:
    Matrix range [2e-05, 2e+04]
    Objective range [1e+00, 1e+00]
    Bounds range [1e+00, 3e+04]
    RHS range [3e-01, 1e+04]
    Presolve removed 2886 rows and 2308 columns
    Presolve time: 0.01s
    Presolved: 2298 rows, 1724 columns, 6321 nonzeros
    Variable types: 1148 continuous, 576 integer (576 binary)

    Root relaxation: objective 2.407602e+06, 2121 iterations, 0.01 seconds

    Nodes | Current Node | Objective Bounds | Work
    Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time

    0 0 2407602.17 0 570 - 2407602.17 - - 0s
    0 0 2408504.38 0 572 - 2408504.38 - - 0s
    0 0 2408524.39 0 569 - 2408524.39 - - 0s
    0 0 2408524.93 0 572 - 2408524.93 - - 0s
    H 0 0 3000985.6667 2408524.93 19.7% - 0s
    0 0 2409943.26 0 573 3000985.67 2409943.26 19.7% - 0s
    H 0 0 2843315.6667 2409943.26 15.2% - 0s
    0 0 2409966.22 0 573 2843315.67 2409966.22 15.2% - 0s
    0 0 2409970.38 0 573 2843315.67 2409970.38 15.2% - 0s
    0 0 2409972.19 0 573 2843315.67 2409972.19 15.2% - 0s
    H 0 0 2570021.0000 2409972.19 6.23% - 0s
    0 0 2410426.73 0 573 2570021.00 2410426.73 6.21% - 0s
    0 0 2410447.08 0 573 2570021.00 2410447.08 6.21% - 0s
    0 0 2410452.15 0 573 2570021.00 2410452.15 6.21% - 0s
    0 0 2410453.42 0 573 2570021.00 2410453.42 6.21% - 0s
    0 0 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    0 0 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    0 2 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    H 993 961 2475419.0000 2410487.02 2.62% 14.6 0s
    H 1077 1028 2470163.3333 2410487.02 2.42% 14.3 0s
    * 1862 1581 203 2464907.6667 2410487.02 2.21% 10.7 0s
    * 2016 1536 297 2459652.0000 2410499.68 2.00% 11.0 0s
    H 2141 1546 2454396.3333 2411223.05 1.76% 11.2 3s
    H 2141 1468 2449140.6667 2411223.05 1.55% 11.2 3s
    H 2142 1394 2443885.0000 2411223.05 1.34% 11.2 3s
    H 2142 1323 2438629.3333 2411223.05 1.12% 11.2 3s
    2177 1347 2411995.12 57 571 2438629.33 2411574.83 1.11% 11.1 5s
    2284 1420 2411934.60 34 571 2438629.33 2411934.60 1.09% 18.4 10s
    2363 1474 2412599.00 88 572 2438629.33 2412107.58 1.09% 21.0 15s
    10667 8112 2422387.58 294 277 2438629.33 2412144.27 1.09% 15.8 20s
    21834 18471 2425066.66 418 165 2438629.33 2412159.24 1.09% 13.9 25s
    30772 26722 2414442.84 167 441 2438629.33 2412185.83 1.08% 13.3 30s
    35950 31540 2413894.22 164 573 2438629.33 2412189.14 1.08% 13.1 39s
    35952 31541 2415648.38 232 570 2438629.33 2412189.14 1.08% 13.1 40s
    36003 31575 2422187.68 320 573 2438629.33 2412904.14 1.05% 13.0 45s
    36042 31601 2415447.10 205 573 2438629.33 2413191.36 1.04% 13.0 50s
    36087 31631 2413402.08 139 573 2438629.33 2413402.08 1.03% 13.0 55s
    H36100 30055 2433373.6667 2413439.64 0.82% 13.0 57s
    36114 30064 2420829.36 306 573 2433373.67 2413467.73 0.82% 13.0 60s
    36146 30087 2415230.43 224 573 2433373.67 2413480.80 0.82% 13.5 65s
    36168 30102 2413505.25 155 572 2433373.67 2413505.25 0.82% 13.5 70s
    36192 30118 2425135.78 386 573 2433373.67 2413528.27 0.82% 13.5 75s
    36216 30134 2427287.68 432 573 2433373.67 2413538.55 0.82% 13.5 80s
    36225 30143 2413540.79 76 571 2433373.67 2413540.79 0.82% 13.8 88s
    36227 30146 2413584.97 77 570 2433373.67 2413552.52 0.81% 13.8 93s
    36402 30256 2413978.16 85 549 2433373.67 2413574.47 0.81% 13.9 95s
    39455 32275 2416173.19 193 422 2433373.67 2413574.47 0.81% 14.4 100s
    45246 35958 2426287.86 423 190 2433373.67 2413574.47 0.81% 14.5 105s
    59367 44215 2415677.70 184 429 2433373.67 2413592.38 0.81% 15.1 110s
    76850 54362 2413692.73 86 561 2433373.67 2413617.65 0.81% 15.9 115s
    77949 54938 2414270.66 126 518 2433373.67 2413617.65 0.81% 16.0 120s
    91241 62419 2413781.35 99 548 2433373.67 2413627.62 0.81% 16.8 125s
    104019 69704 2413875.48 106 541 2433373.67 2413634.28 0.81% 17.4 130s
    112417 74285 2413770.13 87 560 2433373.67 2413639.50 0.81% 17.8 140s
    123250 80412 2413822.86 102 539 2433373.67 2413644.35 0.81% 18.2 145s
    137501 91964 2414354.13 125 520 2433373.67 2413649.60 0.81% 18.7 150s
    149184 102426 2414317.22 119 526 2433373.67 2413655.91 0.81% 18.9 155s
    152116 105100 2414051.44 111 531 2433373.67 2413655.96 0.81% 19.0 160s
    161261 113289 2413737.35 91 555 2433373.67 2413658.54 0.81% 19.1 165s
    175828 126712 2424019.00 338 280 2433373.67 2413661.12 0.81% 19.2 170s
    189735 139129 2423423.57 364 245 2433373.67 2413666.51 0.81% 19.4 175s
    191393 140622 2424391.03 413 188 2433373.67 2413666.51 0.81% 19.5 180s
    200868 149085 2424766.16 278 351 2433373.67 2413669.98 0.81% 19.6 185s
    215896 162844 2414129.72 125 521 2433373.67 2413673.95 0.81% 19.7 190s
    228870 174648 2425091.81 331 293 2433373.67 2413676.14 0.81% 19.9 198s
    231069 176606 2414457.40 135 507 2433373.67 2413676.20 0.81% 19.9 200s
    240199 184836 2427753.33 317 306 2433373.67 2413678.22 0.81% 19.9 209s
    240211 184845 2427769.57 318 305 2433373.67 2413678.98 0.81% 19.9 210s
    251522 195239 2413757.91 96 551 2433373.67 2413681.06 0.81% 20.0 215s
    265320 207656 2427780.26 486 118 2433373.67 2413684.66 0.81% 20.1 220s
    276803 218155 2418251.69 220 403 2433373.67 2413686.67 0.81% 20.2 225s
    291499 231360 2414010.38 108 537 2433373.67 2413689.64 0.81% 20.3 231s
    303604 242178 2414179.38 115 527 2433373.67 2413690.35 0.81% 20.4 235s
    315911 253160 cutoff 306 2433373.67 2413692.89 0.81% 20.4 240s
    323639 260269 2420150.67 299 318 2433373.67 2413694.05 0.81% 20.5 245s
    338466 273964 2414188.28 101 527 2433373.67 2413696.79 0.81% 20.5 250s
    352920 287007 2422498.12 306 314 2433373.67 2413698.00 0.81% 20.6 255s
    367651 300162 2421801.32 294 328 2433373.67 2413700.89 0.81% 20.6 260s
    380075 311429 2413918.68 90 547 2433373.67 2413704.40 0.81% 20.7 265s
    394532 324265 2422958.55 304 319 2433373.67 2413706.61 0.81% 20.7 270s
    406322 335228 2414003.91 100 547 2433373.67 2413707.38 0.81% 20.8 275s
    417939 345669 2416675.78 169 469 2433373.67 2413708.43 0.81% 20.8 280s
    431424 357757 2415815.37 169 472 2433373.67 2413710.38 0.81% 20.9 285s
    441827 367133 2425700.24 305 314 2433373.67 2413711.97 0.81% 20.9 290s
    450900 375324 2413844.14 108 539 2433373.67 2413712.31 0.81% 20.9 295s
    461449 384842 2414103.67 116 530 2433373.67 2413713.85 0.81% 20.9 300s
    474186 396257 2424292.12 277 355 2433373.67 2413715.45 0.81% 21.0 305s
    487650 408425 2413883.34 101 542 2433373.67 2413716.74 0.81% 21.0 310s
    497368 417266 2416663.33 185 452 2433373.67 2413717.32 0.81% 21.0 315s
    508838 427678 2415088.32 127 502 2433373.67 2413719.13 0.81% 21.0 320s
    521697 439251 2420212.07 242 387 2433373.67 2413720.09 0.81% 21.1 325s
    531896 448408 2426177.85 442 114 2433373.67 2413721.21 0.81% 21.1 330s
    540014 455860 cutoff 242 2433373.67 2413721.40 0.81% 21.2 335s
    553811 468376 cutoff 359 2433373.67 2413722.71 0.81% 21.2 340s
    566299 479622 2415134.08 155 485 2433373.67 2413723.83 0.81% 21.2 345s
    578046 490158 2416292.41 173 462 2433373.67 2413724.44 0.81% 21.2 350s
    591060 501951 2414474.43 133 511 2433373.67 2413725.73 0.81% 21.2 355s
    601539 511537 2422298.16 275 351 2433373.67 2413726.06 0.81% 21.2 364s
    601551 511509 2422314.48 276 350 2433373.67 2413726.06 0.81% 21.2 365s
    612672 521145 2417485.92 195 438 2433373.67 2413727.01 0.81% 21.3 370s
    624326 531559 2414231.28 114 530 2433373.67 2413728.10 0.81% 21.3 375s
    636630 542540 2417715.77 201 432 2433373.67 2413728.92 0.81% 21.3 380s
    649715 554262 2421707.35 276 328 2433373.67 2413729.97 0.81% 21.3 385s
    659882 563430 2413802.71 88 557 2433373.67 2413730.68 0.81% 21.3 390s
    671433 573610 2414336.40 130 515 2433373.67 2413731.28 0.81% 21.3 395s
    683291 584173 2413832.02 105 537 2433373.67 2413732.00 0.81% 21.4 400s
    692587 592207 2424173.35 314 308 2433373.67 2413732.37 0.81% 21.4 409s
    692899 592498 2426801.34 373 244 2433373.67 2413732.58 0.81% 21.4 410s
    702467 600993 2424194.59 319 283 2433373.67 2413733.36 0.81% 21.4 415s
    714507 611814 2419717.88 214 418 2433373.67 2413733.98 0.81% 21.4 420s
    724603 620867 2415987.56 161 471 2433373.67 2413735.02 0.81% 21.4 425s
    735914 630810 2427525.24 397 218 2433373.67 2413735.47 0.81% 21.4 430s
    748688 642467 2425769.83 337 275 2433373.67 2413736.34 0.81% 21.4 435s
    759752 652509 2420003.36 226 401 2433373.67 2413736.97 0.81% 21.4 440s
    770803 662368 2416153.02 157 484 2433373.67 2413737.43 0.81% 21.5 445s
    780128 670686 2413871.38 98 549 2433373.67 2413738.05 0.81% 21.5 450s
    783766 674027 2414118.57 96 543 2433373.67 2413738.26 0.81% 21.5 456s
    789325 679103 2427515.68 320 309 2433373.67 2413738.59 0.81% 21.5 460s
    801621 690385 2426819.16 340 267 2433373.67 2413739.18 0.81% 21.5 465s
    810337 698101 2413889.03 97 548 2433373.67 2413739.56 0.81% 21.5 470s
    822045 708710 2417986.46 212 418 2433373.67 2413739.78 0.81% 21.5 475s
    833571 718843 2414022.99 106 540 2433373.67 2413740.47 0.81% 21.6 480s

    Cutting planes:
    Gomory: 222
    Implied bound: 21
    MIR: 812
    Flow cover: 762

    Explored 833962 nodes (17984338 simplex iterations) in 480.01 seconds
    Thread count was 12 (of 12 available processors)

    Solution count 10: 2.43337e+06 2.43863e+06 2.44388e+06 ... 2.57002e+06

    Time limit reached
    Best objective 2.433373666667e+06, best bound 2.417606666667e+06, gap 0.6479%
    0
    Comment actions Permalink
  • Sonja Mars

    Hi,

    This seems to be a tricky model where Gurobi needs to process a lot of nodes to get anywhere. As Greg already mentioned node files are a good way of reducing the memory load. Also you can try to decrease the number of threads to 2 or 4. This will also decrease the memory pressure. 

    Have you tried to do some parameter tuning on this model? Sometimes it helps to focus on feasible solutions (MIPFocus=1) or to concentrate on cuts (MIPFocus=3). If you can converge faster to your desired MIP gap you will need less nodes and this means less memory.

    Best regards,

      Sonja

    1
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk