Skip to main content

Limit memory usage

Maël Madon
Gurobi-versary
First Comment
First Question

Hi Gurobi experts,

I was wondering if it was possible to specify to the solver a memory usage upper bound.

I've been running an optimization on a 64 bits linux, with a 12-core Intel i7-8700 CPU, 16Gb of RAM and 8Gb of swap. I used all the treads and specified no NodefileStart parameter. In about 3h, the kernel stopped my processes because it exceeded both the memory and the swap.

Thank you for your answer and do not hesitate if you want more information

Maël

0

Comments

6 comments

Date Votes
  • Official comment
    Greg Glockner
    Gurobi Staff Gurobi Staff

    Thanks. You should enable node files, and look into tuning to reduce the number of nodes.

  • Official comment
    Simranjit Kaur
    Gurobi Staff Gurobi Staff
    This post is more than three years old. Some information may not be up to date. For current information, please check the Gurobi Documentation or Knowledge Base. If you need more help, please create a new post in the community forum. Or why not try our AI Gurobot?.
  • Greg Glockner
    Gurobi Staff Gurobi Staff

    Please post your logs so that we can understand the model.

    0
  • Maël Madon
    Gurobi-versary
    First Comment
    First Question

    Hi,

    Thanks for your answer. Here is the log (I removed the middle lines) of the problem I am solving. Do you also want the .lp file?

    Gurobi 8.1.1 (linux64) logging started Thu May 16 15:17:46 2019

    Changed value of parameter LogFile to courbes/2019-05-16 15:17:46.888624.log
       Prev: gurobi.log  Default: 
    Optimize a model with 5184 rows, 4032 columns and 11518 nonzeros
    Variable types: 2880 continuous, 1152 integer (1152 binary)
    Coefficient statistics:
      Matrix range     [2e-05, 2e+04]
      Objective range  [1e+00, 1e+00]
      Bounds range     [1e+00, 3e+04]
      RHS range        [3e-01, 1e+04]
    Presolve removed 2886 rows and 2308 columns
    Presolve time: 0.01s
    Presolved: 2298 rows, 1724 columns, 6321 nonzeros
    Variable types: 1148 continuous, 576 integer (576 binary)

    Root relaxation: objective 2.407602e+06, 2121 iterations, 0.01 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0 2407602.17    0  570          - 2407602.17      -     -    0s
         0     0 2408504.38    0  572          - 2408504.38      -     -    0s
         0     0 2408524.39    0  569          - 2408524.39      -     -    0s
         0     0 2408524.93    0  572          - 2408524.93      -     -    0s
    H    0     0                    3000985.6667 2408524.93  19.7%     -    0s
         0     0 2409943.26    0  573 3000985.67 2409943.26  19.7%     -    0s
    H    0     0                    2843315.6667 2409943.26  15.2%     -    0s
         0     0 2409966.22    0  573 2843315.67 2409966.22  15.2%     -    0s
         0     0 2409970.38    0  573 2843315.67 2409970.38  15.2%     -    0s
         0     0 2409972.19    0  573 2843315.67 2409972.19  15.2%     -    0s
    H    0     0                    2570021.0000 2409972.19  6.23%     -    0s
         0     0 2410426.73    0  573 2570021.00 2410426.73  6.21%     -    0s
         0     0 2410447.08    0  573 2570021.00 2410447.08  6.21%     -    0s
         0     0 2410452.15    0  573 2570021.00 2410452.15  6.21%     -    0s
         0     0 2410453.42    0  573 2570021.00 2410453.42  6.21%     -    0s
         0     0 2410469.43    0  573 2570021.00 2410469.43  6.21%     -    0s
         0     0 2410469.43    0  573 2570021.00 2410469.43  6.21%     -    0s
         0     2 2410469.43    0  573 2570021.00 2410469.43  6.21%     -    0s
    H  993   961                    2475419.0000 2410487.02  2.62%  14.6    0s
    H 1077  1028                    2470163.3333 2410487.02  2.42%  14.3    0s
    * 1862  1581             203    2464907.6667 2410487.02  2.21%  10.7    0s
    * 2016  1536             297    2459652.0000 2410499.68  2.00%  11.0    0s
    H 2141  1546                    2454396.3333 2411223.05  1.76%  11.2    3s
    H 2141  1468                    2449140.6667 2411223.05  1.55%  11.2    3s
    H 2142  1394                    2443885.0000 2411223.05  1.34%  11.2    3s
    H 2142  1323                    2438629.3333 2411223.05  1.12%  11.2    3s
      2178  1347 2438629.33  285  571 2438629.33 2411594.01  1.11%  11.0    5s
      2291  1424 2411962.70   40  571 2438629.33 2411962.70  1.09%  18.4   10s
      2364  1474 2438629.33  213  572 2438629.33 2412107.73  1.09%  21.0   15s
     10667  8112 2422387.58  294  277 2438629.33 2412144.27  1.09%  15.8   20s
     22838 19322 2412193.82   54  563 2438629.33 2412174.36  1.08%  13.8   25s
     30772 26722 2414442.84  167  441 2438629.33 2412185.83  1.08%  13.3   30s
     35950 31540 2413894.22  164  573 2438629.33 2412189.14  1.08%  13.1   39s
     35953 31542 2424401.72  354  568 2438629.33 2412189.14  1.08%  13.1   40s
     36008 31579 2420328.63  341  573 2438629.33 2412936.73  1.05%  13.0   45s
     36047 31605 2426410.72  372  573 2438629.33 2413202.26  1.04%  13.0   50s
     36090 31633 2419427.77  266  573 2438629.33 2413407.18  1.03%  13.0   55s
    H36100 30055                    2433373.6667 2413439.64  0.82%  13.0   56s
     36117 30066 2421709.86  325  573 2433373.67 2413470.63  0.82%  13.0   60s
     36150 30090 2413894.22  164  573 2433373.67 2413487.10  0.82%  13.5   65s
     36170 30103 2416348.76  211  573 2433373.67 2413505.96  0.82%  13.5   70s
     36194 30119 2413528.80   93  573 2433373.67 2413528.80  0.82%  13.5   75s
     36218 30135 2417883.34  237  573 2433373.67 2413538.90  0.82%  13.5   80s
     36225 30143 2413540.79   76  571 2433373.67 2413540.79  0.82%  13.8   87s
     36227 30146 2413584.97   77  570 2433373.67 2413552.52  0.81%  13.8   92s
     36658 30433 2414007.22   95  535 2433373.67 2413574.47  0.81%  14.0   95s
     40983 33192 2418311.56  251  359 2433373.67 2413574.47  0.81%  14.6  100s
     45258 35973 2426240.97  424  191 2433373.67 2413574.47  0.81%  14.5  105s
     62347 45881 2413653.91   83  564 2433373.67 2413604.08  0.81%  15.2  110s
     77937 54924 2414236.39  125  519 2433373.67 2413617.65  0.81%  16.0  119s
     77949 54938 2414270.66  126  518 2433373.67 2413617.65  0.81%  16.0  120s
     91241 62419 2413781.35   99  548 2433373.67 2413627.62  0.81%  16.8  125s
     104019 69704 2413875.48  106  541 2433373.67 2413634.28  0.81%  17.4  130s
     112417 74285 2413770.13   87  560 2433373.67 2413639.50  0.81%  17.8  140s
     123250 80412 2413822.86  102  539 2433373.67 2413644.35  0.81%  18.2  145s
     137501 91964 2414354.13  125  520 2433373.67 2413649.60  0.81%  18.7  150s
     149184 102426 2414317.22  119  526 2433373.67 2413655.91  0.81%  18.9  155s
     152116 105100 2414051.44  111  531 2433373.67 2413655.96  0.81%  19.0  161s
     160898 112930 2422892.73  310  314 2433373.67 2413658.54  0.81%  19.1  165s
     174178 125159 2420076.84  275  357 2433373.67 2413661.12  0.81%  19.2  170s
     188151 137625 2419804.13  271  349 2433373.67 2413666.51  0.81%  19.4  175s
     191393 140622 2424391.03  413  188 2433373.67 2413666.51  0.81%  19.5  181s
     199152 147499 2414324.44  121  523 2433373.67 2413669.60  0.81%  19.6  185s
     214131 161242 2413727.80   95  551 2433373.67 2413673.95  0.81%  19.8  190s

    [...]

    15016285 13351891 2416129.56 158 480 2433373.67 2413855.56 0.80% 23.0 9040s
    15023515 13358361 2415076.96 144 495 2433373.67 2413855.58 0.80% 23.0 9045s
    15032529 13366375 2413957.77 109 535 2433373.67 2413855.59 0.80% 23.0 9050s
    15041192 13374030 2414049.59 113 531 2433373.67 2413855.62 0.80% 23.0 9055s
    15044648 13377144 cutoff 353 2433373.67 2413855.64 0.80% 23.0 9064s
    15044663 13377144 2414071.65 93 545 2433373.67 2413855.64 0.80% 23.0 9065s
    15053002 13384348 2419433.53 234 392 2433373.67 2413855.67 0.80% 23.0 9070s
    15061786 13392157 2423697.64 306 321 2433373.67 2413855.69 0.80% 23.0 9075s
    15072376 13401780 2414869.23 142 500 2433373.67 2413855.72 0.80% 23.0 9080s
    15081879 13410260 2414116.16 102 533 2433373.67 2413855.75 0.80% 23.0 9085s
    15090633 13418134 cutoff 377 2433373.67 2413855.75 0.80% 23.0 9090s
    15098557 13425186 2416071.70 172 463 2433373.67 2413855.81 0.80% 23.0 9095s
    15106074 13431914 2414476.58 121 514 2433373.67 2413855.82 0.80% 23.0 9100s
    15114962 13439735 2415197.08 146 491 2433373.67 2413855.87 0.80% 23.0 9105s

    This execution has been stopped by the kernel because the memory was full.

    0
  • Maël Madon
    Gurobi-versary
    First Comment
    First Question

    For a complete execution (with time limit) :

    Gurobi 8.1.1 (linux64) logging started Thu May 16 10:37:43 2019

    Changed value of parameter LogFile to courbes/2019-05-16 10:37:43.528653.log
    Prev: gurobi.log Default: 
    Optimize a model with 5184 rows, 4032 columns and 11518 nonzeros
    Variable types: 2880 continuous, 1152 integer (1152 binary)
    Coefficient statistics:
    Matrix range [2e-05, 2e+04]
    Objective range [1e+00, 1e+00]
    Bounds range [1e+00, 3e+04]
    RHS range [3e-01, 1e+04]
    Presolve removed 2886 rows and 2308 columns
    Presolve time: 0.01s
    Presolved: 2298 rows, 1724 columns, 6321 nonzeros
    Variable types: 1148 continuous, 576 integer (576 binary)

    Root relaxation: objective 2.407602e+06, 2121 iterations, 0.01 seconds

    Nodes | Current Node | Objective Bounds | Work
    Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time

    0 0 2407602.17 0 570 - 2407602.17 - - 0s
    0 0 2408504.38 0 572 - 2408504.38 - - 0s
    0 0 2408524.39 0 569 - 2408524.39 - - 0s
    0 0 2408524.93 0 572 - 2408524.93 - - 0s
    H 0 0 3000985.6667 2408524.93 19.7% - 0s
    0 0 2409943.26 0 573 3000985.67 2409943.26 19.7% - 0s
    H 0 0 2843315.6667 2409943.26 15.2% - 0s
    0 0 2409966.22 0 573 2843315.67 2409966.22 15.2% - 0s
    0 0 2409970.38 0 573 2843315.67 2409970.38 15.2% - 0s
    0 0 2409972.19 0 573 2843315.67 2409972.19 15.2% - 0s
    H 0 0 2570021.0000 2409972.19 6.23% - 0s
    0 0 2410426.73 0 573 2570021.00 2410426.73 6.21% - 0s
    0 0 2410447.08 0 573 2570021.00 2410447.08 6.21% - 0s
    0 0 2410452.15 0 573 2570021.00 2410452.15 6.21% - 0s
    0 0 2410453.42 0 573 2570021.00 2410453.42 6.21% - 0s
    0 0 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    0 0 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    0 2 2410469.43 0 573 2570021.00 2410469.43 6.21% - 0s
    H 993 961 2475419.0000 2410487.02 2.62% 14.6 0s
    H 1077 1028 2470163.3333 2410487.02 2.42% 14.3 0s
    * 1862 1581 203 2464907.6667 2410487.02 2.21% 10.7 0s
    * 2016 1536 297 2459652.0000 2410499.68 2.00% 11.0 0s
    H 2141 1546 2454396.3333 2411223.05 1.76% 11.2 3s
    H 2141 1468 2449140.6667 2411223.05 1.55% 11.2 3s
    H 2142 1394 2443885.0000 2411223.05 1.34% 11.2 3s
    H 2142 1323 2438629.3333 2411223.05 1.12% 11.2 3s
    2177 1347 2411995.12 57 571 2438629.33 2411574.83 1.11% 11.1 5s
    2284 1420 2411934.60 34 571 2438629.33 2411934.60 1.09% 18.4 10s
    2363 1474 2412599.00 88 572 2438629.33 2412107.58 1.09% 21.0 15s
    10667 8112 2422387.58 294 277 2438629.33 2412144.27 1.09% 15.8 20s
    21834 18471 2425066.66 418 165 2438629.33 2412159.24 1.09% 13.9 25s
    30772 26722 2414442.84 167 441 2438629.33 2412185.83 1.08% 13.3 30s
    35950 31540 2413894.22 164 573 2438629.33 2412189.14 1.08% 13.1 39s
    35952 31541 2415648.38 232 570 2438629.33 2412189.14 1.08% 13.1 40s
    36003 31575 2422187.68 320 573 2438629.33 2412904.14 1.05% 13.0 45s
    36042 31601 2415447.10 205 573 2438629.33 2413191.36 1.04% 13.0 50s
    36087 31631 2413402.08 139 573 2438629.33 2413402.08 1.03% 13.0 55s
    H36100 30055 2433373.6667 2413439.64 0.82% 13.0 57s
    36114 30064 2420829.36 306 573 2433373.67 2413467.73 0.82% 13.0 60s
    36146 30087 2415230.43 224 573 2433373.67 2413480.80 0.82% 13.5 65s
    36168 30102 2413505.25 155 572 2433373.67 2413505.25 0.82% 13.5 70s
    36192 30118 2425135.78 386 573 2433373.67 2413528.27 0.82% 13.5 75s
    36216 30134 2427287.68 432 573 2433373.67 2413538.55 0.82% 13.5 80s
    36225 30143 2413540.79 76 571 2433373.67 2413540.79 0.82% 13.8 88s
    36227 30146 2413584.97 77 570 2433373.67 2413552.52 0.81% 13.8 93s
    36402 30256 2413978.16 85 549 2433373.67 2413574.47 0.81% 13.9 95s
    39455 32275 2416173.19 193 422 2433373.67 2413574.47 0.81% 14.4 100s
    45246 35958 2426287.86 423 190 2433373.67 2413574.47 0.81% 14.5 105s
    59367 44215 2415677.70 184 429 2433373.67 2413592.38 0.81% 15.1 110s
    76850 54362 2413692.73 86 561 2433373.67 2413617.65 0.81% 15.9 115s
    77949 54938 2414270.66 126 518 2433373.67 2413617.65 0.81% 16.0 120s
    91241 62419 2413781.35 99 548 2433373.67 2413627.62 0.81% 16.8 125s
    104019 69704 2413875.48 106 541 2433373.67 2413634.28 0.81% 17.4 130s
    112417 74285 2413770.13 87 560 2433373.67 2413639.50 0.81% 17.8 140s
    123250 80412 2413822.86 102 539 2433373.67 2413644.35 0.81% 18.2 145s
    137501 91964 2414354.13 125 520 2433373.67 2413649.60 0.81% 18.7 150s
    149184 102426 2414317.22 119 526 2433373.67 2413655.91 0.81% 18.9 155s
    152116 105100 2414051.44 111 531 2433373.67 2413655.96 0.81% 19.0 160s
    161261 113289 2413737.35 91 555 2433373.67 2413658.54 0.81% 19.1 165s
    175828 126712 2424019.00 338 280 2433373.67 2413661.12 0.81% 19.2 170s
    189735 139129 2423423.57 364 245 2433373.67 2413666.51 0.81% 19.4 175s
    191393 140622 2424391.03 413 188 2433373.67 2413666.51 0.81% 19.5 180s
    200868 149085 2424766.16 278 351 2433373.67 2413669.98 0.81% 19.6 185s
    215896 162844 2414129.72 125 521 2433373.67 2413673.95 0.81% 19.7 190s
    228870 174648 2425091.81 331 293 2433373.67 2413676.14 0.81% 19.9 198s
    231069 176606 2414457.40 135 507 2433373.67 2413676.20 0.81% 19.9 200s
    240199 184836 2427753.33 317 306 2433373.67 2413678.22 0.81% 19.9 209s
    240211 184845 2427769.57 318 305 2433373.67 2413678.98 0.81% 19.9 210s
    251522 195239 2413757.91 96 551 2433373.67 2413681.06 0.81% 20.0 215s
    265320 207656 2427780.26 486 118 2433373.67 2413684.66 0.81% 20.1 220s
    276803 218155 2418251.69 220 403 2433373.67 2413686.67 0.81% 20.2 225s
    291499 231360 2414010.38 108 537 2433373.67 2413689.64 0.81% 20.3 231s
    303604 242178 2414179.38 115 527 2433373.67 2413690.35 0.81% 20.4 235s
    315911 253160 cutoff 306 2433373.67 2413692.89 0.81% 20.4 240s
    323639 260269 2420150.67 299 318 2433373.67 2413694.05 0.81% 20.5 245s
    338466 273964 2414188.28 101 527 2433373.67 2413696.79 0.81% 20.5 250s
    352920 287007 2422498.12 306 314 2433373.67 2413698.00 0.81% 20.6 255s
    367651 300162 2421801.32 294 328 2433373.67 2413700.89 0.81% 20.6 260s
    380075 311429 2413918.68 90 547 2433373.67 2413704.40 0.81% 20.7 265s
    394532 324265 2422958.55 304 319 2433373.67 2413706.61 0.81% 20.7 270s
    406322 335228 2414003.91 100 547 2433373.67 2413707.38 0.81% 20.8 275s
    417939 345669 2416675.78 169 469 2433373.67 2413708.43 0.81% 20.8 280s
    431424 357757 2415815.37 169 472 2433373.67 2413710.38 0.81% 20.9 285s
    441827 367133 2425700.24 305 314 2433373.67 2413711.97 0.81% 20.9 290s
    450900 375324 2413844.14 108 539 2433373.67 2413712.31 0.81% 20.9 295s
    461449 384842 2414103.67 116 530 2433373.67 2413713.85 0.81% 20.9 300s
    474186 396257 2424292.12 277 355 2433373.67 2413715.45 0.81% 21.0 305s
    487650 408425 2413883.34 101 542 2433373.67 2413716.74 0.81% 21.0 310s
    497368 417266 2416663.33 185 452 2433373.67 2413717.32 0.81% 21.0 315s
    508838 427678 2415088.32 127 502 2433373.67 2413719.13 0.81% 21.0 320s
    521697 439251 2420212.07 242 387 2433373.67 2413720.09 0.81% 21.1 325s
    531896 448408 2426177.85 442 114 2433373.67 2413721.21 0.81% 21.1 330s
    540014 455860 cutoff 242 2433373.67 2413721.40 0.81% 21.2 335s
    553811 468376 cutoff 359 2433373.67 2413722.71 0.81% 21.2 340s
    566299 479622 2415134.08 155 485 2433373.67 2413723.83 0.81% 21.2 345s
    578046 490158 2416292.41 173 462 2433373.67 2413724.44 0.81% 21.2 350s
    591060 501951 2414474.43 133 511 2433373.67 2413725.73 0.81% 21.2 355s
    601539 511537 2422298.16 275 351 2433373.67 2413726.06 0.81% 21.2 364s
    601551 511509 2422314.48 276 350 2433373.67 2413726.06 0.81% 21.2 365s
    612672 521145 2417485.92 195 438 2433373.67 2413727.01 0.81% 21.3 370s
    624326 531559 2414231.28 114 530 2433373.67 2413728.10 0.81% 21.3 375s
    636630 542540 2417715.77 201 432 2433373.67 2413728.92 0.81% 21.3 380s
    649715 554262 2421707.35 276 328 2433373.67 2413729.97 0.81% 21.3 385s
    659882 563430 2413802.71 88 557 2433373.67 2413730.68 0.81% 21.3 390s
    671433 573610 2414336.40 130 515 2433373.67 2413731.28 0.81% 21.3 395s
    683291 584173 2413832.02 105 537 2433373.67 2413732.00 0.81% 21.4 400s
    692587 592207 2424173.35 314 308 2433373.67 2413732.37 0.81% 21.4 409s
    692899 592498 2426801.34 373 244 2433373.67 2413732.58 0.81% 21.4 410s
    702467 600993 2424194.59 319 283 2433373.67 2413733.36 0.81% 21.4 415s
    714507 611814 2419717.88 214 418 2433373.67 2413733.98 0.81% 21.4 420s
    724603 620867 2415987.56 161 471 2433373.67 2413735.02 0.81% 21.4 425s
    735914 630810 2427525.24 397 218 2433373.67 2413735.47 0.81% 21.4 430s
    748688 642467 2425769.83 337 275 2433373.67 2413736.34 0.81% 21.4 435s
    759752 652509 2420003.36 226 401 2433373.67 2413736.97 0.81% 21.4 440s
    770803 662368 2416153.02 157 484 2433373.67 2413737.43 0.81% 21.5 445s
    780128 670686 2413871.38 98 549 2433373.67 2413738.05 0.81% 21.5 450s
    783766 674027 2414118.57 96 543 2433373.67 2413738.26 0.81% 21.5 456s
    789325 679103 2427515.68 320 309 2433373.67 2413738.59 0.81% 21.5 460s
    801621 690385 2426819.16 340 267 2433373.67 2413739.18 0.81% 21.5 465s
    810337 698101 2413889.03 97 548 2433373.67 2413739.56 0.81% 21.5 470s
    822045 708710 2417986.46 212 418 2433373.67 2413739.78 0.81% 21.5 475s
    833571 718843 2414022.99 106 540 2433373.67 2413740.47 0.81% 21.6 480s

    Cutting planes:
    Gomory: 222
    Implied bound: 21
    MIR: 812
    Flow cover: 762

    Explored 833962 nodes (17984338 simplex iterations) in 480.01 seconds
    Thread count was 12 (of 12 available processors)

    Solution count 10: 2.43337e+06 2.43863e+06 2.44388e+06 ... 2.57002e+06

    Time limit reached
    Best objective 2.433373666667e+06, best bound 2.417606666667e+06, gap 0.6479%
    0
  • Sonja Mars
    Gurobi Staff Gurobi Staff

    Hi,

    This seems to be a tricky model where Gurobi needs to process a lot of nodes to get anywhere. As Greg already mentioned node files are a good way of reducing the memory load. Also you can try to decrease the number of threads to 2 or 4. This will also decrease the memory pressure. 

    Have you tried to do some parameter tuning on this model? Sometimes it helps to focus on feasible solutions (MIPFocus=1) or to concentrate on cuts (MIPFocus=3). If you can converge faster to your desired MIP gap you will need less nodes and this means less memory.

    Best regards,

      Sonja

    1

Post is closed for comments.

Didn't find what you were looking for?

New post