Skip to main content

Out of memory though memory is not exceeded

Awaiting user input

Comments

7 comments

  • Riley Clement
    Gurobi Staff Gurobi Staff

    Hi Christian,

    Are you using GRBfreemodel and GRBfreeenv to free up the memory of models once you're done with them?  If not, this would explain the out of memory error if the models are not large.

    I also don´t know why it says gurobi 11.0.1 in the first logs since that version is not installed on my machine

    This logfile was produced with Gurobi 11.0.1 - no doubt about it.  Are you connecting to a cluster manager?  This would suggest so: `Set parameter Username`.  I think this could explain a few things you observe, arising from machine attached to the cluster with different versions of Gurobi installed, and machines attached to the cluster that are not licensed (or at least Gurobi cannot find the license on them).

    - Riley

    0
  • Christian Perau
    Curious
    Gurobi-versary
    Conversationalist

    Hi Riley,

    yes I am using first GRBfreemodel and right after GRBfreeenv to free the memory. But I also monitored it in during runtime and the total memory usage of the machine was at its peak less than 10%.

    No, I am running the code on a local machine. And I checked again and there is no gurobi1101 version installed (there might have been in the past but I deinstalled it and upgraded to 1102). So I guess I´ll try to uninstall all gurobi versions an reinstall the newest 11 version and see if that helps. I´ll share my findings afterwards.

    Best regards,

    Christian

    0
  • Riley Clement
    Gurobi Staff Gurobi Staff

    Hi Christian,

    Are you setting the Username parameter in the code?  If not then I'd open your license file in a text editor for clues.

    - Riley

    0
  • Christian Perau
    Curious
    Gurobi-versary
    Conversationalist

    Hi Riley,

    I don´t set the Username in the code. However, the license file seems ok and it does work the first 253 iterations.

    I removed all gurobi versions from my machine and installed 11.0.3 again. I added following lines to my code:

    int major, minor, technical;
    GRBversion(&major, &minor, &technical);
    printf("Gurobi library version %d.%d.%d\n", major, minor, technical);
    fflush(NULL);

    Which outputs

    Gurobi library version 11.0.1

    So I am really confused why it does not get the right version. Maybe I did something wrong with my setup? I am working in Visual Studio and made the following changes in my solution properties to get gurobi going:

    - Under C/C++ -> General -> Additional Include Directories I added "C:\gurobi1103\win64\include"

    - Under Linker -> General -> Additional Library Directories I added "C:\gurobi1103\win64\lib"

    - Under Linker -> Input - > Additional Dependencies I added "gurobi110.lib"

    Is there anything you can think of that I forgot or did wrong? In the command console it works correctly:

    C:\>gurobi_cl --version
    Gurobi Optimizer version 11.0.3 build v11.0.3rc0 (win64 - Windows Server 2022.0 (20348.2))
    Copyright (c) 2024, Gurobi Optimization, LLC

     

    0
  • Riley Clement
    Gurobi Staff Gurobi Staff

    There's a 11.0.1 library sneaking in somewhere.  Can you check if your setup is consistent with the following article: How do I configure a new Gurobi C++ project with Microsoft Visual Studio? 

    I'd also look at the values of your environment variables and see if you can spot any gurobi1101 directories in their values.

    0
  • Christian Perau
    Curious
    Gurobi-versary
    Conversationalist

    After a while I found the source. I call the C-code from within python (via ctypes dll). In the corresponding anaconda environment I had gurobipy 11.0.1 installed. After installing gurobipy 11.0.3 the output was correct. I still don´t get why this solve the issue with the other versions though. Maybe gurobipy sets some global variable that is called with in the C-function (my only guess).

    However, the original Problem regarding the Out of memory and subsequent error in the writing the log files remains. The output is:

    // solve master model in iteration 254
    Out of memory
    // free env and model
    // build environment and model again and try to solve
    Unable to write to file 'master_model_it_254.lp'
    // solve sub model (withouth any problem)
    // next iteration, solve master model 255
    Unable to write to file 'master_model_it_255.lp'
    // and so on

    If I look in the .log files (they are created) number 254 says


    Gurobi 11.0.3 (win64) logging started Fri Jul 26 19:27:54 2024

    Set parameter LogToConsole to value 0
    Set parameter Username
    Set parameter ScaleFlag to value 0
    Set parameter BarHomogeneous to value 1
    Set parameter AggFill to value 1000
    Set parameter IISMethod to value 1
    Set parameter LogFile to value "solver_output_master_model_it_254.log"
    Set parameter Threads to value 32
    Academic license - for non-commercial use only - expires 2025-02-01
    Warning for adding constraints: zero or small (< 1e-13) coefficients, ignored
    Gurobi Optimizer version 11.0.3 build v11.0.3rc0 (win64 - Windows Server 2022.0 (20348.2))

    CPU model: AMD EPYC 7763 64-Core Processor, instruction set [SSE2|AVX|AVX2]
    Thread count: 128 physical cores, 256 logical processors, using up to 32 threads

    Optimize a model with 1713 rows, 885 columns and 159602 nonzeros
    Model fingerprint: 0x604635c0
    Variable types: 3 continuous, 882 integer (882 binary)
    Coefficient statistics:
      Matrix range     [1e-04, 4e+07]
      Objective range  [1e+00, 4e+03]
      Bounds range     [1e+00, 1e+00]
      RHS range        [1e+00, 4e+07]
    Found heuristic solution: objective 2169.3698223
    Presolve removed 847 rows and 637 columns
    Presolve time: 0.41s
    Presolved: 866 rows, 248 columns, 96574 nonzeros
    Found heuristic solution: objective 235.6792793
    Variable types: 3 continuous, 245 integer (220 binary)
    Found heuristic solution: objective 232.0928374

    Root relaxation: objective 8.039470e+01, 104 iterations, 0.01 seconds (0.01 work units)

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0   80.39470    0   11  232.09284   80.39470  65.4%     -    0s
    H    0     0                     218.9332928   80.39470  63.3%     -    0s
    H    0     0                     206.4067925   80.39470  61.1%     -    0s

    Explored 0 nodes (0 simplex iterations) in 0.74 seconds (0.39 work units)
    Thread count was 1 (of 256 available processors)

    Solution count 0

    Solve interrupted (error code 10001)
    Best objective -, best bound -, gap -

    Gurobi 11.0.3 (win64) logging started Fri Jul 26 19:29:28 2024

    Set parameter LogToConsole to value 0
    Set parameter Username
    Set parameter ScaleFlag to value 0
    Set parameter BarHomogeneous to value 1
    Set parameter AggFill to value 1000
    Set parameter IISMethod to value 1
    Set parameter LogFile to value "solver_output_master_model_it_254.log"
    Set parameter Threads to value 28
    Academic license - for non-commercial use only - expires 2025-02-01
    Warning for adding constraints: zero or small (< 1e-13) coefficients, ignored

    and 255 says


    Gurobi 11.0.3 (win64) logging started Fri Jul 26 19:32:46 2024

    Set parameter LogToConsole to value 0
    Set parameter Username
    Set parameter ScaleFlag to value 0
    Set parameter BarHomogeneous to value 1
    Set parameter AggFill to value 1000
    Set parameter IISMethod to value 1
    Set parameter LogFile to value "solver_output_master_model_it_255.log"
    Set parameter Threads to value 32
    Academic license - for non-commercial use only - expires 2025-02-01
    Warning for adding constraints: zero or small (< 1e-13) coefficients, ignored

    So it does create the .log files and write something to it. But somehow crashes afterwards.

    Here the code:

    GRBenv* env = NULL;
    GRBmodel* model = NULL;
    int error = 0;
    int status = 0;
    int i;

    // Create empty environment
    error = GRBemptyenv(&env);
    if (error) goto QUIT;

    if (log_file != NULL)
    {
        error = GRBsetstrparam(env, "LogFile", log_file);
        if (error) goto QUIT;
    }

    // Set parameters
    for (i = 0; i < n_int_params; i++)
    {
        error = GRBsetintparam(env, int_param_names[i], int_param_values[i]);
        if (error) goto QUIT;
    }
    for (i = 0; i < n_double_params; i++)
    {
        error = GRBsetdblparam(env, double_param_names[i], double_param_values[i]);
        if (error) goto QUIT;
    }
    for (i = 0; i < n_str_params; i++)
    {
        error = GRBsetstrparam(env, str_param_names[i], str_param_values[i]);
        if (error) goto QUIT;
    }

    // Start environment
    error = GRBstartenv(env);
    if (error) goto QUIT;

    // create model and solve it...

    if (error)
    {
        printf("ERROR: Failed to build and solve the model.\n");
        fflush(NULL);
        printf("ERROR: %s\n", GRBgeterrormsg(env));
        fflush(NULL);
        status = error;
    }

    // Free data, model and environment
    GRBfreemodel(model);
    GRBfreeenv(env);
    return status;

    Do you have any idea why this could happen?

     

     

    0
  • Riley Clement
    Gurobi Staff Gurobi Staff

    Hi Christian,

    I'll turn this into a ticket through our Help Center, it'll be easier for me to keep tabs on it.

    You'll receive an email to this effect shortly.

    - Riley

    0

Please sign in to leave a comment.