Skip to main content

"Error: converted symmetric Q matrix has more than two billion nonzeros" despite bilinear terms with binaries

Answered

Comments

3 comments

  • Jaromił Najman
    Gurobi Staff Gurobi Staff

    Hi Thomas,

    You are right that Gurobi does not store the whole symmetric Q matrix. However, for some internal (presolve) algorithms double the space is required, i.e., some sub-processes require additional space for non-squares. So in the end, Gurobi might need #squares + 2*#non-squares. Thus, the error is triggered already with ~1 bil quadratic objective terms in your case.

    I have to admit that I never faced a model with > 1 bil quadratic objective terms before so this is quite a new challenge overall. Is there a way to decrease the size of your model? Maybe divide it into smaller chunks? Or perform some manual presolve to reduce the number of quadratic objective terms?

    Best regards, 
    Jaromił

    0
  • Thomas Leveringhaus
    Collaborator
    Investigator
    Gurobi-versary

    Hi Jaromił,

    thanks again for your continuous support.

    I am in a(nother) testing phase of my work and it is not an actual problem to reduce the problem size for examples.
    I already reduced the size below 1 billion quadratic objective terms and I don't receive the error mentioned above.
    But shortly afterwards I got an

    • "Out of memory" in the root-node-simplex in one example,
    • and a program abort during presolve withut error message in another example. I assume, it also happend due to memory issues.

    Density is a problem of my approaches, even with ~500 GB of memory on my machine.
    Even if I would buy more memory I would probably run into performance issues.
    So I need to elaborate some ideas to sparsify my problem - I have a lot of ideas, so work goes on ;-)

    Thanks again

    Thomas

    0
  • Jaromił Najman
    Gurobi Staff Gurobi Staff

    Hi Thomas,

    • "Out of memory" in the root-node-simplex in one example,
    • and a program abort during presolve withut error message in another example. I assume, it also happend due to memory issues.

    For quadratic presolve, often more space is required compared to linear presolve. In particular, because we have to run over 2 variables in a product instead of only one. You could try running without presolve but this would probably deteriorate performance by a lot.

    So I need to elaborate some ideas to sparsify my problem - I have a lot of ideas, so work goes on ;-)

    That's good to hear. Hopefully you can come up with something to make your algorithm work even with the memory limits.

    Best regards, 
    Jaromił

    0

Please sign in to leave a comment.