Skip to main content

I have sufficient memory but get memory error running the code on HPC.

Answered

Comments

4 comments

  • Mario Ruthmair
    Gurobi Staff Gurobi Staff

    Hi Araz,

    The code part you are showing does not contain any Gurobi elements, Gurobi seems to be called in one of the functions that are called from here (initiate).

    The Gurobi error 10001 is raised when a component in the solver tries to allocate additional memory for internal use and does not get it from the operating system. So, there seems to be an issue with memory on your HPC machine. You could try to get more details about the memory consumption of the run with Julia tools or via the operating system.

    Best regards,
    Mario

    0
  • Araz Nasirian
    Conversationalist
    Gurobi-versary
    First Question

    I assume the problem is related to this part of the code that I copied and pasted below.

    Running this code on HPC I request 32GB of memory.

    • When I set the parameters as H=J=7 and T=\Omega=10 it solves the problem and consumes 1.2 GB of memory.
    • When I set the parameters as H=J=7 and T=40, \Omega=320 it says it consumes 1.43 GB of memory. It also gave me Gurobi Error 10001

    I have two questions:

    1.  Is there anyway of writing this code more efficiently?
    2.  Why I don't get Guroby error when I run this code in my Surface labtop with 8GB of RAM?

     

    function solve_sub_prob(χʳ,H,J,T,Ω)
        sp =[ Model(Gurobi.Optimizer) for t in 1:T, ω in 1:Ω]
        θ_1 = fill(0.0,T,Ω)
        γ_1 = fill(0.0,J,T,Ω)
        st_1 = fill(0.0,T,Ω)
        for v in 1:T
            for u in 1:Ω
                @variable(sp[v,u], 0 ≤ γ[1:J , [v] , [u] ], Int )
                @variable(sp[v,u], α[1:H , 1:J, [v] , [u] ], Bin )
                @objective(sp[v,u], Min, sum(zᶜ[v,j] * γ[j,v,u] for j in 1:J) )
                @constraint(sp[v,u], con_23b[h in 1:H , j in 1:J], α[h,j,v,u] ≤ χʳ[h,j] )
                @constraint(sp[v,u], con_23c[h in 1:H], sum( α[h,j,v,u] for j in 1:J) ≤ 1 )
                @constraint(sp[v,u], con_23d[j in 1:J], sum( α[h,j,v,u] for h in 1:H ) + γ[j,v,u] ≥ D[j,v,u] )
                optimize!(sp[v,u])
                θ_1[v,u] = objective_value(sp[v,u])
                γ_1[:,v,u] = round.(value.(γ[:,v,u]) , digits=1)
                st_1[v,u] = solve_time( sp[v,u] )
            end
        end
        return θ_1, γ_1 , sum(st_1)
    end

     

    0
  • Mario Ruthmair
    Gurobi Staff Gurobi Staff

    How do you measure the memory consumption? I am surprised that the memory usage for 12,800 (T,\Omega) pairs only slightly increases compared to 100 pairs.

    To your questions:

    1. Since the model for each (T,\Omega) pair seems independent from the others, you could create the Model(Gurobi.Optimizer) in the inner loop and free it again after retrieving the results.
    2. This is certainly interesting, so obviously, there is a difference in the setup on your laptop and the one on the cluster. You request 32 GB on the cluster, could this probably be shared memory, and you actually have much less available? Are there other differences in your setup, e.g., with respect to software versions, build workflow, etc.?
    0
  • Araz Nasirian
    Conversationalist
    Gurobi-versary
    First Question

    Thank you for your hint

    I redefine the function as below and it works well now.

    Now, the solution time is much better. :)

    function solve_sub_prob(χʳ,H,J,T,Ω)
        θ_1 = fill(0.0,T,Ω)
        γ_1 = fill(0.0,J,T,Ω)
        st_1 = fill(0.0,T,Ω)
        for v in 1:T
            for u in 1:Ω
                sp = Model(Gurobi.Optimizer)
                @variable(sp, 0 ≤ γ[1:J, [v], [u]], Int)
                @variable(sp, α[1:H, 1:J, [v], [u] ], Bin )
                @objective(sp, Min, sum(zᶜ[v,j] * γ[j,v,u] for j in 1:J) )
                @constraint(sp, con_23b[h in 1:H , j in 1:J], α[h,j,v,u] ≤ χʳ[h,j] )
                @constraint(sp, con_23c[h in 1:H], sum( α[h,j,v,u] for j in 1:J) ≤ 1 )
                @constraint(sp, con_23d[j in 1:J], sum( α[h,j,v,u] for h in 1:H ) + γ[j,v,u] ≥ D[j,v,u] )
                optimize!(sp)
                θ_1[v,u] = objective_value(sp)
                γ_1[:,v,u] = round.(value.(γ[:,v,u]) , digits=1)
                st_1[v,u] = solve_time( sp )
            end
        end
        return θ_1, γ_1, sum(st_1)
    end
     
    0

Please sign in to leave a comment.