Running into a memory error when running optimization over a large time period
Answered
Hi all,
Gurobi Optimizer version 9.5.2 build v9.5.2rc0 (win64)
Thread count: 4 physical cores, 8 logical processors, using up to 8 threads
Optimize a model with 171 rows, 431 columns and 509 nonzeros
Model fingerprint: 0x3a1c1f0c
Model has 90 quadratic constraints
Model has 36210 general constraints
Variable types: 20 continuous, 411 integer (200 binary)
I am trying to run a optimization for 365 days in a year (preferably even longer), however I am running into a memory error when I try this. The model does work for shorter periods of time (atleast up to 100 days), the output above is from 10 days. Are there ways of improving this model so that it runs faster/ uses less memory? I used the following constraints:
# ---- Constraints ----
#Constraint 0: Demand is satisfied
con0 = {}
con0 = model.addConstr(quicksum(y[k] for k in K) == quicksum(Production[k] for k in K))
#Constraint 1: Tank storage capacity needs to be equal or greater than the maximum used storage
con1 = {}
for k in K:
con1[k] = model.addConstr(v[k] <= w)
#Constraint 2: On site ammonia in storage calculation for time step 0
con2 = {}
for k in K:
if k == 0:
con2[k] = model.addConstr(v[k] == - y[k] + Production[k])
#Constraint 3: On site ammonia in storage calculation for other steps
con3 = {}
for k in K:
if k != 0:
con3[k] = model.addConstr( - y[k] + v[k-1] + Production[k] == v[k])
# #Constraint 4: Transported amount in a time step needs to be smaller or equal to the sum of the produced amount untill that that time step
# con4 = {}
# for k in K:
# con4[k] = model.addConstr(y[k] <= quicksum( Production[k] for k in K))
#Constraint 5: The transported amount in a time step needs to be smaller or equal to the capacity of the ship transporting in that time step
con5 = {}
for k in K:
con5[k] = model.addConstr(quicksum(x[i,k] * S_capacity[i] for i in I)>= y[k])
#Constraint 6: All ships are available in the first time step
con6 = {}
for i in I:
for k in K:
if k == 0:
con6 = model.addConstr(z[i,k] == 1)
#Constraint 7: A ship becomes available if their counter is greater or equal to their trip time
con7 = {}
for i in I:
for k in K:
con7[i,k] = model.addConstrs((z[i,k] == 1) >> (d[i,k] >= T_time[i]) for i in I for k in range(0, len(K)))
con7[i,k] = model.addConstrs((z[i,k] == 0) >> (d[i,k] <= (T_time[i]-1) ) for i in I for k in range(0, len(K)))
#Constraint 8: In the first time step the counter is equal to the travel time
con8 = {}
for i in I:
for k in K:
if k == 0:
con8[i,k] = model.addConstr(d[i,k] == T_time[i])
#Constraint 9:
con9 = {}
for i in I:
for k in K:
if k != 0:
con9[i,k] = model.addConstrs((c[i,k+1] == 0) >> (x[i,k] == 1) for i in I for k in range(0, len(K)-1))
con9[i,k] = model.addConstrs((c[i,k+1] == 1) >> (x[i,k] == 0) for i in I for k in range(0,len(K)-1))
#Constraint 10:
con10 = {}
for i in I:
for k in K:
if k != 0:
con10[i,k] = model.addConstr(c[i,k] *((d[i,k-1]) +1) == d[i,k])
#Constraint 11:
con11 = {}
for i in I:
for k in K:
con11[i,k] = model.addConstr(x[i,k] <= z[i,k])
#Constraint 12:
con12 = {}
for i in I:
con12[i] = model.addConstr(s[i] == max_(x[i,k] for k in K))
#Constraint 13: The storage capacity of the tankt needs to be larger or equal to the capacity of the largest ship time 1.5
con13 = {}
for i in I:
con13[i] = model.addConstr(max(S_capacity[i]) * 1.5 * s[i] <= w)
#Constraint 14:
con14 = {}
for k in K:
con14[k] = model.addConstr(quicksum(x[i,k] for i in I) <= 1)
Thanks in advance!
0
-
That's a good question, I'm struggling with a simular problem
0 -
Hi Pim,
Thanks posting. Let me see if I can help.
Let me start with a question: the memory error you encountered (for 365 days) happened during model building, or during model solving?
Assuming it happened during model solving, I will recommend this article useful: How do I avoid an out-of-memory condition.
For possible model improvement, can you share a self-contained code, say, for "10 days"?
0 -
Hi Jue,
Thank you for your response!
The problem arose when I was solving the model. I've already tried the tips that were in the article but unfortunately this didn't help.
import numpy as np
import math
from gurobipy import *
import pandas as pd
import gurobipy as gp
import matplotlib.pyplot as plt
# Data
# ---- Indices ----
Ships = ('S1', 'S2','S3', 'S4', 'S5')
Month = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'June', 'July', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec')
# ---- Parameters ----
# ---- Ship parameters ----
S_capacity = np.array([[22600], [22600], [22600], [22600], [22600],
[9000],[9000],[9000],[9000],[9000]])
S_Pcosts = np.array([[39000000],[39000000],[39000000],[39000000],[39000000],
[22000000],[22000000],[22000000],[22000000],[22000000]])
S_Tcosts = np.array([[444598],[444598],[444598],[444598],[444598],
[272823],[272823],[272823],[272823],[272823]])
T_time = np.array([[17],[17],[17],[17],[17],[19],[19],[19],[19],[19]])
# ---- Site parameters
Production = []
Days = []
for n in range(10):
Production += [[803.65]]
Days += [n]
Production = np.array(Production)
T_CAPEX_m3 = 552.339 #USD per m3. Dit vermenigvuldigen met m3 ammonia van een tank
T_OPEX_per = 0.03 #percentage van capex is deel van de opex
T_OPEX_elek = 2.497 #elecricity price per year per m3
# ---- Indices as sets ----
#sets
I = range(len(S_capacity))
#J = range(len(T_capacity))
K = range(len(Production))
# Decision variables
# ---- Number of ships i in month k
x = {}
for i in I:
for k in K:
x[i,k] = model.addVar(lb=0, vtype= GRB.BINARY, name='X[' + str(i) + str(k) + ']')
#---- Number of storage tanks of capacity j
w = {}
w = model.addVar(lb=0, vtype= GRB.INTEGER)
# Eigenlijk moet deze ook een upperbound krijgen toch? Want storage tank heeft een capaciteit van 50.000/60.0000. Maar dat zou dan een constraint moeten zijn
# ---- Tank storage capacity used in month k
v = {}
for k in K:
v[k] = model.addVar(lb=0, vtype= GRB.CONTINUOUS, name= 'V[' + str(k) + ']' )
#De vervoerde hoeveelheid
y = {}
for k in K:
y[k] = model.addVar(lb=0, vtype= GRB.CONTINUOUS, name= 'Y[' + str(k) + ']')
z = {}
for i in I:
for k in K:
z[i,k] = model.addVar(lb=0, vtype = GRB.BINARY, name ='Z[' + str(i) + str(k) + ']')
d = {}
for i in I:
for k in K:
d[i,k] = model.addVar(lb=0, vtype= GRB.INTEGER, name = 'D[' + str(i) + str(k) + ']')
c = {}
for i in I:
for k in K:
c[i,k] = model.addVar(lb=0, vtype= GRB.INTEGER, name= 'C[' + str(i) + str(k) + ']')
s = {}
for i in I:
s[i] = model.addVar(lb=0, vtype= GRB.INTEGER, name= 'S[' + str(i) + ']')
# Integrate new variables
model.update()
# ---- Objective Function ----
model.setObjective(quicksum(x[i,k] * S_Tcosts[i] for i in I for k in K) + quicksum(s[i]* S_Pcosts[i] for i in I) +
w * T_CAPEX_m3 + (w * T_CAPEX_m3) * 0.03 + T_OPEX_elek * w
, GRB.MINIMIZE)
model.update()
# ---- Constraints ----
#Constraint 0: Demand is satisfied
con0 = {}
con0 = model.addConstr(quicksum(y[k] for k in K) == quicksum(Production[k] for k in K))
#Constraint 1: Tank storage capacity needs to be equal or greater than the maximum used storage
con1 = {}
for k in K:
con1[k] = model.addConstr(v[k] <= w)
#Constraint 2: On site ammonia in storage calculation for time step 0
con2 = {}
for k in K:
if k == 0:
con2[k] = model.addConstr(v[k] == - y[k] + Production[k])
#Constraint 3: On site ammonia in storage calculation for other steps
con3 = {}
for k in K:
if k != 0:
con3[k] = model.addConstr( - y[k] + v[k-1] + Production[k] == v[k])
# #Constraint 4: Transported amount in a time step needs to be smaller or equal to the sum of the produced amount untill that that time step
# con4 = {}
# for k in K:
# con4[k] = model.addConstr(y[k] <= quicksum( Production[k] for k in K))
#Constraint 5: The transported amount in a time step needs to be smaller or equal to the capacity of the ship transporting in that time step
con5 = {}
for k in K:
con5[k] = model.addConstr(quicksum(x[i,k] * S_capacity[i] for i in I)>= y[k])
#Constraint 6: All ships are available in the first time step
con6 = {}
for i in I:
for k in K:
if k == 0:
con6 = model.addConstr(z[i,k] == 1)
#Constraint 7: A ship becomes available if their counter is greater or equal to their trip time
con7 = {}
for i in I:
for k in K:
con7[i,k] = model.addConstrs((z[i,k] == 1) >> (d[i,k] >= T_time[i]) for i in I for k in range(0, len(K)))
con7[i,k] = model.addConstrs((z[i,k] == 0) >> (d[i,k] <= (T_time[i]-1) ) for i in I for k in range(0, len(K)))
#Constraint 8: In the first time step the counter is equal to the travel time
con8 = {}
for i in I:
for k in K:
if k == 0:
con8[i,k] = model.addConstr(d[i,k] == T_time[i])
#Constraint 9:
con9 = {}
for i in I:
for k in K:
if k != 0:
con9[i,k] = model.addConstrs((c[i,k+1] == 0) >> (x[i,k] == 1) for i in I for k in range(0, len(K)-1))
con9[i,k] = model.addConstrs((c[i,k+1] == 1) >> (x[i,k] == 0) for i in I for k in range(0,len(K)-1))
#Constraint 10:
con10 = {}
for i in I:
for k in K:
if k != 0:
con10[i,k] = model.addConstr(c[i,k] *((d[i,k-1]) +1) == d[i,k])
#Constraint 11:
con11 = {}
for i in I:
for k in K:
con11[i,k] = model.addConstr(x[i,k] <= z[i,k])
#Constraint 12:
con12 = {}
for i in I:
con12[i] = model.addConstr(s[i] == max_(x[i,k] for k in K))
#Constraint 13: The storage capacity of the tankt needs to be larger or equal to the capacity of the largest ship time 1.5
con13 = {}
for i in I:
con13[i] = model.addConstr(max(S_capacity[i]) * 1.5 * s[i] <= w)
#Constraint 14:
con14 = {}
for k in K:
con14[k] = model.addConstr(quicksum(x[i,k] for i in I) <= 1)
# ---- Solve ----
model.update()
model.setParam('OutputFlag', True) # silencing gurobi output or not
model.setParam('MIPGap', 0) # find the optimal solution
model.write("output.lp") # print the model in .lp format file
model.update()
#model.Params.timeLimit = 1800
model.optimize()This is the code for 10 days, which runs in 2 seconds.
0 -
Hi Pim,
Thanks for sharing the code and sorry for getting back longer than I expected.
After looking into the code, you have created many redundant constraints. Specifically, cons7 and cons9.
Please consider the following changes and let me know how it works now.
#Constraint 7: A ship becomes available if their counter is greater or equal to their trip time
con7 = {}
for i in I:
for k in K:
model.addConstr((z[i,k] == 1) >> (d[i,k] >= T_time[i]))
con7[i,k] = model.addConstr((z[i,k] == 0) >> (d[i,k] <= (T_time[i]-1)))
# con7[i,k] = model.addConstrs((z[i,k] == 1) >> (d[i,k] >= T_time[i]) for i in I for k in range(0, len(K)))
# con7[i,k] = model.addConstrs((z[i,k] == 0) >> (d[i,k] <= (T_time[i]-1) ) for i in I for k in range(0, len(K)))#Constraint 9:
con9 = {}
for i in I:
for k in K:
if k != 0:
con9[i,k] = model.addConstr((c[i,k] == 0) >> (x[i,k-1] == 1))
con9[i,k] = model.addConstr((c[i,k] == 1) >> (x[i,k-1] == 0))
# con9[i,k] = model.addConstrs((c[i,k+1] == 0) >> (x[i,k] == 1) for i in I for k in range(0, len(K)-1))
# con9[i,k] = model.addConstrs((c[i,k+1] == 1) >> (x[i,k] == 0) for i in I for k in range(0,len(K)-1))0 -
Hi Jue,
This significantly reduces the solving time, thank you so much for your help!
0
Please sign in to leave a comment.
Comments
5 comments