What is a Minimal Reproducible Example (MRE)?
A MRE is a code example that:
- is as small as possible
- contains all data, variables, functions etc needed to reproduce the problem (but nothing more)
- has been tested by you to make sure it is reproducible
The final bullet point means that you should run your example in a new file (or with a new kernel if you're using Python with Jupyter notebooks) before posting it in the community forum.
Do use code blocks.
Do make your question clear (try highlighting a sentence in bold).
Don't paste screen shots of code.
Don't paste code that is not properly formatted (and indented if Python).
Why do we want a Minimal Reproducible Example?
People who answer your questions in the community forum are often doing so in their own time - and this includes Gurobi support staff. Your time is precious, but so is theirs. You should aim to make your problem as easy as possible for others to understand and to solve.
If people see that you have made an effort to make your example minimal and reproducible, then you are more likely to get a faster and more thorough response.
A MRE will make the problem easier to identify for a number of reasons. If your example is not reproducible, then a significant amount of time may be spent with replies back and forth until enough information has been provided to make it reproducible.
Let's see how it's done!
Take, for example, the following code.
from gurobipy import *
import numpy as np
from gurobipy import Model, LinExpr, QuadExpr, quicksum, GRB
from itertools import combinations
#data
def make_data():
epsilon = 1e-6
# input data
k = 5
x = np.array([[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
m = 3
return epsilon, x, k, m
def make_model(data):
epsilon, x, k, m = data
# initialize empty model
model = Model("model")
model.setParam("NonConvex", 2)
model.setParam("MIPGap", 0.01)
# add variables
var_w = model.addVars(range(k), lb=0, ub=1, vtype=GRB.CONTINUOUS)
var_cos = model.addVars(combinations(range(k), 2), lb=-1, ub=1, vtype=GRB.CONTINUOUS)
ox_1 = model.addVars(range(m), lb=0, ub=1, vtype=GRB.CONTINUOUS)
ox_2 = model.addVars(range(m), combinations(range(k), 2), lb=0, ub=1, vtype=GRB.CONTINUOUS)
sox_2 = model.addVars(range(m), combinations(range(k), 2), lb=0, ub=1, vtype=GRB.CONTINUOUS)
sox_3 = model.addVars(range(m), combinations(range(k), 2), lb=0, ub=1, vtype=GRB.CONTINUOUS)
# normalize x
x_normed = x / x.sum(axis=0)
for i in range(m):
model.addConstr(ox_1[i] == sum(var_w[h] * x_normed[i][h] for h in range(k)))
for i in range(m):
for h, l in combinations(range(k), 2):
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
model.addGenConstrPow(ox_2[i][h][l], sox_2[i][h][l], 0.5, "gf", "FuncPieces=1000")
model.addConstr(sox_3[i][h][l] == sox_2[i][h][l] * var_cos[h][l])
for i in range(m):
ox_3 = 0
for h, l in combinations(range(k), 2):
ox_3 += 2 * soc_3[i][h][l]
return model
def main():
data = make_data()
model = make_model(data)
model.optimize()
if model.getAttr("Status") == GRB.OPTIMAL:
for var in model.getVars():
if var.getAttr("x") != 0:
print(var.getAttr("VarName"), var.getAttr("X"))
print("\n Reach optimum with: {}".format(model.getObjective().getValue()))
else:
print("\n Maybe INFEASIBLE model.")
if __name__ == '__main__':
main()
When we run this code, we get the error: "KeyError: 0"
Let's create a community post! Our code is already reproducible - if you were to copy and paste it onto your own machine you would get the same result as everybody else (assuming import statements succeed). But this is a lot of code - it is far from minimal.
Now, providing the short error message in the community post is better than simply stating "I have an error", but we can do better. When constructing a MRE, we want to provide as much useful information as possible. If information is not helpful then it is unhelpful! And if we have unhelpful information, then our example cannot be minimal.
When an error is produced, we will generally receive a traceback that shows exactly which line of code produced the error. Is this helpful? Yes!
For example, when executing the above code in a Python script from the terminal, we get the following traceback:
Traceback (most recent call last):
File "/Users/tutorials/MRE.py", line 67, in <module>
main()
File "/Users/tutorials/MRE.py", line 53, in main
model = make_model(data)
File "/Users/tutorials/MRE.py", line 41, in make_model
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
If we execute the same code in a Jupyter Notebook, we get a slightly different traceback (but with the same error), which is even more informative:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[441], line 67
64 print("\n Maybe INFEASIBLE model.")
66 if __name__ == '__main__':
---> 67 main()
Cell In[441], line 53, in main()
51 def main():
52 data = make_data()
---> 53 model = make_model(data)
54 model.optimize()
56 if model.getAttr("Status") == GRB.OPTIMAL:
Cell In[441], line 41, in make_model(data)
39 for i in range(m):
40 for h, l in combinations(range(k), 2):
---> 41 model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
42 model.addGenConstrPow(ox_2[i][h][l], sox_2[i][h][l], 0.5, "gf", "FuncPieces=1000")
43 model.addConstr(sox_3[i][h][l] == sox_2[i][h][l] * var_cos[h][l])
KeyError: 0
Both tracebacks tell us something very important. The error is in the following line:
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
Now that we know which line has the error, we should ask ourselves the following question:
This line is executed inside for loops and should be executed many times. Does it get executed even once? Or does it run several times before producing the error?
To answer this let's put a print statement just above the erroneous line, i.e.,
print(i,h,l)
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
Now when we run the code, we see the output "0 0 1" printed before the error message. This is the only output, so we now know that the line is not even executed once. We are now in a good position to start removing unhelpful pieces of code.
Remove anything that would execute after the line with the error
If code will never get executed because the error occurs before it, then this code is not helpful. We should remove some lines (commented out below):
for i in range(m):
for h, l in combinations(range(k), 2):
print(i,h,l)
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
# model.addGenConstrPow(ox_2[i][h][l], sox_2[i][h][l], 0.5, "gf", "FuncPieces=1000")
# model.addConstr(sox_3[i][h][l] == sox_2[i][h][l] * var_cos[h][l])
# for i in range(m):
# ox_3 = 0
# for h, l in combinations(range(k), 2):
# ox_3 += 2 * soc_3[i][h][l]
# return model
Since the error occurs in the make_model function, we can also remove any code that occurs after it in the main() function:
def main():
data = make_data()
model = make_model(data)
# model.optimize()
# if model.getAttr("Status") == GRB.OPTIMAL:
# for var in model.getVars():
# if var.getAttr("x") != 0:
# print(var.getAttr("VarName"), var.getAttr("X"))
# print("\n Reach optimum with: {}".format(model.getObjective().getValue()))
# else:
# print("\n Maybe INFEASIBLE model.")
if __name__ == '__main__':
main()
Remove for-loops and fix the values of i,j and l
Since we know the erroneous line will produce the error when (i, h, l) is (0, 0, 1), we don't need the loops. They are not helpful, and, again, we should remove anything that is not helpful:
# normalize x
x_normed = x / x.sum(axis=0)
for i in range(m):
model.addConstr(ox_1[i] == sum(var_w[h] * x_normed[i][h] for h in range(k)))
i,h,l = 0,0,1
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
Remove code that is not related to the error
Let's guess that the model parameters that are set near the beginning of the make_model function do not affect the problem and remove them (we'll check at the end to make sure we still get the same error). We can also try and remove several imports that aren't being used. If we accidentally remove something that is needed to reproduce the error, then we will know when we test for reproducibility!
Tip: Some code editors and IDEs such as Visual Studio Code make it very easy to identify which variables and imports are not used, by coloring them differently.
There are several variables, constraints, and import statements which have nothing to do with the erroneous line. After removing them, we have the following:
import numpy as np
from gurobipy import Model, GRB
from itertools import combinations
#data
def make_data():
epsilon = 1e-6
# input data
k = 5
x = np.array([[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
m = 3
return epsilon, x, k, m
def make_model(data):
epsilon, x, k, m = data
# initialize empty model
model = Model("model")
# add variables
var_w = model.addVars(range(k), lb=0, ub=1, vtype=GRB.CONTINUOUS)
ox_2 = model.addVars(range(m), combinations(range(k), 2), lb=0, ub=1, vtype=GRB.CONTINUOUS)
# normalize x
x_normed = x / x.sum(axis=0)
i, h, l == 0, 0, 1
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
def main():
data = make_data()
model = make_model(data)
if __name__ == '__main__':
main()
We have just deleted a lot of code. We should run this code to check it still gives the error we expect... it does!
Simplify the script by removing functions
Often, putting your code into functions will be a good idea. Even if the functions are called once, they can still help by being self-documenting if named well - which is what has been done in this example. However, we don't need these functions to create a MRE - if it is not helpful, then it is unhelpful! Let's remove them:
import numpy as np
from gurobipy import Model, GRB
from itertools import combinations
epsilon = 1e-6
# input data
k = 5
x = np.array([[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
m = 3
# initialize empty model
model = Model("model")
# add variables
var_w = model.addVars(range(k), lb=0, ub=1, vtype=GRB.CONTINUOUS)
ox_2 = model.addVars(range(m), combinations(range(k), 2), lb=0, ub=1, vtype=GRB.CONTINUOUS)
# normalize x
x_normed = x / x.sum(axis=0)
i, h, l == 0, 0, 1
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
This is now looking a lot better, and we've certainly narrowed down on the important parts. Can we do better? We could try to figure out what part of the line is producing the error!
Where exactly is the error?
The line that produces the error has many things happening in it. There is the call to addConstr, indexings, multiplications, constraint definition... let's separate out some of these things into their own lines, from simplest to more complex, by adding some new lines:
i, h, l == 0, 0, 1
#---- new lines -----
ox_2[i][h][l]
var_w[h]
x_normed[i][h]
var_w[l]
x_normed[i][l]
var_w[h] * x_normed[i][h]
var_w[h] * x_normed[i][h] * var_w[l]
var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l]
ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l]
#--------------------
model.addConstr(ox_2[i][h][l] == var_w[h] * x_normed[i][h] * var_w[l] * x_normed[i][l])
Can you see what we're trying to do? When we execute the code we are hoping that the traceback will tell us exactly where the error is. And the traceback that results is:
Traceback (most recent call last):
File "/Users/tutorials/MRE.py", line 23, in <module>
ox_2[i][h][l]
KeyError: 0
So we encountered the problem on our very first line! There is something wrong with ox_2[i][h][l].
The simplest MRE possible?
Now we know exactly what the error is we can reduce our MRE to
from gurobipy import Model, GRB
from itertools import combinations
# input data
k = 5
m = 3
model = Model("model")
ox_2 = model.addVars(range(m), combinations(range(k), 2), lb=0, ub=1, vtype=GRB.CONTINUOUS)
i, h, l == 0, 0, 1
ox_2[i][h][l]
This is pretty good. Although technically the following is the the simplest MRE for this example:
from gurobipy import Model
model = Model()
ox_2 = model.addVars([0], [0], [1])
ox_2[0][0][1]
In this example, there's everything that is needed to reproduce the error and nothing that isn't.
Appendix: The pleasure of finding things out
Although the purpose of this article was to show how a MRE can be produced, we can go one little step further - figuring out what the error is - we're so very close in this example.
We know there is something wrong with what we're doing with ox_2. What is ox_2? If we execute the following code
print(type(ox_2))
then Python will tell us that ox_2 is a gurobipy.tupledict
If we find the documentation for gurobipy.tupledict then we see, half way down the page, the following:
For example,d[1,2]returns the value associated with tuple(1,2).
Perhaps ox_2[0][0][1] is the wrong syntax? Should it have been ox_2[0, 0, 1]?
We test this in our MRE and find we no longer have an error. Problem solved!