model.addConstrs breaking sometimes after Python version change

回答済み

Javier Jiménez Sicardo

2024年09月09日 00:09 編集済み

• 
• 
• 

Hello:

I was using Python 3.10 and this works, but it doesn't in Python 3.12.

Setup: Anaconda Python last version, using Spyder IDE. Changing Python versions from environments. Python 3.12 is default environment and has more packages, but these shouldn't interfere.

Proof of concept code:

import gurobipy as g

m = g.Model(name='Caso TFM base')

# Model indexes
NAMES  = ["N1", "N2"]
TIMES  = [1,2,3]
OTHERS = [1,2]

NAMES_TIMES_OTHER = tuple((name, time, other) for name in NAMES for time in TIMES for other in OTHERS)

# Ponderator
p = {k: 0.1*(i+1) for (i,k) in enumerate(NAMES)} 

sa = m.addVars(NAMES_TIMES_OTHER, vtype=g.GRB.CONTINUOUS, lb=0, name="sa")
v  = m.addVars(TIMES,             vtype=g.GRB.CONTINUOUS, lb=0, name="v")

m.update()

m.addConstrs( (v[t] == g.quicksum([g.quicksum(sa.select(i,t,'*'))*p[i] for i in NAMES]) for t in TIMES) )

 

In Python 3.10, this code runs without any problem and generates expected constraints. However, in Python 3.12 (not sure if Python 11), I get an error in the last sentence:

runfile('C:/<path>/test_error_python_version.py', wdir='C:/<path>')

Traceback (most recent call last):

File C:\ProgramData\anaconda3\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec

exec(code, globals, locals)

File c:\<path>\test_error_python_version.py:26

m.addConstrs( (v[t] == g.quicksum([g.quicksum(sa.select(i,t,'*'))*ponderator[i] for i in NAMES]) for t in TIMES) )

File src\\gurobipy\\model.pxi:3878 in gurobipy.Model.addConstrs

File src\\gurobipy\\model.pxi:255 in genexpr

File src\\gurobipy\\model.pxi:255 in genexpr

KeyError: 'i'

 

What is more intriguing, trying to debug this, you can run the full excerpt in Python 3.12 without m.addConstrs() on terminal and get the expected behaviour (v is the ponderated sum of names for a given time), so it shouldn't be a problem, also extract the generator object in a list to get desired constraints but doesn't add to the model

list((v[t] == g.quicksum([g.quicksum(sa.select(i,t,'*'))*ponderator[i] for i in NAMES]) for t in TIMES))

[<gurobi.TempConstr: v[1] == 0.1 sa[N1,1,1] + 0.1 sa[N1,1,2] + 0.2 sa[N2,1,1] + 0.2 sa[N2,1,2]>,
 <gurobi.TempConstr: v[2] == 0.1 sa[N1,2,1] + 0.1 sa[N1,2,2] + 0.2 sa[N2,2,1] + 0.2 sa[N2,2,2]>,
 <gurobi.TempConstr: v[3] == 0.1 sa[N1,3,1] + 0.1 sa[N1,3,2] + 0.2 sa[N2,3,1] + 0.2 sa[N2,3,2]>]

so this rules out any problem related to the iterator i unless I am missing something.

A workaround is resorting to use the older version of Python or doing it equation by equation:

for t in TIMES:
    expr = g.quicksum([g.quicksum(sa.select(i, t, '*')) * ponderator[i] for i in NAMES])
    m.addConstr(v[t] == expr)

 

But, what is that and why does that happen? Is it a bug or a feature change onwards from Python 3.10? All the other m.addConstrs I use have no such problem.

Thank you in advance.

 

Edit: made names shorter so code and output can be read easier.

関連

• Model Building
• Python

0

• 

  Riley Clement

  • Gurobi Staff

  2024年09月09日 00:23 編集済み

  Hi Javier,

  I don't have a full answer at this stage, just wanted to note that it seems to work ok in Python 3.12 if you remove the square brackets from the constraint, i.e.

  m.addConstrs( (v[t] == g.quicksum(g.quicksum(sa.select(i,t,'*'))*ponderator[i] for i in NAMES) for t in TIMES) )

  I suspect the issue is coming from nested quicksums.  For stylistic reasons I try to avoid nested quicksums wherever possible, e.g.

  m.addConstrs( (v[t] == g.quicksum(v*ponderator[i] for v in sa.select(i,t,'*') for i in NAMES) for t in TIMES) )

  I'll see if any of our developers have a comment on this issue.

  - Riley

   

  0
• 

  Javier Jiménez Sicardo

  2024年09月09日 01:08

  • 
  • 
  • 

  Hello Riley,

  Thank you for your prompt response and comment.

  The first code (removing the square brackets) works well and does what it's needed. I like adding these to quicksums to inspect before adding in a bigger model.

  The second code only works for me if I put the NAMES before the variable query, i.e.:

  m.addConstrs( (v[t] == g.quicksum( val*p[i] for i in NAMES for val in sa.select(i,t,'*')  ) for t in TIMES) )

  I've changed v to val for clarity. These are equal but I like to translate the mathematical model directly. I'll consider that in the future when I get and interiorise the change.

  So this solves the problem for me.

  I'm very curious for a technical explanation on this issue: if I recall correctly addConstrs just exhausts the generator or unpacks what it is sent, so both should be functionally identical. There's a big chance I wouldn't understand it, however.

  Thank you very much for your time,

  Javier

  0
• 

  Riley Clement

  • Gurobi Staff

  2024年09月09日 01:31

  Hi Javier,

  There is a bit of black magic happening under the hood of gurobipy when you use addConstrs.  This function does introspection on the provided generator frame and extracting the keys correctly requires looking both at the local scope variables and the bytecode of the generator expression.

  Without looking into the details I guess that there was a small change in CPython 3.12 for how the bytecode is created and with the nested quicksums you have found an edge case which breaks what gurobipy is trying to do.

  Although addConstrs is still part of many examples found on our website, we no longer recommend it.  From another of our recent posts on addConstrs, the following comment was made by our dev Simon:

  To give some context: gurobipy is now ~15 years old; some features are there to support compact modelling statements in older versions of Python, and are maintained for users who rely on them. I would say that this particular method is now redundant, since f-strings arrived in Python 3.6 and made producing name strings very easy. For example this code from the documentation:

  constrs = m.addConstrs(
      (x[i,j] == 0 for i in range(4) for j in range(4) if i != j),
      name='c'
  )

  is exactly equivalent to this:

  constrs = gp.tupledict({
      (i, j): m.addConstr(x[i, j] == 0, name=f"c[{i},{j}]")
      for i in range(4)
      for j in range(4)
      if i != j
  })

  or this, if you don’t really need to keep track of the constraint objects returned:

  for i in range(4):
      for j in range(4):
          if i != j:
              m.addConstr(x[i, j] == 0, name=f"c[{i},{j}]")

  or this, to make better use of the Python standard library for this specific case:

  for i, j in itertools.permutations(range(4), r=2):
      m.addConstr(x[i, j] == 0, name=f"c[{i},{j}]")

  To me the latter examples are preferable; they are self-documenting and typically faster. I recommend using that style instead of addConstrs.

  - Riley

  0

サインインしてコメントを残してください。