Skip to main content

Incompatible dimensions in Python

Answered
Qiran Wu
Edited
  • Gurobi-versary
  • First Comment
  • First Question

Hi,

I'm using Gurobi in Python to solve a Mixed Integer Programming.

This is a reformulation of a Neural Network problem. 

My `x` is a 1-D vector of 200 variables. The first layer has 50 neurons. 

`y = Wx + b` where `W` has the shape of (50, 200) and `b` has the shape of (50,). 

m = gp.Model("MIP")
x = m.addMVar(200)
y = m.addMVar((num_layers, 50))  # num_layers is 3
m.addConstr(W @ x + b == y[0])

However, ``W @ x `` shows me

<gurobi.MLinExpr, 50 rows, 200 cols, 1990 nnz>

But (W @ x).shape shows me (50,). Is this correct?

 

And my last layer only has one neuron, so the output is just a scalar.

I tried the following code

output = m.addVar()
m.addConstr(W @ y[2] + bias == output)

But this constraint shows me the Error code -1: Incompatible dimensions

 

 

0

Comments

10 comments

Date Votes
  • Official comment
    Simranjit Kaur
    • Gurobi Staff
    Edited
    This post is more than three years old. Some information may not be up to date. For current information, please check the Gurobi Documentation or Knowledge Base. If you need more help, please create a new post in the community forum. Or why not try our AI Gurobot?.
  • Jaromił Najman
    • Gurobi Staff
    Edited

    Hi Qiran,

    The output

    <gurobi.MLinExpr, 50 rows, 200 cols, 1990 nnz>

    is correct as \(\texttt{W @ x}\) represents the constraint matrix which consists of 50 constraints and 200 variables.

    The output \(\texttt{(50, )}\) of \(\texttt{(W @ x).shape}\) is also correct as \(\texttt{shape}\) always returns a 1-dimensional object as described in the documentation of the MLinExpr object.

    Modeling the last layer cannot work this way, since the dimension of \(\texttt{y[2]}\) is 50 and not 200. Moreover, even if the dimension of \(\texttt{y[2]}\) would be 200, the dimensions of \(\texttt{bias}\) and \(\texttt{output}\) would have to be 50 in order to fit the dimension of the resulting constraint matrix. This is due to the fact that the operation \(\texttt{@}\) does not perform the classic matrix multiplication, which one might expect, but rather constructs a linear matrix expression. I would recommend to write the model to an LP file via

    m.write("myLP.lp")

    after you add the

    W @ x + b == y[0]

    constraint in order to get a better feeling of how the \(\texttt{@}\) operator works.

    Best regards,
    Jaromił

    0
  • Qiran Wu
    Edited
    • Gurobi-versary
    • First Comment
    • First Question

    Hi Jaromił,

     

    I forgot to specify in the last layer the `W`'s shape is `(50,)`. Sorry for the misleading.

    Hope this code makes it more clear.

    output = m.addVar()
    m.addConstr(W[2] @ y[2] + b[2] == output)

    `\text{b[2]}` is a scalar as the output. 

    `\text{(W[2] @ y[2]).shape}` gives me `(1,)` , `\text{(W[2] @ y[2] + bias[2]).shape}` also gives `(1,)`. So I think so far is correct. 

    The error happens at the "`\text{== output}`". 

     

    Thanks and regards,

    0
  • Jaromił Najman
    • Gurobi Staff
    Edited

    Hi Qiran,

    Thank you for the clarification.

    The error occurs, because the variable \(\texttt{output}\) is not handled as an \(\texttt{MVar()}\) of size 1. Using

    output = m.addMVar(1)

    solves the issue. I have noted this as a future feature request. Thank you for pointing this out.

    Best regards,
    Jaromił

    0
  • Qiran Wu
    Edited
    • Gurobi-versary
    • First Comment
    • First Question

    Thanks, Jaromił. This works.

    And I want to know if Gurobipy supports broadcast and element-wise multiplication.

    For example, I want to add a constraint

    m.addConstr(y <= c*(1-x))

    where `y` and `x` are 1-D variables with the same shape. `c` is a 1-D vector whose shape is equal to `x`'s and `y`'s.

    My question is:

    1. Does `\text{1-x}` broadcast automatically? i.e,  `text{1-x = [1- x[i] for i in range(len(x))]}`. Although `(1-x).shape` gives the same shape as `x`, I don't know if it first broadcasts `1` to a vector than do the minus element-wisely. 
    2. Since Gurobipy supports element-wise comparison, i.e, `\text{a <= b}` is equivalent to `\text{a[i] <= b[i] for i in range(len(a))}`, I want to do element-wise multiplication for vector `c` and `(1-x)` so that I can avoid some cumbersome expression. 
    0
  • Jaromił Najman
    • Gurobi Staff
    Edited

    Hi Qiran,

    The constraint as it is, tries to combine two features which are not compatible.
    For

    x = m.addMVar(2, name = "x")
    y = m.addMVar(2, name = "y")
    c = np.array([4.0, -1.0])

    the constraint

    m.addConstr(y <= 1 - x)

    produces the constraints

    R0: x[0] + y[0] <= 1
    R1: x[1] + y[1] <= 1

    In order to multiply the \(\texttt{MVar}\) \(x\) with \(c\), \(c\) has to be a matrix, e.g.,

     c = np.random.rand(2,2)

    making the multiplication with \(1-x\) not possible. So the answer to your first point is that we do not broadcast the 1 to do the minus element-wisely. I would recommend to use the Model.write() in order to analyze what is happening at each operation step.

    Regarding your second question, I assume that you want to achieve that \(c \cdot (1-x)\) results in

    c[0]*(1-x[0]) + c[1]*(1-x[1]) + ...

    correct?

    You can achieve that via

    x = m.addMVar(2, name = "x")
    y = m.addMVar(1, name = "y")
    c = np.array((4,-1))
    m.addConstr(y <= sum(c) - c @ x )

    which provides the constraint

     R0: 4 x[0] - x[1] + y[0] <= 3

    Again, I would recommend to make use of the Model.write() function to analyze which operation produces which set of constraints.

    Best regards,
    Jaromił

    0
  • Qiran Wu
    • Gurobi-versary
    • First Comment
    • First Question

    Hi Jaromił,

     

    Model.write() is helpful, Thank you!

    For this constraint, 

    y <= c*(1-x)

    what I want is 

    y[0] <= c[0]*(1-x[0])
    y[1] <= c[1]*(1-x[1])
    y[2] <= c[2]*(1-x[2])
    ...

    I know I can write 

    m.addConstrs(y[i] <= c[i]*(1-x[i]) for i in range(len(x)))

    But I just wonder that if there is a more elegant way to realize this element-wise multiplication. 

    0
  • Jaromił Najman
    • Gurobi Staff

    Hi Qiran,

    I understand, thank you for clarifying.

    I have to admit that to me

    m.addConstrs(y[i] <= c[i]*(1-x[i]) for i in range(len(x)))

    is already an elegant way to achieve what you want.

    A different way using \(\texttt{MVars}\) would be

    x = m.addMVar(2, name = "x")
    y = m.addMVar(2, name = "y")
    c = np.array([4.0, -1.0])
    d = np.diag(c)
    m.addConstr(y <= c - d @ x)

    Note that one has to expand the term \(c \cdot (1-x)\) as the desired multiplication is currently not supported by \(\texttt{MVars}\).

    Best regards,
    Jaromił

    0
  • Qiran Wu
    • Gurobi-versary
    • First Comment
    • First Question

    Great! Thank you!

    0
  • Maliheh Aramon
    • Gurobi Staff
    Hi Qiran,
     
    Gurobi 10.0 was recently released. Included in this release is the extension of Gurobi Matrix API which enables natural model building using matrix based expressions relying on NumPy concepts such as vectorization and broadcasting.
     
    As an example, the code snippet below runs as you intuitively expect and adds 3 constraints to the model.
    m = gp.Model()

    x = m.addMVar(3, name="x")
    y = m.addMVar(3, name="y")
    c = np.random.rand(3)

    m.addConstr(y <= c * (1 - x))
    Checkout the Matrix-friendly Modeling with Gurobipy webinar if you would like to learn more about this new functionality. 
     
    Please continue submitting new community posts for any bugs you might encounter in the future or for any comments/questions you might have. Users like you help to make Gurobi better!
     
    Best regards,
    Maliheh
    0

Post is closed for comments.

Didn't find what you were looking for?

New post