Solving linnear regression in Gurobi python

I am new to gurobi and decided to solve a well known problem as regression. I found this notebook

https://gurobi.github.io/modeling-examples/linear_regression/l0_regression.html

where an L0 penalized regression model was solved and I took just the part of the regression model out of it. However, when I solve this problem, I get a really strange solution, totally different from the actual correct regression solution.

The code I am running is:

import gurobipy as gp
from gurobipy import GRB
import numpy as np
from sklearn.datasets import load_boston
from itertools import product

boston = load_boston()
x = boston.data
x = x[:, [0, 2, 4, 5, 6, 7, 10, 11, 12]] # select non-categorical variables
response = boston.target

samples, dim = x.shape

regressor = gp.Model()

# Append a column of ones to the feature matrix to account for the y-intercept
x = np.concatenate([x, np.ones((samples, 1))], axis=1)

# Decision variables
beta = regressor.addVars(dim + 1, name="beta") # Beta

# Objective Function (OF): minimize 1/2 * RSS using the fact that
# if x* is a minimizer of f(x), it is also a minimizer of k*f(x) iff k > 0
Quad = np.dot(x.T, x)
lin = np.dot(response.T, x)
obj = sum(0.5 * Quad[i, j] * beta[i] * beta[j] for i, j in product(range(dim + 1), repeat=2))
obj -= sum(lin[i] * beta[i] for i in range(dim + 1))
obj += 0.5 * np.dot(response, response)

regressor.setObjective(obj, GRB.MINIMIZE)

regressor.optimize()
beta_sol_gurobi = np.array([beta[i].X for i in range(dim+1)])

The solution provided by this code is

array([1.22933632e-14, 2.40073891e-15, 1.10109084e-13, 2.93142174e+00,
       6.14486489e-16, 3.93021623e-01, 5.52707727e-15, 8.61271603e-03,
       1.55963041e-15, 3.19117429e-13])

While the true linear regression solution should be

from sklearn import linear_model
lr = linear_model.LinearRegression()
lr.fit(x_boston, response)
lr.coef_
lr.intercept_

That yields,

array([-5.23730841e-02, -3.35655253e-02, -1.39501039e+01,  4.40955833e+00,
       -7.33680982e-03, -1.24312668e+00, -9.59615262e-01,  8.60275557e-03,
       -5.17452533e-01])
29.531492975441015

Official comment

Simranjit Kaur

Gurobi Staff

June 23, 2025 03:03 Edited

This post is more than three years old. Some information may not be up to date. For current information, please check the Gurobi Documentation or Knowledge Base. If you need more help, please create a new post in the community forum. Or why not try our AI Gurobot?.

Jaromił Najman

December 07, 2020 10:08

This is a cross-post to Absurd solution using Gurobi Python in regression

Solving linnear regression in Gurobi python

Comments

Didn't find what you were looking for?