Skip to main content

Nondeterministic behavior when solving same model fingerprint

Answered
Luke Kenworthy
  • Edited
Gurobi-versary
First Comment
First Question

I have a problem where I load the same model from the same .mps file and solve it multiple times, but get different answers between solves. I use the following code:

import gurobipy as gp
import sys

for j in range(100):
    m = gp.read(f"./test1_40.mps")
    m.optimize()
    gv = m.getVars()
    names = m.getAttr('VarName', gv)
    for i in range(m.SolCount):
        m.params.SolutionNumber = i
        xn = m.getAttr('Xn', gv)
        lines = ["{} {}".format(v1, v2) for v1, v2 in zip(names, xn)]
        with open(f"output_vals_{j}.txt", "w") as f:
            f.write("# Solution for model {}\n".format(m.modelName))
            f.write("# Objective value = {}\n".format(m.PoolObjVal))
            f.write("\n".join(lines))
When I compare the output of output_vals_0.txt and output_vals_20.txt, they are different. I get 
Pilot:_10000__Event:_9_Slot:_0 -0.0
in output_vals_0.txt and
Pilot:_10000__Event:_9_Slot:_0 1.0
in output_vals_20.txt. If I run the above multiple times, it's not always output_vals_20.txt that's different from output_vals_0.txt, but invariably at least one of the 100 output_vals files will be different from output_vals_0.txt
 
You can see the gist for the test1_40.mps file here: https://gist.github.com/lukekenworthy/4b9065bcffde9d2fe55d478b943e3040
 

Related to

1

Comments

6 comments

Date Votes
  • Riley Clement
    Gurobi Staff Gurobi Staff

    Hi Luke,

    Under default parameters we would expect the same results between runs.  I have tried reproducing this issue on my machine, using your code, with Gurobi 10.0.3 but every output file is the same.

    How are you running Gurobi?  Is the execution occurring on the same machine each time, or is it perhaps being executed on a cluster of machines?

    Also note that when you run

    with open(f"output_vals_{j}.txt", "w") as f:

    any prior content in the file is overwritten.  I'm guessing you want to append, rather than overwrite, and so should replace "w" with "a".

    - Riley

    0
  • Luke Kenworthy
    • Edited
    Gurobi-versary
    First Comment
    First Question

    Gotcha. I am running this on a single 2019 MacBook Pro with an Intel i7 processor. I'm using Gurobi Optimizer version 9.1.2 build v9.1.2rc0 (mac64) and Python 3.8.9.

     

    As for the "w" vs "a" issue, for this particular .mps file, I don't think it matters since the solution count is 1 for all of the solutions. But I modified the code as follows and am getting the same results.

    import gurobipy as gp
    import sys

    for j in range(100):
        m = gp.read(f"./test1_40.mps")
        m.optimize()
        gv = m.getVars()
        names = m.getAttr('VarName', gv)
        with open(f"output_vals_{j}.txt", "w") as f:
            f.write(f"solution count: {m.SolCount}\n")
            for i in range(m.SolCount):
                m.params.SolutionNumber = i
                xn = m.getAttr('Xn', gv)
                lines = ["{} {}".format(v1, v2) for v1, v2 in zip(names, xn)]
                f.write("# Solution for model {}\n".format(m.modelName))
                f.write("# Objective value = {}\n".format(m.PoolObjVal))
                f.write("\n".join(lines))
                f.write("---------------------------")

     

     

    0
  • Luke Kenworthy
    Gurobi-versary
    First Comment
    First Question

    I'm also running the following code to compare the outputs and find which ones are different from the first. I put the following code into a file titled "compare_outputs.py":

    import difflib
    with open("output_vals_0.txt") as f:
        first = f.readlines()

    for i in range(1, 100):
        filename = f"output_vals_{i}.txt"
        with open(filename) as f:
            cmp = f.readlines()
        diff = difflib.unified_diff(first, cmp, fromfile="output_vals_0.txt", tofile=filename)
        print(i)
        for line in diff:
            print(line)
        print("\n\n\n")

    and then I run it using

    python3 compare_outputs.py > diffs.txt
    0
  • Riley Clement
    Gurobi Staff Gurobi Staff
    • Edited

    Thanks Luke, what version of Gurobi are you using?

    Any chance you could upload log files for runs with different solutions to a share drive, or are the log files identical?

    - Riley

    0
  • Luke Kenworthy
    Gurobi-versary
    First Comment
    First Question

    I'm using Gurobi Optimizer version 9.1.2 build v9.1.2rc0 (mac64). 

    Okay so I modified the code to write to a log file by changing the start of the code to:

    import gurobipy as gp
    import sys
    for j in range(100):
        m = gp.read(f"./test1_40.mps")
        m.setParam("LogFile", f"log_{j}.log")
        m.optimize()
    In the diff file, I noticed that output_vals_0.txt and output_vals_21.txt were different. Their log files are linked here and here respectively. I noticed that, in log_0.log, the second time Root relaxation shows up, it displays \(\texttt{Root relaxation: interrupted, 0 iterations, 0.00 seconds}\). In log_21.log, the second Root relation shows \(\texttt{Root relaxation: objective 3.700000e+01, 40 iterations, 0.00 seconds}\). So that might account for the difference, though it's still unclear why it would be behaving nondeterministically.
    0
  • Riley Clement
    Gurobi Staff Gurobi Staff

    Hi Luke,

    I can reproduce this issue on 9.1.2.  I'll open up a ticket through our help center so I can address the issue there and throw more resources at it.  You should receive an email to that effect soon.

    - Riley

     

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post