Recovering from DISCONNECTED in Gurobi Instant Cloud

Answered

Timo Voipio

October 27, 2025 14:21

• 
• 
• 

Hi,

I'm using Gurobi Instant Cloud to solve MIP problems constructed with Python (gurobipy 12.0). The client is an AWS EC2 instance. Usually this works well, but one of my longer runs was interruped by a disconnection while running Model.optimize(). The error took place while solving the root simplex.

From the client side, the error message was (copypasted from the Jupyter notebook):

File src/gurobipy/_model.pyx:901, in gurobipy._model.Model.optimize()

GurobiError: Job state is 'DISCONNECTED'

The log on the server side does not show anything, it just… stops.

My intuition was that this was some sort of transient network error, but as the optimization was running unattended, I could not immediately retry to see if connectivity was restored within seconds or minutes.

Is there a way to prepare for, detect, and recover from such errors, preferably by not losing all work done so far by the compute server? This error took place about 90 minutes in, and I would have preferred not to have that time go to waste…

0

• 

  Elisabeth Rodriguez Heck

  • Gurobi Staff

  October 27, 2025 14:37

  Hi Timo,

  We'll create a ticket from your post and handle this directly.

  Best regards,

  Elisabeth

  0
• 

  Gwyneth Butera

  • Gurobi Staff

  August 04, 2026 16:34

  The issue here was likely caused by some transient network issues from AWS. One thing you can try is increasing the ServerTimeout parameter from the default 60 seconds. This will give your client machine more time to recover from those network issues. 

  0
• 

  Igor Ganapolsky

  August 13, 2026 18:49

  Adding a practical recovery pattern on top of Gwyneth's ServerTimeout tip: 1. Treat DISCONNECTED as a hard session loss — do not reuse the old Env/Model. Build a fresh gp.Env() with current Instant Cloud credentials, then a new Model. 2. Before/during long MIPs, checkpoint progress yourself: write periodic .mst MIP starts (or incumbent solutions) from a callback / external watchdog so a reconnect does not lose 90 minutes of work. 3. On the client EC2 side, wrap Model.optimize() in retry logic that only rebuilds after DISCONNECTED, reloads the latest .mst, and continues. 4. Increase ServerTimeout as Gwyneth suggested, and keep the client process from sleeping (Jupyter idle/timeout can look identical to a cloud disconnect). That combination (timeout + restartable Env + MIP-start checkpoints) is what usually turns a one-shot long cloud run into something you can resume.

  0

Please sign in to leave a comment.