Software Testing

Answered

Naman Singla

• August 18, 2023 07:59

Hi All,

In Gurobi model if there are any minor changes in constraints or objective it creates lot of different solutions, from software engineering perspective, what is the best way to cover testing scenarios. For such changes as there are lot of permutation that can be feasible and covering all scenarios might not be possible by leveraging on unit test cases. Can someone please recommend best testing strategy? 

1

• 

  Francisco Zenteno

  • December 28, 2023 19:59

  

  This is a good question, hopefully the staff can elaborate on the best testing strategies in Gurobi

  0
• 

  Annika Burmeister

  • March 06, 2024 09:47

  

  

  As we are dealing with unit testing in gurobi applications right now, I'm also highly interested in this topic.

  0
• 

  Alison Cozad

  Gurobi Staff

  • March 11, 2024 15:32

  This is certainly a tricky topic. Luckily, Yash Puranik, a Principal Data Scientist at Aimpoint Digital, has put some thought into this topic and shared his and Aimpoint's principles for deploying, managing, and maintaining optimization models effectively in production which touches on testing.
  
  I recommend taking a look at this webinar recording: Gurobi and Aimpoint Digital - Deploying and Maintaining Optimization Models in Production

  1
• 

  Yash Puranik

  • March 11, 2024 15:58

  

  

  Thanks Alison Cozad!

  Naman Singla: Testing optimization models covers many many aspects. I believe you are specifically interested in unit testing? If yes, I would recommend doing testing to answer a few specific questions instead of trying to assert the solution returned is equal to solution expected. For example:

  1. Is the solution feasible? (Does it satisfy all model constraints? https://www.gurobi.com/documentation/9.5/refman/maxvio.html and lhttps://www.gurobi.com/documentation/current/refman/optimization_status_codes.html  may be helpful. I usually implement completely independent feasibility checkers (and have someone not familiar with the math model implement the feasibility checkers if possible) to allow for a truly unbiased check.)

  2. Is the solution implementable? (This may require including some business knowledge. For example, am I trying to sell products without ever turning any machines on and making them)

  3. Is the solution non-trivial? (Often, setting all variables to 0 or 1 is a valid but practically useless solution)

  4. Are the numbers realistic? (I would be wary of both, very small numbers (say 1e-5) and very large numbers (1e12)). This would also suggest the need for better scaling in the modeling phase as well.

  Minor changes in constraints, objectives, dataset can indeed lead to significant changes in optimal solutions: that is the nature of optimization problems and the value of using a solver like Gurobi that helps you find those optimal solutions fast even if the solutions/changes are non-intuitive.The best approach is to have software engineers, business stakeholders and OR experts working in tandem to ensure robustness, maintainability and long term value.

  2

Please sign in to leave a comment.