Skip to main content

Software Testing

Answered

Comments

4 comments

  • Francisco Zenteno
    First Comment

    This is a good question, hopefully the staff can elaborate on the best testing strategies in Gurobi

    0
  • Annika Burmeister
    Gurobi-versary
    First Comment

    As we are dealing with unit testing in gurobi applications right now, I'm also highly interested in this topic.

    0
  • Alison Cozad
    Gurobi Staff Gurobi Staff

    This is certainly a tricky topic. Luckily, Yash Puranik, a Principal Data Scientist at Aimpoint Digital, has put some thought into this topic and shared his and Aimpoint's principles for deploying, managing, and maintaining optimization models effectively in production which touches on testing.

    I recommend taking a look at this webinar recording: Gurobi and Aimpoint Digital - Deploying and Maintaining Optimization Models in Production

    1
  • Yash Puranik
    First Comment
    Gurobi-versary

    Thanks Alison Cozad!

    Naman Singla: Testing optimization models covers many many aspects. I believe you are specifically interested in unit testing? If yes, I would recommend doing testing to answer a few specific questions instead of trying to assert the solution returned is equal to solution expected. For example:

    1. Is the solution feasible? (Does it satisfy all model constraints? https://www.gurobi.com/documentation/9.5/refman/maxvio.html and lhttps://www.gurobi.com/documentation/current/refman/optimization_status_codes.html  may be helpful. I usually implement completely independent feasibility checkers (and have someone not familiar with the math model implement the feasibility checkers if possible) to allow for a truly unbiased check.)

    2. Is the solution implementable? (This may require including some business knowledge. For example, am I trying to sell products without ever turning any machines on and making them)

    3. Is the solution non-trivial? (Often, setting all variables to 0 or 1 is a valid but practically useless solution)

    4. Are the numbers realistic? (I would be wary of both, very small numbers (say 1e-5) and very large numbers (1e12)). This would also suggest the need for better scaling in the modeling phase as well.

    Minor changes in constraints, objectives, dataset can indeed lead to significant changes in optimal solutions: that is the nature of optimization problems and the value of using a solver like Gurobi that helps you find those optimal solutions fast even if the solutions/changes are non-intuitive.The best approach is to have software engineers, business stakeholders and OR experts working in tandem to ensure robustness, maintainability and long term value.

    2

Please sign in to leave a comment.