Pyomo performance enhancement list

[jsiirola]

List of Open Pyomo Performance Enhancement Ideas

This issue compiles individual ideas for improving Pyomo's overall performance. Each idea is tied to a (closed, but unresolved) issue where discussion specific to that idea is archived. The intent is that when an idea will receive active developer attention, that issue will be re-opened.

Expressions

• Re-evaluate the need for NPV expression objects #1202: Re-evaluate the need for NPV expression objects
• Define custom expression objects to short-circuit common checks #1184: Define custom expression objects to short-circuit common checks
• Using Python macros to accelerate Pyomo expression kernels #1161: Using Python macros to accelerate Pyomo expression kernels
• ▶️ Performance suggestion: templates #1153: Expression templates
• [PEP] Revisit design of method used to create the canonical repn #1073: Revisit design of method used to create the canonical repn
• Reconsider performance of expressions with many constants #324: Reconsider performance of expressions with many constants

Solver interfaces

• ▶️ with cplex, solver_io='python' is much slower than executable=(solver_path)  #1169: with cplex, solver_io='python' is much slower than executable=(solver_path)
• Fast(er) direct solver interfaces #1160: Fast(er) direct solver interfaces
• Use C/C++ extensions for file writers #1158: Use C/C++ extensions for file writers
• Performance Suggestion: "automatic" persistent solvers #1154: "automatic" persistent solvers
• Improve the NL writer #1151: Improve the NL writer
• Huge memory consumption while generating model file #251: Huge memory consumption while generating model file
• Enable creating of NL files with binary data #177: Enable creating of NL files with binary data
• [PEP] Redesign of Pyomo Solvers #1030: Redesign of Pyomo Solvers
• Shift to using unsorted problem files with Python 3.6 and later #677: Shift to using unsorted problem files with Python 3.6 and later
• Fast serialization of solutions to/from disk #106: Fast serialization of solutions to/from disk

Modeling components

• Performance Suggestion: Add "compiled" model support #1203: Add "compiled" model support
• Implement Parallel Blocks #1192: Implement Parallel Blocks
• Performance suggestion: Investigate "array-like" components #1165: Investigate "array-like" components
• Matrix/Vector Representations #1159: Matrix/Vector Representations

Packaging and distribution

• ▶️ Pyomo installation with Cython #1178: Pyomo installation with Cython

Other

• Profile Pyomo.DAE and identify bottlenecks #1157: Profile Pyomo.DAE and identify bottlenecks
• Testing Performance #1156: Testing Performance

[bernalde]

Hi, I just wanted to make you aware of this technical report recently published by independent researchers https://www.mii.lt/files/doc/lt/doktorantura/ataskaitine_konferencija/n009_ma_2020_jusevicius.pdf
Contrary to what was shown in the JuMP paper, Pyomo is not trailing that badly behind other AMLs. They have a Github repository where they used the GAMS Model library to benchmark AMPL, GAMS, AIMMS, JuMP and Pyomo https://github.com/vaidasj/alg-mod-rev

[carldlaird]

Thanks, @bernalde.

[jsiirola]

This issue has been superseded by the corresponding wiki page: https://github.com/Pyomo/pyomo/wiki/Performance-enhancement-ideas