Skip to content

Large language models for optimization: A survey

Modeling

ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling

⭐️⭐️⭐️⭐️⭐️

This paper focusing on training open-source LLMs for optimization modeling and developing solver codes. They develop the OR-INSTRUCT, a semiautomated data synthesis framework. The also introduce the IndustryOR benchmark, which is a benchmark for testing LLMs on solving optimization problems.

LLMs as Operators

LLMs and EA

When Large Language Model Meets Optimization

  • paper
  • Survey paper, LLMs, Evolutionary algorithms

This paper surveys the combination of LLMs and optimization algorithms, mainly focusing on evolutionary algorithms. The authors categorize the research into the following two main directions:

  1. LLMs assist optimization algorithms
  2. LLMS as the Black-box Optimization Search Model
    • Guided Search Operators
    • Heuristic Search Operators
    • Multi-objective Optimization
  3. LLMs as the Generator of Optimization Algorithms
    • Option Generation with the Cognitive of LLMs
    • Algorithm Generation with Chain-of-Thought
  4. Optimization algorithms optimize LLMs
  5. Optimization model tuning
    • Multi-task learning optimization
    • Based on structural pruning
  6. Self-tuning optimization
  7. prompt optimization
    • Continuous prompt optimization
    • Discrete prompt optimization
    • Black-box optimization prompt tuning
  8. Optimization network architecture search

Finally, they introduce the applications of LLMs-based optimization in various fields.

Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

  • paper
  • Survey paper, LLMs, Evolutionary algorithms

This paper provides a survey of the combination of EAs and LLMs, focusing on the following three main aspects:

  • LLM-enhanced EA
  • LLMs as search operators
  • Algorithm generation
  • Others
  • EA-enhanced LLM
  • Prompt engineering
  • LLM architecture search
  • Others
  • Applications
  • Code generation
  • Software engineering
  • Neural architecture search
  • Other generative tasks

ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling