Draft:General Optimal control Problem Solver

General Optimal control Problems Solver (GOPS) is an open-source reinforcement learning (RL) package that aims to address optimal control problems in industrial fields. GOPS is developed by iDLab (Intelligent Driving Laboratory) at Tsinghua University. It is built with a modular structure, enabling the creation of controllers for diverse industrial tasks.

Overview
Addressing optimal control problems is essential for meeting the basic requirements of industrial control tasks. Traditional approaches such as model predictive control often encounter significant computational burdens during real-time execution. GOPS is developed for building real-time controllers in industrial applications using RL techniques. GOPS has a modular architecture, which provides flexibility for further development, catering to the diverse needs of industrial control tasks. GOPS includes a conversion tool that enables integration with Matlab/Simulink, facilitating environment construction, controller design, and performance validation. GOPS also incorporate both serial and parallel trainers with embedded buffers and samplers to tackle large-scale control problems. Moreover, GOPS offers a range of common approximate functions for policy and value functions, including polynomial, multilayer perceptron, and convolutional neural network models.

Features
GOPS presents a set of features specifically designed for industrial control applications:


 * Modular Configuration: GOPS is built with a modular structure, allowing for customization and development of environments and algorithms.
 * Diverse Training Modes: GOPS supports different training modes, including serial and parallel setups, on-policy and off-policy approaches, as well as model-free and model-based algorithms.
 * Compatibility with Matlab/Simulink: GOPS provides a conversion tool for Matlab/Simulink, which converts Simulink models into GOPS-compatible environments and sends learned policies back to Simulink for further integration and evaluation.

Applications
Applications of GOPS in industrial control scenarios include: coupled velocity and energy management optimization, travel pattern analysis and demand prediction, design of reward functions in vehicle control, improving freeway merging efficiency, vehicle speed control strategies, multi-agent RL for platoon following, origin-destination ride-hailing demand prediction, accelerating model predictive path integral, drill boom hole-seeking control, etc.

Documentation and Usage
The GOPS package is available on GitHub at Intelligent-Driving-Laboratory/GOPS, where users can access the source code and contribute to its development. Further details, including installation instructions, usage guidelines, and examples, are provided in the GOPS documentation.