Optimizing Distillation Column Internals Design Using Reinforcement Learning Algorithms for Hybrid Discrete–Continuous Action Spaces

Introduction: Column internals design is an integral part of chemical engineering applications. Traditionally reliant on computational/time-intensive optimization, this process is often limited in terms of efficiency and adaptability. With the recent emergence of A.I. and reinforcement learning, integrating these tools into the design process offers opportunities to further efficiency and optimization. The main challenge faced in this integration process is the navigation of complex, hybrid action spaces that contain, or combine, both continuous and discrete variables. Methods: By using custom reinforcement learning environments integrated with Aspen Plus, a digital twin framework was developed, allowing a machine learning agent to interact with process simulations. Two reinforcement learning algorithms were implemented, a hybrid Soft Actor–Critic and Deep Q-Network approach, which allocates continuous actions to the SAC algorithm and the discrete actions to the DQN algorithm, and a more unified Parametrized Deep Q-Network approach, which integrates discrete–continuous actions into one architecture. In both cases, the reward function is based on the percent approach to flooding at each section of the column, providing insight into the hydraulic stability of the column. While this is the metric chosen for current studies, the framework can be extended to include others. Results: Our results indicate that both reinforcement learning strategies navigated the hybrid action space, generated hydraulically feasible designs, and adapted to different column configurations. Conclusions: This research indicates that reinforcement learning is a plausible option for optimizing distillation internals design. The reinforcement learning strategies develop a pathway for scalable, multi-objective optimization in process design.