Cracking the Code of Metabolic Regulation in Biology using Maximum Entropy/Caliber and Reinforcement Learning.

Experimental measurement or computational inference/prediction of the enzyme regulation needed in a metabolic pathway is hard problem. Consequently, regulation is known only for well-studied reactions of central metabolism in a few organisms. In this study, we use statistical thermodynamics and metabolic control theory as a theoretical framework to determine the enzyme activities that are needed to control metabolite concentrations such that they are consistent with experimentally measured values. A reinforcement learning approach is utilized to learn optimal regulation policies that match physiological levels of metabolites while maximizing the entropy production rate and minimizing the work to maintain a steady state. The learning takes a minimal amount of time, and efficient regulation schemes were learned that agree surprisingly well with known regulation. The learning is facilitated by a new approach in which steady state solutions are obtained by convex optimization based on maximum entropy rather than ODE solvers, making the time to solution seconds rather than days. The optimization is based on the Marcelin-De Donder formulation of mass action kinetics, from which rate constants are inferred. Consequently, a full ODE-based, mass action simulation with rate parameters and post-translational regulation is obtained. We demonstrate the process on three pathways in the central metabolism E. coli (gluconeogenesis, glycolysis-TCA, Pentose Phosphate-TCA) that each require different regulation schemes.