Reinforcement Learning for Elasticity Control in Containerized Cloud Applications: A Model-Based Approach with Elastic Docker Swarm - Naimisha Shashikantbhai Trivedi (International Journal of Intelligent Systems and Applications in Engineering, 2024)
Software containers are becoming increasingly popular for managing and executing distributed applications on cloud computing resources. By leveraging the horizontal and vertical elasticity of containers "on the fly," workload fluctuations can be accommodated. The majority of current control systems do not consider horizontal and vertical scaling to be interconnected. In this article, we provide Reinforcement Learning (RL) strategies for controlling the vertical and horizontal elasticity of container-based systems to enhance their adaptability to various workloads. Although RL is an interesting technique, it may have a long learning period if nothing is known about the system beforehand. To accelerate learning and discover more effective adaptation strategies, our proposed reinforcement learning approaches—Q-learning, Dyna-Q, and Model-based methods—leverage varying degrees of knowledge about system dynamics. The recommended policies are incorporated into Elastic Docker Swarm, an add-on for the container orchestration platform Docker Swarm. Through prototype-based experiments and simulations, we demonstrate the effectiveness and adaptability of model-based RL techniques.