Learning to schedule multi-NUMA virtual machines via reinforcement learning

作者：Sheng, Junjie; Hu, Yiqiu; Zhou, Wenli; Zhu, Lei; Jin, Bo; Wang, Jun; Wang, Xiangfeng^*

来源：Pattern Recognition, 2022, 121: 108254.

DOI：10.1016/j.patcog.2021.108254

• Dynamic virtual machine scheduling
• Multi-NUMA
• Reinforcement learning
• Cloud computing

With the rapid development of cloud computing, the importance of dynamic virtual machine scheduling is increasing. Existing works formulate the VM scheduling as a bin-packing problem and design greedy methods to solve it. However, cloud service providers widely adopt multi-NUMA architecture servers in recent years, and existing methods do not consider the architecture. This paper formulates the multiNUMA VM scheduling into a novel structured combinatorial optimization and transforms it into a reinforcement learning problem. We propose a reinforcement learning algorithm called SchedRL with a delta reward scheme and an episodic guided sampling strategy to solve the problem efficiently. Evaluating on a public dataset of Azure under two different scenarios, our SchedRL outperforms FirstFit and BestFit on the fulfill number and allocation rate.