<span class="var-sub_title">A Reference Architecture for Datacenter Scheduling: Design, Validation, and Experiments</span> SC18 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

A Reference Architecture for Datacenter Scheduling: Design, Validation, and Experiments


Authors: Georgios Andreadis (Delft University of Technology, Vrije University Amsterdam), Laurens Versluis (Vrije University Amsterdam), Fabian Mastenbroek (Delft University of Technology), Alexandru Iosup (Vrije University Amsterdam, Delft University of Technology)

Abstract: Datacenters act as cloud-infrastructure to stakeholders across industry, government, and academia. To meet growing demand yet operate efficiently, datacenter operators employ increasingly more sophisticated scheduling systems, mechanisms, and policies. Although many scheduling techniques already exist, relatively little research has gone into the abstraction of the scheduling process itself, hampering design, tuning, and comparison of existing techniques. In this work, we propose a reference architecture for datacenter schedulers. The architecture follows five design principles: components with clearly distinct responsibilities, grouping of related components where possible, separation of mechanism from policy, scheduling as complex workflow, and hierarchical multi-scheduler structure. To demonstrate the validity of the reference architecture, we map to it state-of-the-art datacenter schedulers. We find scheduler-stages are commonly underspecified in peer-reviewed publications. Through trace-based simulation and real-world experiments, we show underspecification of scheduler-stages can lead to significant variations in performance.


Presentation: file


Back to Technical Papers Archive Listing