香港科技大学：Information-Agnostic Flow Scheduling for Commodity Data Centers

Ing Lab 香港科技大學 THE HONG KONG UNIVERSITY OF HKUST SCIENCE AND TECHNOLOGY Information-Agnostic Flow Scheduling for Commodity Data Centers Wei bai with Li Chen, Kai chen Dongsu Han Chen Tian hao Wang SING Group HKUsT SJTU June 1st, 2015

Wei Bai with Li Chen, Kai Chen, Dongsu Han, Chen Tian, Hao Wang SING Group @ HKUST Information-Agnostic Flow Scheduling for Commodity Data Centers SJTU, June 1st, 2015 1

Data Centers

Data Centers 2

This talk is about the transport inside the data center

This talk is about the transport inside the data center 3

Data Center Networks High bandwidth Low latency Commodity switches Single administrative domain

Data Center Networks 4 • High bandwidth • Low latency • Commodity switches • Single administrative domain

Data Center transport Cloud applications Desire low latency for short messages Goal: Minimize flow completion time(FCT) Many flow scheduling proposals search ae SQL mEmCACHED

Data Center Transport • Cloud applications – Desire low latency for short messages • Goal: Minimize flow completion time (FCT) – Many flow scheduling proposals… 5

The state-of-the-art Solutions PDQ [SIGCOMM,121 fAbric[SIGCOMM'13 PASE [ SIGCOMM'141 All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First(SJF) with customized network elements

The State-of-the-art Solutions • PDQ [SIGCOMM’12] • pFabric [SIGCOMM’13] • PASE [SIGCOMM’14] • … All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First (SJF) with customized network elements 6

The state-of-the-art Solutions PDQ [SIGCOMM,121 fAbric[SIGCOMM'13 PASE [ SIGCOMM'141 All assume prior knowledge of ftow size informatiorto approximate ideal preemptive Shortest Job First(SJF) with customized network elements Not feasible for some applications

The State-of-the-art Solutions • PDQ [SIGCOMM’12] • pFabric [SIGCOMM’13] • PASE [SIGCOMM’14] • … All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First (SJF) with customized network elements Not feasible for some applications 7

The state-of-the-art Solutions PDQ [SIGCOMM,121 fAbric[SIGCOMM'13 PASE [ SIGCOMM'141 All assume prior knowledge of ftow size imformrationtto approximate ideal preemptive Shortest Job First(SJF) with customized network elements Hard to deploy in practice

The State-of-the-art Solutions • PDQ [SIGCOMM’12] • pFabric [SIGCOMM’13] • PASE [SIGCOMM’14] • … All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First (SJF) with customized network elements Hard to deploy in practice 8

Question Without prior knowledge of flow size information, how to minimize fct in commodity data centers?

Question Without prior knowledge of flow size information, how to minimize FCT in commodity data centers? 9

Design goal 1 Without prior knowledge of flow size information, how to minimize fct in commodity data centers? Information-agnostic: not assume a priori knowledge of flow size information available from the applications

Design Goal 1 Without prior knowledge of flow size information, how to minimize FCT in commodity data centers? Information-agnostic: not assume a priori knowledge of flow size information available from the applications 10