Integrating Fault-tolerance and Elasticity in a Distributed Data Stream Processing System

Kasper Grud Skat Madsen, Philip Pontoppidan Thyssen, Yongluan Zhou

Publikation: Kapitel i bog/rapport/konference-proceedingKonferencebidrag i proceedingsForskningpeer review

Abstrakt

Recently there has been an increasing interest in building distributed platforms for processing of fast data streams. In this demonstration, we highlight the need for elasticity in distributed data stream processing systems and present Enorm, a data stream processing platform with focus on elasticity, i.e. the ability to dynamically scale resource usage according to the runtime workload fluctuations. In order to achieve dynamic scaling with minimal overhead and latency, we use an integrated approach for both fault-tolerance and elasticity. The idea is that both fault-tolerance and elasticity essentially require replicating or migrating computation states among different nodes. Integrating and sharing the state management operations between the two modules can not only provide abundant opportunities to reduce the system's runtime overhead but also simplify the system's architecture.
OriginalsprogEngelsk
TitelProceedings of the 26th International Conference on Scientific and Statistical Database Management
Antal sider4
ForlagAssociation for Computing Machinery
Publikationsdato2014
Artikelnummer48
ISBN (Trykt)978-1-4503-2722-0
DOI
StatusUdgivet - 2014
Begivenhed26th International Conference on Scientific and Statistical Database Management - Aalborg, Danmark
Varighed: 30. jun. 20142. jul. 2014

Konference

Konference26th International Conference on Scientific and Statistical Database Management
LandDanmark
ByAalborg
Periode30/06/201402/07/2014
SponsorDanish Otto Monsted Foundation, TARGIT

Bibliografisk note

SSDBM '14 Proceedings of the 26th International Conference on Scientific and Statistical Database Management

Fingeraftryk Dyk ned i forskningsemnerne om 'Integrating Fault-tolerance and Elasticity in a Distributed Data Stream Processing System'. Sammen danner de et unikt fingeraftryk.

Citationsformater