Efficient Pattern Detection Over a Distributed Framework

Ahmed Khan Leghari, Martin Wolf, Yongluan Zhou

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

In recent past, work has been done to parallelize pattern detection queries over event stream, by partitioning the event stream on certain keys or attributes. In such partitioning schemes the degree of parallelization totally relies on the available partition keys. A limited number of partitioning keys, or unavailability of such partitioning attributes noticeably affect the distribution of data among multiple nodes, and is a reason of potential data skew and improper resource utilization. Moreover, majority of the past implementations of complex event detection are based on a single machine, hence, they are immune to potential data skew that could be seen in a real distributed environment. In this study, we propose an event stream partitioning scheme that without considering any key attributes partitions the stream over time-windows. This scheme efficiently distributes the event stream partitions across network, and detects pattern sequences in distributed fashion. Our scheme also provides an effective means to minimize potential data skew and handles a substantial number of pattern queries across network.
Original languageEnglish
Title of host publicationEnabling Real-Time Business Intelligence : International Workshops, BIRTE 2013, Riva del Garda, Italy, August 26, 2013, and BIRTE 2014, Hangzhou, China, September 1, 2014, Revised Selected Papers
EditorsMalu Castellanos, Umeshwar Dayal, Torben Bach Pedersen, Nesime Tatbul
PublisherSpringer
Publication date2015
Pages133-149
ISBN (Print)978-3-662-46838-8
ISBN (Electronic)978-3-662-46839-5
DOIs
Publication statusPublished - 2015
Event8th International Workshop on Business Intelligence for the Real-Time Enterprise - Hangzhou, China
Duration: 1. Sep 2014 → …

Workshop

Workshop8th International Workshop on Business Intelligence for the Real-Time Enterprise
CountryChina
CityHangzhou
Period01/09/2014 → …
SeriesLecture Notes in Business Information Processing
Volume206
ISSN1865-1348

Cite this

Leghari, A. K., Wolf, M., & Zhou, Y. (2015). Efficient Pattern Detection Over a Distributed Framework. In M. Castellanos, U. Dayal, T. B. Pedersen, & N. Tatbul (Eds.), Enabling Real-Time Business Intelligence: International Workshops, BIRTE 2013, Riva del Garda, Italy, August 26, 2013, and BIRTE 2014, Hangzhou, China, September 1, 2014, Revised Selected Papers (pp. 133-149). Springer. Lecture Notes in Business Information Processing, Vol.. 206 https://doi.org/10.1007/978-3-662-46839-5_9
Leghari, Ahmed Khan ; Wolf, Martin ; Zhou, Yongluan. / Efficient Pattern Detection Over a Distributed Framework. Enabling Real-Time Business Intelligence: International Workshops, BIRTE 2013, Riva del Garda, Italy, August 26, 2013, and BIRTE 2014, Hangzhou, China, September 1, 2014, Revised Selected Papers. editor / Malu Castellanos ; Umeshwar Dayal ; Torben Bach Pedersen ; Nesime Tatbul. Springer, 2015. pp. 133-149 (Lecture Notes in Business Information Processing, Vol. 206).
@inproceedings{274ba2a9e0134cd68c2516f333d9dfc6,
title = "Efficient Pattern Detection Over a Distributed Framework",
abstract = "In recent past, work has been done to parallelize pattern detection queries over event stream, by partitioning the event stream on certain keys or attributes. In such partitioning schemes the degree of parallelization totally relies on the available partition keys. A limited number of partitioning keys, or unavailability of such partitioning attributes noticeably affect the distribution of data among multiple nodes, and is a reason of potential data skew and improper resource utilization. Moreover, majority of the past implementations of complex event detection are based on a single machine, hence, they are immune to potential data skew that could be seen in a real distributed environment. In this study, we propose an event stream partitioning scheme that without considering any key attributes partitions the stream over time-windows. This scheme efficiently distributes the event stream partitions across network, and detects pattern sequences in distributed fashion. Our scheme also provides an effective means to minimize potential data skew and handles a substantial number of pattern queries across network.",
author = "Leghari, {Ahmed Khan} and Martin Wolf and Yongluan Zhou",
year = "2015",
doi = "10.1007/978-3-662-46839-5_9",
language = "English",
isbn = "978-3-662-46838-8",
pages = "133--149",
editor = "Malu Castellanos and Umeshwar Dayal and Pedersen, {Torben Bach} and Nesime Tatbul",
booktitle = "Enabling Real-Time Business Intelligence",
publisher = "Springer",
address = "Germany",

}

Leghari, AK, Wolf, M & Zhou, Y 2015, Efficient Pattern Detection Over a Distributed Framework. in M Castellanos, U Dayal, TB Pedersen & N Tatbul (eds), Enabling Real-Time Business Intelligence: International Workshops, BIRTE 2013, Riva del Garda, Italy, August 26, 2013, and BIRTE 2014, Hangzhou, China, September 1, 2014, Revised Selected Papers. Springer, Lecture Notes in Business Information Processing, vol. 206, pp. 133-149, 8th International Workshop on Business Intelligence for the Real-Time Enterprise, Hangzhou, China, 01/09/2014. https://doi.org/10.1007/978-3-662-46839-5_9

Efficient Pattern Detection Over a Distributed Framework. / Leghari, Ahmed Khan; Wolf, Martin; Zhou, Yongluan.

Enabling Real-Time Business Intelligence: International Workshops, BIRTE 2013, Riva del Garda, Italy, August 26, 2013, and BIRTE 2014, Hangzhou, China, September 1, 2014, Revised Selected Papers. ed. / Malu Castellanos; Umeshwar Dayal; Torben Bach Pedersen; Nesime Tatbul. Springer, 2015. p. 133-149 (Lecture Notes in Business Information Processing, Vol. 206).

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

TY - GEN

T1 - Efficient Pattern Detection Over a Distributed Framework

AU - Leghari, Ahmed Khan

AU - Wolf, Martin

AU - Zhou, Yongluan

PY - 2015

Y1 - 2015

N2 - In recent past, work has been done to parallelize pattern detection queries over event stream, by partitioning the event stream on certain keys or attributes. In such partitioning schemes the degree of parallelization totally relies on the available partition keys. A limited number of partitioning keys, or unavailability of such partitioning attributes noticeably affect the distribution of data among multiple nodes, and is a reason of potential data skew and improper resource utilization. Moreover, majority of the past implementations of complex event detection are based on a single machine, hence, they are immune to potential data skew that could be seen in a real distributed environment. In this study, we propose an event stream partitioning scheme that without considering any key attributes partitions the stream over time-windows. This scheme efficiently distributes the event stream partitions across network, and detects pattern sequences in distributed fashion. Our scheme also provides an effective means to minimize potential data skew and handles a substantial number of pattern queries across network.

AB - In recent past, work has been done to parallelize pattern detection queries over event stream, by partitioning the event stream on certain keys or attributes. In such partitioning schemes the degree of parallelization totally relies on the available partition keys. A limited number of partitioning keys, or unavailability of such partitioning attributes noticeably affect the distribution of data among multiple nodes, and is a reason of potential data skew and improper resource utilization. Moreover, majority of the past implementations of complex event detection are based on a single machine, hence, they are immune to potential data skew that could be seen in a real distributed environment. In this study, we propose an event stream partitioning scheme that without considering any key attributes partitions the stream over time-windows. This scheme efficiently distributes the event stream partitions across network, and detects pattern sequences in distributed fashion. Our scheme also provides an effective means to minimize potential data skew and handles a substantial number of pattern queries across network.

U2 - 10.1007/978-3-662-46839-5_9

DO - 10.1007/978-3-662-46839-5_9

M3 - Article in proceedings

SN - 978-3-662-46838-8

SP - 133

EP - 149

BT - Enabling Real-Time Business Intelligence

A2 - Castellanos, Malu

A2 - Dayal, Umeshwar

A2 - Pedersen, Torben Bach

A2 - Tatbul, Nesime

PB - Springer

ER -

Leghari AK, Wolf M, Zhou Y. Efficient Pattern Detection Over a Distributed Framework. In Castellanos M, Dayal U, Pedersen TB, Tatbul N, editors, Enabling Real-Time Business Intelligence: International Workshops, BIRTE 2013, Riva del Garda, Italy, August 26, 2013, and BIRTE 2014, Hangzhou, China, September 1, 2014, Revised Selected Papers. Springer. 2015. p. 133-149. (Lecture Notes in Business Information Processing, Vol. 206). https://doi.org/10.1007/978-3-662-46839-5_9