IoT Supercharged: Complex event processing for MQTT with Eclipse technologies

November 4, 2015    

We present a new technology that bring complex event processing (CEP) to the world of Eclipse.

As today’s IoT applications are getting more and more complex, developers often find themselves having to find scalable solutions to processing a large volume of messages originating from many connected devices. Luckily, Eclipse already supports several useful technologies built on MQTT foundations that help with getting information from devices to the cloud reliably. However, even with the resources of the cloud, it remains a tough challenge to extract important information from these messages in a timely fashion on a system-wide scale, e.g. to detect if something important has happened in a physical environment monitored by smart devices.

To address this challenge, in this talk, we present a new technology that bring complex event processing (CEP) to the world of Eclipse. CEP is well-known technique in business systems to capture a combination of data and timing conditions and efficiently recognize corresponding events over data streams. We introduce Eclipse’s new CEP engine as an extension to VIATRA, a new high performance Eclipse model transformation framework.

VIATRA-CEP is an event processing platform over live models, a new design paradigm that promotes the use of the Eclipse Modeling Framework (EMF) to represent runtime data structures. This might seem like an odd choice at first, but we’ll show how this way, developers can make really good use of the powerful Eclipse Modeling toolset (e.g. for visualization) while developing and debugging their IoT applications, yet not lose performance compared to standard approaches.

The highlights of the talk are:

  • Building a simple integrated system for MQTT event processing: We highlight how easy it is to assemble IoT applications using Eclipse technologies such as Mosquito, Paho, runtime EMF models and VIATRA-CEP.

  • Capturing complex events in an Xtext-based DSL: We highlight how our approach is capable of capturing simple as well as complex event definitions using VEPL, a DSL powered by Eclipse Xtext.

  • Pushing event processing results to a web dashboard via MQTT: We show some illustrative examples how complex event processing results can be visualized using modern web technologies, using MQTT as a transport.

  • Scalability: Finally, we demonstrate how live model-based processing can match the scalability of standard approaches, and give some tips on how to make such applications scale really well.

The work presented is developed in collaboration between IncQuery Labs Ltd., the Fault Tolerant Systems Research Group of the Budapest University Technology and Economics (BUTE). BUTE’s participation is partially funded by the MONDO EU FP7 project.